Commit Graph

2807 Commits

Author SHA1 Message Date
Yueh-Hsuan Chiang 90ee85f8e1 Improve listener_test to avoid possible false alarm
Summary:
Improve listener_test to avoid possible false alarm

Test Plan:
./listener_test
2014-11-24 18:28:06 -08:00
Lei Jin 2946e37a08 remove unreliable test in db/cuckoo_table_db_test.cc
Summary:
This compaction trigger does not seem to test any thing specific to
cuckoo table. Remove it.

Test Plan: make all check

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29523
2014-11-24 15:18:09 -08:00
Lei Jin 9c7ca65d21 free builders in VersionSet::DumpManifest
Summary:
Reported by bootcamper
This causes ldb tool to fail the assertion in ~ColumnFamilyData()

Test Plan:
./ldb --db=/tmp/test_db1 --create_if_missing put a1 b1
./ldb manifest_dump --path=/tmp/test_db1/MANIFEST-000001

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29517
2014-11-24 15:03:08 -08:00
Igor Canadi 7530c75ab0 Merge pull request #413 from saghmrossi/master
first rdb commit
2014-11-24 16:54:44 -05:00
Igor Canadi d699d70343 Make RocksDB compile without gflags
Summary: We want to make sure people without gflags can compile RocksDB.

Test Plan: remove gflags, make all

Reviewers: sdong, rven, yhchiang, ljin

Reviewed By: ljin

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29469
2014-11-24 15:53:23 -05:00
Venkatesh Radhakrishnan 3257221499 Fixes valgrind error in GetSnapshotLink. Free checkpoint now.
Summary: Free checkpoint after its directory is removed.

Test Plan: Run valgrind with GetSnapshotLink.

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29493
2014-11-24 10:20:50 -08:00
Igor Canadi ada3d78734 Merge pull request #415 from fyrz/RocksJava-Makefile
[RocksJava] Makefile correction
2014-11-22 22:53:01 -05:00
Yueh-Hsuan Chiang 569853ed10 Fix leak when create_missing_column_families=true on ThreadStatus
Summary:
An entry of ConstantColumnFamilyInfo is created when:
1. DB::Open
2. CreateColumnFamily.

However, there are cases that DB::Open could also call CreateColumnFamily
when create_missing_column_families=true.  As a result, it will create
duplicate ConstantColumnFamilyInfo and one of them would be leaked.

Test Plan: ./deletefile_test

Reviewers: igor, sdong, ljin

Reviewed By: ljin

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29307
2014-11-22 00:04:41 -08:00
fyrz c4b65f70fb [RocksJava] Makefile correction
Prevent non exist error message while executing
clean twice.
2014-11-21 23:49:31 +01:00
Shaohua Li 1410180167 Make arena use hugepage if possible
Summary:
arena doesn't use huge page by default. This change will make it happen
if possible. A new paramerter is added for Arena(). If it's set, Arena
will use huge page always. If huge page allocation fails, Arena
allocation will fallback to malloc().

Test Plan:
Change util/arena_test to support huge page allocation.
Run below tests:
1. normal regression test:
  make check
2. Check if huge page allocation works
  echo 50 > /proc/sys/vm/nr_hugepages
  make check

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D28647
2014-11-21 14:11:40 -08:00
sdong 3a40c427b9 Fix db_bench on CLANG mode
Summary: "build all" breaks in Clang mode with db_bench. Fix it.

Test Plan: USE_CLANG=1 make all

Reviewers: ljin, rven, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D29379
2014-11-21 11:30:22 -08:00
Yueh-Hsuan Chiang 9222a2d024 Fixed iOS build caused by GetThreadList feature.
Summary:
Fixed iOS build caused by GetThreadList feature.
2014-11-21 11:00:42 -08:00
Yueh-Hsuan Chiang aa31fc5068 Improve listener_test by ensuring flushes are completed before assert.
Summary: Improve listener_test by ensuring flushes are completed before assert.

Test Plan: listener_test

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29319
2014-11-21 10:22:05 -08:00
Igor Canadi 7ec71f101c Provide default implementation of LinkFile, don't break the build
Summary: By providing default implementation of LinkFile, we don't break other implementations of Env.

Test Plan: none

Reviewers: rven, dhruba

Reviewed By: dhruba

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29355
2014-11-21 11:05:48 -05:00
Igor Canadi cd278584c9 Clean up StringSplit
Summary: stringSplit is not how we name our functions. Also, we had two StringSplit's in the codebase

Test Plan: make check

Reviewers: yhchiang, dhruba

Reviewed By: dhruba

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29361
2014-11-21 11:05:28 -05:00
Igor Canadi d84069995c Fix mac compile 2014-11-21 09:42:45 -05:00
Yueh-Hsuan Chiang 4f882924dd Merge pull request #404 from fyrz/RocksJava-Backup-Restore-3.8
[RocksJava] Update BackupableDB and RestoreBackupableDB to 3.8.0.
2014-11-21 00:07:13 -08:00
Yueh-Hsuan Chiang 4b63fcbff3 Add enable_thread_tracking to DBOptions
Summary:
Add enable_thread_tracking to DBOptions to allow
tracking thread status related to the DB.  Default is off.

Test Plan:
export ROCKSDB_TESTS=ThreadList
./db_test

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29289
2014-11-20 21:13:18 -08:00
Saghm Rossi bafce61979 first rdb commit
Summary: First commit for rdb shell

Test Plan: unit_test.js does simple assertions on most of the main functionality; will update with rest of tests

Reviewers: igor, rven, lijn, yhciang, sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28749
2014-11-20 23:33:00 -05:00
Bryan Rosario 9e285d4238 Added CompatibleOptions for compatibility with LevelDB Options
Summary: Created a CompatibleOptions object that can be used as a LevelDB Options object and then converted to a RocksDB Options object using the ConvertOptions() method.

Test Plan: Unit test included in diff.

Reviewers: ljin

Reviewed By: ljin

Subscribers: sdong, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28893
2014-11-20 19:24:39 -08:00
Yueh-Hsuan Chiang 353307758b Add IOS_CROSS_COMPILE to macro guard for GetThreadList feature. 2014-11-20 16:13:20 -08:00
Yueh-Hsuan Chiang eecdebe65b Fixed the destruction order of static variables in ThreadStatusImpl. 2014-11-20 16:02:03 -08:00
Venkatesh Radhakrishnan 004f416b77 Moved checkpoint to utilities
Summary:
Moved checkpoint to utilities.
Addressed comments by Igor, Siying, Dhruba

Test Plan: db_test/SnapshotLink

Reviewers: dhruba, igor, sdong

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29079
2014-11-20 15:54:47 -08:00
Yueh-Hsuan Chiang beabc6879c Fixed ~ThreadStatusImpl(). 2014-11-20 15:45:56 -08:00
fyrz faa8d32be0 [RocksJava] Integrated changes from D29019. 2014-11-20 23:34:07 +01:00
fyrz 3d78c7a8cf [RocksJava] Lint adjustments 2014-11-20 22:51:31 +01:00
fyrz d7529b2de9 [RocksJava] Cleanup Backupable implementations
- Correct usage of isInitialized()
- Adjusted JavaDoc
2014-11-20 22:51:30 +01:00
fyrz fa703efb28 [RocksJava] Improved BackupableDBTest
- Splitted methods to meaningful tests
- Added tests for additional functionality
- Covered missing parts
2014-11-20 22:51:30 +01:00
fyrz 24fdc47416 [RocksJava] Backupable/Restorable DB update 3.8.0
- GarbageCollectMethod() available.
- GetCorruptedBackups() available.
2014-11-20 22:51:29 +01:00
fyrz 9972f969ee [RocksJava] BackupableDBOptions alginment + 3.8
- Updated the BackupableDBOptions functionality to 3.8.0.
- Aligned Options implementation with remaining source code.
- Invented test-case.
2014-11-20 22:51:29 +01:00
Yueh-Hsuan Chiang fbc42a0933 Fixed -Werror=unused-but-set-variable in thread_status_impl
Summary:
Fixed -Werror=unused-but-set-variable in thread_status_impl
2014-11-20 13:46:04 -08:00
Yueh-Hsuan Chiang a564be715d Fix asan error in thread_status_impl.cc 2014-11-20 11:47:18 -08:00
Yueh-Hsuan Chiang 7165d18869 Fix clang compile error 2014-11-20 11:00:21 -08:00
Yueh-Hsuan Chiang d0c5f28a5c Introduce GetThreadList API
Summary:
Add GetThreadList API, which allows developer to track the
status of each process.  Currently, calling GetThreadList will
only get the list of background threads in RocksDB with their
thread-id and thread-type (priority) set.  Will add more support
on this in the later diffs.

ThreadStatus currently has the following properties:

  // An unique ID for the thread.
  const uint64_t thread_id;

  // The type of the thread, it could be ROCKSDB_HIGH_PRIORITY,
  // ROCKSDB_LOW_PRIORITY, and USER_THREAD
  const ThreadType thread_type;

  // The name of the DB instance where the thread is currently
  // involved with.  It would be set to empty string if the thread
  // does not involve in any DB operation.
  const std::string db_name;

  // The name of the column family where the thread is currently
  // It would be set to empty string if the thread does not involve
  // in any column family.
  const std::string cf_name;

  // The event that the current thread is involved.
  // It would be set to empty string if the information about event
  // is not currently available.

Test Plan:
./thread_list_test
export ROCKSDB_TESTS=GetThreadList
./db_test

Reviewers: rven, igor, sdong, ljin

Reviewed By: ljin

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D25047
2014-11-20 10:49:32 -08:00
Yueh-Hsuan Chiang 1fd1aecb39 Merge pull request #409 from fyrz/RocksJava-Make-cleanup
[RocksJava] Build improvements
2014-11-20 00:01:48 -08:00
fyrz 91c8dcefc3 [RocksJava] Strip library in publish
Currently maven publishing uses the library with debug symbols. What
leads to unnecessary big library sizes. Included strip to remove
unnecessary stuff. 40M -> 2.7M
2014-11-20 07:28:59 +01:00
fyrz e7fcaa4d92 [RocksJava] JavaDoc is executed too often
Previous to this commit too much targets got dependencies
on javadocs target.

Introduced one additional target "javalib" which resolves
that situation. JavaDoc will now be generated once while
executing a task with prefix "rocksdbjava".
2014-11-20 07:28:16 +01:00
fyrz 2cd1794e4f [RocksJava] Make cleanup - Clean Target
- Remove JNI includes on clean
- Remove target folder generated by Maven
- Remove shared object
- Remove jar
2014-11-20 07:21:30 +01:00
Lei Jin be005e17bb fix clang compilation
Summary:
as title
2014-11-18 20:42:32 -08:00
Yueh-Hsuan Chiang 5e69f19c4c Merge pull request #405 from fyrz/RocksJava-Convenient-Options
[RocksJava] Convenience methods for Options
2014-11-18 20:39:51 -08:00
Igor Canadi 55a344872d Merge pull request #408 from fyrz/Missing-include
Missing header in build on CentOS 6.5, GCC4.7
2014-11-18 17:35:34 -08:00
fyrz 9e9a83baf7 Missing header in build on CentOS
While building RocksJava the build fails on
CentOS because of the missing stdexcept header.
2014-11-18 22:21:02 +01:00
fyrz 91ccc8ebef [RocksJava] Integrated changes in D29025
Addressed review comments.
2014-11-18 21:45:12 +01:00
fyrz 5249d0db50 [RocksJava] Convenience methods for Options
RocksDB introduced in 3.7.0 convenience methods
for getting ColumnFamilyOptions and DBOptions
instances from predefined configuration structures.

There is now also a method in RocksJava to load DBOptions
as well as ColumnFamilyOptions from a predefined Properties
based configuration.
2014-11-18 21:43:20 +01:00
Lei Jin 8d3f8f9696 remove all remaining references to cfd->options()
Summary:
The very last reference happens in DBImpl::GetOptions()
I built with both DBImpl::GetOptions() and ColumnFamilyData::options() commented out

Test Plan: make all check

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29073
2014-11-18 10:20:10 -08:00
Lei Jin 1e4a45aac8 remove cfd->options() in DBImpl::NotifyOnFlushCompleted
Summary: We should not reference cfd->options() directly!

Test Plan: make release

Reviewers: sdong, rven, igor, yhchiang

Reviewed By: igor, yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29061
2014-11-18 10:19:48 -08:00
Jonah Cohen 517c28994d Options helper supports k, m, g, and t unit suffixes
Summary:
Add unit support in options helper so we can specify, e.g., 10m for
10 megabytes.

Test Plan: Updated options_test

Reviewers: sdong, igor, ljin

Reviewed By: ljin

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D28977
2014-11-17 13:47:51 -08:00
Yueh-Hsuan Chiang c46c2be8d6 Merge pull request #397 from fyrz/RocksJava-GetIntProperty
[RocksJava] GetIntProperty in RocksDB
2014-11-17 12:53:44 -08:00
fyrz 8efd4bb424 [RocksJava] Improved comments in RocksDB class
Improved comments in RocksDB getLongProperty methods,
to describe the behavior more detailed.
2014-11-17 21:32:00 +01:00
fyrz 5529c1ad1b [RocksJava] GetIntProperty in RocksDB
Expose GetIntProperty methods to RocksJava. As the integer(64-Bit)
value is no integer in Java the method is aligned with the return
type which is long.
2014-11-17 21:32:00 +01:00