Commit graph

2537 commits

Author SHA1 Message Date
Lei Jin efa2fb33b0 make LevelFileNumIterator and LevelFileIteratorState anonymous
Summary: No need to expose them in .h

Test Plan: make release

Reviewers: igor, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27645
2014-10-28 11:42:22 -07:00
Lei Jin eb357af58c unfriend ForwardIterator from VersionSet
Summary: as title

Test Plan:
make release
will run full test on all stacked diffs

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27597
2014-10-28 10:08:41 -07:00
Lei Jin f981e08139 unfriend ColumnFamilyData from VersionSet
Summary: as title

Test Plan:
make release
will run full test on all stacked diffs before committing

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27591
2014-10-28 10:04:38 -07:00
Lei Jin 834c67d77f rename FileLevel to LevelFilesBrief / unfriend CompactedDBImpl
Summary:
We have several different types of data structures for file information.
FileLevel is kinda of confusing since it only contains file range and
fd. Rename it to LevelFilesBrief to make it clear.
Unfriend CompactedDBImpl as a by product

Test Plan:
make release / make all
will run full test with all stacked diffs

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27585
2014-10-28 10:03:13 -07:00
Lei Jin a28b3c4388 unfriend UniversalCompactionPicker,LevelCompactionPicker and FIFOCompactionPicker from VersionSet
Summary: as title

Test Plan:
make release
I will run make all check for all stacked diffs before commit

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27573
2014-10-28 10:00:51 -07:00
Lei Jin 5187d896b9 unfriend Compaction and CompactionPicker from VersionSet
Summary: as title

Test Plan: running make all check

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D27549
2014-10-28 09:59:56 -07:00
Yueh-Hsuan Chiang 75d7e2c374 Merge pull request #369 from fyrz/Small-Fix
[RocksJava] Minor correction to the previous pull request merge
2014-10-27 16:42:46 -07:00
Yueh-Hsuan Chiang db52419cf0 Merge pull request #290 from vladb38/master
Added merge operators to RocksJava
2014-10-27 16:41:33 -07:00
fyrz 45e756f04a [RocksJava] Minor correction to the previous pull request merge 2014-10-28 00:07:53 +01:00
Yueh-Hsuan Chiang f94f1a97d7 Merge pull request #368 from fyrz/RocksJava-Hardening-RocksIterator
[RocksJava] - Hardening RocksIterator
2014-10-27 16:00:31 -07:00
fyrz b08c39e14f [RocksJava] RocksIterator: Assert for valid RocksDB instance & documentation 2014-10-27 23:53:27 +01:00
Igor Canadi b680033e63 Include atomic in env_test 2014-10-27 15:41:05 -07:00
fyrz 56ef2caaa5 [RocksJava] - Hardening RocksIterator
RocksIterator will sometimes Sigsegv on dispose. Mainly thats related
to dispose order. If the related RocksDB instance is freed beforehand
RocksIterator.dispose() will fail.

Within this commit there is a major change to RocksIterator. RocksIterator
will hold a private reference to the RocksDB instance which created the
RocksIterator. So even if RocksDB is freed in the same GC cycle the
RocksIterator instances will be freed prior to related RocksDB instances.

Another aspect targets the dispose logic if the RocksDB is freed previously
and already gc`ed. On dispose of a RocksIterator the dispose logic will check
if the RocksDB instance points to an initialized DB. If not the dispose logic
will not perform any further action.

The crash can be reproduced by using the related test provided within this
commit.

Related information: This relates to @adamretter`s facebook rocksdb-dev group
post about SigSegv on RocksIterator.dispose().
2014-10-27 23:20:12 +01:00
Igor Canadi c1c68bce43 remove atomic_pointer.h references 2014-10-27 15:12:20 -07:00
Igor Canadi 7c303f0e78 Include atomic 2014-10-27 15:03:45 -07:00
Yueh-Hsuan Chiang 179c23021e Merge pull request #351 from fyrz/RocksJava_Snapshot_Support
[RocksJava] Support Snapshots
2014-10-27 14:50:22 -07:00
Igor Canadi 48842ab316 Deprecate AtomicPointer
Summary: RocksDB already depends on C++11, so we might as well all the goodness that C++11 provides. This means that we don't need AtomicPointer anymore. The less things in port/, the easier it will be to port to other platforms.

Test Plan: make check + careful visual review verifying that NoBarried got memory_order_relaxed, while Acquire/Release methods got memory_order_acquire and memory_order_release

Reviewers: rven, yhchiang, ljin, sdong

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D27543
2014-10-27 14:50:21 -07:00
Igor Canadi f37048ad1d Merge pull request #367 from fyrz/FixBrokenJavaBuild
RocksJava Fix after MutableCFOptions change. (relates to f184198)
2014-10-27 14:08:02 -07:00
fyrz 679a9671fe RocksJava Fix after MutableCFOptions change. 2014-10-27 21:56:25 +01:00
Lei Jin 714c63c584 db_stress for dynamic options
Summary: Allow SetOptions() during db_stress test

Test Plan: make crash_test

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25497
2014-10-27 12:11:16 -07:00
Lei Jin f1841985e4 dynamic inplace_update options
Summary:
Make inplace_update_support and inplace_update_num_locks dynamic.
inplace_callback becomes immutable
We are almost free of references to cfd->options() in db_impl

Test Plan: unit test

Reviewers: igor, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25293
2014-10-27 12:10:13 -07:00
Vlad Balan a04929aa49 fixed conflict in java/Makefile 2014-10-27 10:27:02 -07:00
fyrz a1bae76c87 Integrated changes due to review bei ankgup87 2014-10-26 13:27:43 +01:00
fyrz b8ce526487 [RocksJava] Support Snapshots
Summary:
Snapshots integration into RocksJava. Added support for the following functionalities:

- getSnapshot
- releaseSnapshot
- ReadOptions support to set a Snapshot
- ReadOptions support to retrieve Snapshot
- SnapshotTest

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24801
2014-10-25 11:10:19 +02:00
Yueh-Hsuan Chiang bc3bc4bc2f Merge pull request #357 from fyrz/JavaTest-Fix
Fix issues introduced by latest Java additions
2014-10-24 17:08:10 -07:00
Yueh-Hsuan Chiang 42f0eaceb7 Merge pull request #354 from fyrz/RocksJava-memtables-3.6
[RocksJava] Memtables update to 3.6
2014-10-24 16:53:55 -07:00
Igor Canadi 965d9d50b8 Fix timing 2014-10-24 11:58:15 -07:00
Igor Canadi 001ce64dc7 Use chrono for timing
Summary: Since we depend on C++11, we might as well use it for timing, instead of this platform-depended code.

Test Plan: Ran autovector_test, which reports time and confirmed that output is similar to master

Reviewers: ljin, sdong, yhchiang, rven, dhruba

Reviewed By: dhruba

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D25587
2014-10-24 10:11:57 -07:00
sdong 240ed0cd7b Fix uninitialized parameter caused by D24513
Summary: D24513 introduced a bug that a variable is not initialized. It also causes valgrind issue.

Test Plan: Run tests used to fail valgrind and make sure it passes

Reviewers: yhchiang, ljin, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25569
2014-10-23 15:57:36 -07:00
Yueh-Hsuan Chiang 724fba2b39 Improve the log in Universal Compaction to include more debug information.
Summary:
Previously, the log for Universal Compaction does not include the current
number of files in case the compaction is triggered by the number of files.
This diff includes the number of files in the log.

Test Plan:
make
2014-10-23 15:43:51 -07:00
Lei Jin 720c1c056d fix erro during merge
Summary:
as title

Test Plan:
make release
2014-10-23 15:41:37 -07:00
Igor Canadi b794194ade Remove java build from travis 2014-10-23 15:37:51 -07:00
Lei Jin 122f98e0b9 dynamic max_mem_compact_level
Summary: as title

Test Plan: unit test

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25347
2014-10-23 15:37:14 -07:00
Lei Jin 1fee591e74 comments for DynamicCompactionOptions test
Summary: as title

Test Plan: n/a

Reviewers: yhchiang, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24963
2014-10-23 15:35:10 -07:00
Lei Jin 574028679b dynamic max_sequential_skip_in_iterations
Summary:
This is not a critical options. Making it dynamic so that we can remove
more reference to cfd->options()

Test Plan: unit test

Reviewers: yhchiang, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24957
2014-10-23 15:34:21 -07:00
fyrz bd4fbaee37 Fixed cross platform build after introducing Java-7 dependencies 2014-10-23 21:10:49 +02:00
fyrz 1eb545721d Fix incorrectly merged Java - Makefile 2014-10-23 21:10:49 +02:00
fyrz 9aa9668a83 [RocksJava] Memtables update to 3.6
- Adjusted HashLinkedList to 3.6.0
- Adjusted SkipList to 3.6.0
- Introduced a memtable test
2014-10-23 21:10:17 +02:00
Igor Canadi 4b1786e959 Fix SIGSEGV when declaring Arena after ScopedArenaIterator 2014-10-23 12:03:19 -07:00
Igor Canadi 05b2e60ddd Merge pull request #362 from adamretter/travis-jcheck
Added Java build and tests to Travis-CI build
2014-10-23 11:41:51 -07:00
Adam Retter 9383922cc3 Added java tests to travis build 2014-10-23 19:35:58 +01:00
Igor Canadi 90f156402c Fix CompactBetweenSnapshots 2014-10-23 11:18:33 -07:00
Igor Canadi 2717422e21 Merge pull request #361 from adamretter/fix-yosemite-build
Fix for building RocksDB Java on Mac OS X Yosemite
2014-10-23 11:10:01 -07:00
Yueh-Hsuan Chiang 3b5fe3a1f3 Correct the log message in VersionEdit
Summary:
When VersionEdit fails in kNewFile3, previously it logs "new-file2 entry".
However, it should be "new-file3 entry."

Test Plan:
make
2014-10-23 10:44:53 -07:00
Adam Retter c584d2b538 Fix for building RocksDB Java on Mac OS X Yosemite 2014-10-23 15:39:48 +01:00
sdong 2a8e5203d8 db_bench: --batch_size used for write benchmarks too
Summary: Now --bench_size is only used in multireadrandom tests, although the codes allow it to run in all write tests. I don't see a reason why we can't enable it.

Test Plan:
Run
   ./db_bench -benchmarks multirandomwrite --threads=5 -batch_size=16
and see the stats printed out in LOG to make sure batching really happened.

Reviewers: ljin, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25509
2014-10-22 18:54:12 -07:00
sdong d755e53b87 Printing number of keys in DB Stats
Summary: It is useful to print out number of keys in DB Stats

Test Plan:
./db_bench --benchmarks fillrandom --num 1000000 -threads 16 -batch_size=16

and watch the outputs in LOG files

Reviewers: MarkCallaghan, ljin, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24513
2014-10-22 18:41:33 -07:00
Lei Jin 839c376bd1 fix table_test
Summary:
SaveValue expects an internal key but I previously added to table a
user key

Test Plan:
ran the test
2014-10-22 13:53:35 -07:00
Lei Jin 0fd985f427 Avoid reloading filter on Get() if cache_index_and_filter_blocks == false
Summary:
This fixes the case that filter policy is missing in SST file, but we
open the table with filter policy on and cache_index_and_filter_blocks =
false. The current behavior is that we will try to load it every time on
Get() but fail.

Test Plan: unit test

Reviewers: yhchiang, igor, rven, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25455
2014-10-22 11:52:35 -07:00
Yueh-Hsuan Chiang e11a5e776f Improve the comment of util/thread_local.h
Summary: Improve the comment of util/thread_local.h

Test Plan: n/a

Reviewers: ljin

Reviewed By: ljin

Differential Revision: https://reviews.facebook.net/D25449
2014-10-21 17:28:31 -07:00