Commit graph

2709 commits

Author SHA1 Message Date
sdong 240ed0cd7b Fix uninitialized parameter caused by D24513
Summary: D24513 introduced a bug that a variable is not initialized. It also causes valgrind issue.

Test Plan: Run tests used to fail valgrind and make sure it passes

Reviewers: yhchiang, ljin, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25569
2014-10-23 15:57:36 -07:00
Yueh-Hsuan Chiang 724fba2b39 Improve the log in Universal Compaction to include more debug information.
Summary:
Previously, the log for Universal Compaction does not include the current
number of files in case the compaction is triggered by the number of files.
This diff includes the number of files in the log.

Test Plan:
make
2014-10-23 15:43:51 -07:00
Lei Jin 720c1c056d fix erro during merge
Summary:
as title

Test Plan:
make release
2014-10-23 15:41:37 -07:00
Igor Canadi b794194ade Remove java build from travis 2014-10-23 15:37:51 -07:00
Lei Jin 122f98e0b9 dynamic max_mem_compact_level
Summary: as title

Test Plan: unit test

Reviewers: sdong, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25347
2014-10-23 15:37:14 -07:00
Lei Jin 1fee591e74 comments for DynamicCompactionOptions test
Summary: as title

Test Plan: n/a

Reviewers: yhchiang, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24963
2014-10-23 15:35:10 -07:00
Lei Jin 574028679b dynamic max_sequential_skip_in_iterations
Summary:
This is not a critical options. Making it dynamic so that we can remove
more reference to cfd->options()

Test Plan: unit test

Reviewers: yhchiang, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24957
2014-10-23 15:34:21 -07:00
fyrz bd4fbaee37 Fixed cross platform build after introducing Java-7 dependencies 2014-10-23 21:10:49 +02:00
fyrz 1eb545721d Fix incorrectly merged Java - Makefile 2014-10-23 21:10:49 +02:00
fyrz 9aa9668a83 [RocksJava] Memtables update to 3.6
- Adjusted HashLinkedList to 3.6.0
- Adjusted SkipList to 3.6.0
- Introduced a memtable test
2014-10-23 21:10:17 +02:00
Igor Canadi 4b1786e959 Fix SIGSEGV when declaring Arena after ScopedArenaIterator 2014-10-23 12:03:19 -07:00
Igor Canadi 05b2e60ddd Merge pull request #362 from adamretter/travis-jcheck
Added Java build and tests to Travis-CI build
2014-10-23 11:41:51 -07:00
Adam Retter 9383922cc3 Added java tests to travis build 2014-10-23 19:35:58 +01:00
Igor Canadi 90f156402c Fix CompactBetweenSnapshots 2014-10-23 11:18:33 -07:00
Igor Canadi 2717422e21 Merge pull request #361 from adamretter/fix-yosemite-build
Fix for building RocksDB Java on Mac OS X Yosemite
2014-10-23 11:10:01 -07:00
Yueh-Hsuan Chiang 3b5fe3a1f3 Correct the log message in VersionEdit
Summary:
When VersionEdit fails in kNewFile3, previously it logs "new-file2 entry".
However, it should be "new-file3 entry."

Test Plan:
make
2014-10-23 10:44:53 -07:00
Adam Retter c584d2b538 Fix for building RocksDB Java on Mac OS X Yosemite 2014-10-23 15:39:48 +01:00
sdong 2a8e5203d8 db_bench: --batch_size used for write benchmarks too
Summary: Now --bench_size is only used in multireadrandom tests, although the codes allow it to run in all write tests. I don't see a reason why we can't enable it.

Test Plan:
Run
   ./db_bench -benchmarks multirandomwrite --threads=5 -batch_size=16
and see the stats printed out in LOG to make sure batching really happened.

Reviewers: ljin, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25509
2014-10-22 18:54:12 -07:00
sdong d755e53b87 Printing number of keys in DB Stats
Summary: It is useful to print out number of keys in DB Stats

Test Plan:
./db_bench --benchmarks fillrandom --num 1000000 -threads 16 -batch_size=16

and watch the outputs in LOG files

Reviewers: MarkCallaghan, ljin, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24513
2014-10-22 18:41:33 -07:00
Lei Jin 839c376bd1 fix table_test
Summary:
SaveValue expects an internal key but I previously added to table a
user key

Test Plan:
ran the test
2014-10-22 13:53:35 -07:00
Lei Jin 0fd985f427 Avoid reloading filter on Get() if cache_index_and_filter_blocks == false
Summary:
This fixes the case that filter policy is missing in SST file, but we
open the table with filter policy on and cache_index_and_filter_blocks =
false. The current behavior is that we will try to load it every time on
Get() but fail.

Test Plan: unit test

Reviewers: yhchiang, igor, rven, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25455
2014-10-22 11:52:35 -07:00
Yueh-Hsuan Chiang e11a5e776f Improve the comment of util/thread_local.h
Summary: Improve the comment of util/thread_local.h

Test Plan: n/a

Reviewers: ljin

Reviewed By: ljin

Differential Revision: https://reviews.facebook.net/D25449
2014-10-21 17:28:31 -07:00
Igor Canadi 6398e6a6a5 Fix DeleteFile() + enable deleting files oldest files in level 0
Summary:
DeleteFile() call was broken for non-default column family. This fixes it. We might need this feature for mongo.

I also introduced a possibility of deleting oldest file in level 0.

Test Plan: added unit test to deletefile_test

Reviewers: ljin, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24909
2014-10-21 11:23:06 -07:00
Yueh-Hsuan Chiang 0d3145198e Merge pull request #231 from adamretter/master
Java API for Comparator and Slice
2014-10-21 10:26:01 -07:00
Adam Retter a6fb7f312d Fix code review comments raised in https://reviews.facebook.net/D22779 2014-10-21 15:52:28 +01:00
Adam Retter c63494fb61 Tests for ComparatorOptions, Comparator and DirectComparator, and by
proxy we also exercise Slice and DirectSlice
2014-10-21 15:52:27 +01:00
Adam Retter 5e25274110 Fix code style problems identified by lint 2014-10-21 15:52:27 +01:00
Adam Retter 25641bfc9c Fix to memory dealocation when creating a slice from a byte buffer 2014-10-21 15:52:27 +01:00
Adam Retter fc12cb83f2 Add locking to comparator jni callback methods which must be thread-safe 2014-10-21 15:52:27 +01:00
Adam Retter d6fe8dacc8 Feature - Implement Java API for Comparator and Slice. Allows use of
either byte[] or DirectByteBuffer for accessing underlying data.
2014-10-21 15:52:27 +01:00
Adam Retter 700f6ec3ff Ignore IntelliJ idea project files and ignore java/out folder 2014-10-21 15:52:27 +01:00
sdong 5bfb7f5d0b db_bench: seekrandom can specify --seek_nexts to read specific keys after seek.
Summary:
Add a function as tittle.
Also use the same parameter to fillseekseq too.

Test Plan: Run seekrandom using the new parameter

Reviewers: ljin, MarkCallaghan

Reviewed By: MarkCallaghan

Subscribers: rven, igor, yhchiang, leveldb

Differential Revision: https://reviews.facebook.net/D25035
2014-10-20 11:55:33 -07:00
Lei Jin ff8f74c204 remove checking lower bound of level size
Summary:
as title

Test Plan:
make db_test
2014-10-17 21:18:51 -07:00
Lei Jin 2dd9bfe3a8 Sanitize block-based table index type and check prefix_extractor
Summary:
Respond to issue reported
https://www.facebook.com/groups/rocksdb.dev/permalink/651090261656158/
Change the Sanitize signature to take both DBOptions and CFOptions

Test Plan: unit test

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25041
2014-10-17 21:18:36 -07:00
Vlad Balan dbcfe27d6c Merge branch 'master' of https://github.com/facebook/rocksdb 2014-10-17 15:44:52 -07:00
Yueh-Hsuan Chiang 6c66918645 Speed up DB::Open() and Version creation by limiting the number of FileMetaData initialization.
Summary:
This diff speeds up DB::Open() and Version creation by limiting the number of FileMetaData initialization. The behavior of Version::UpdateAccumulatedStats() is changed as follows:

* It only initializes the first 20 uninitialized FileMetaData from file.  This guarantees the size of the latest 20 files will always be compensated when they have any deletion entries.  Previously it may initialize all FileMetaData by loading all files at DB::Open().
* In case none the first 20 files has any data entry, UpdateAccumulatedStats() will initialize the FileMetaData of the oldest file.

Test Plan: db_test

Reviewers: igor, sdong, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24255
2014-10-17 14:58:30 -07:00
Yueh-Hsuan Chiang 5db9e76644 Fix Mac compile error: C++11 forbids default arguments for lambda expressions
Summary:
Fix the following Mac compile error.
db/db_test.cc:8686:52: error: C++11 forbids default arguments for lambda expressions [-Werror,-Wlambda-extensions]
  auto gen_l0_kb = [this](int start, int size, int stride = 1) {
                                                   ^        ~
Test Plan:
db_test
2014-10-17 14:47:26 -07:00
Lei Jin f4363fb81c Fix DynamicMemtableOptions test
Summary: as title

Test Plan: make release

Reviewers: igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25029
2014-10-17 10:09:45 -07:00
Yueh-Hsuan Chiang 8f01bf80cc Merge pull request #353 from fyrz/FixBloomfilterRocksJava
[RocksJava] Filters getting disposed by System.gc before EOL
2014-10-17 10:08:38 -07:00
Igor Canadi ee80fb4b4a Total memtables size counter
Summary: Added one new counter for GetProperty

Test Plan: Not sure if needs a test case. compiles

Reviewers: sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D25023
2014-10-17 09:26:27 -07:00
Igor Canadi c12f571d31 Fix mac compile, second try 2014-10-17 12:09:45 -04:00
Igor Canadi d2e60f5cee Fix mac compile 2014-10-17 12:05:32 -04:00
Lei Jin 274dc81c92 fix build failure
Summary: missed default value during merge

Test Plan: ./db_test

Reviewers: igor, sdong, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24975
2014-10-16 17:33:09 -07:00
Lei Jin d6c8dba727 Log MutableCFOptions in SetOptions
Summary: as title

Test Plan: make release

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24903
2014-10-16 17:22:28 -07:00
Lei Jin 4d5708aa56 dynamic soft_rate_limit and hard_rate_limit
Summary: as title

Test Plan:
unit test
I am only able to build the test case for hard_rate_limit.
soft_rate_limit is essentially the same thing as hard_rate_limit

Reviewers: igor, sdong, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24759
2014-10-16 17:21:31 -07:00
Lei Jin 065a67c4f0 dynamic disable_auto_compactions
Summary: Add more tests as well

Test Plan: unit test

Reviewers: igor, sdong, yhchiang

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24747
2014-10-16 17:14:17 -07:00
Lei Jin dc50a1a593 make max_write_buffer_number dynamic
Summary: as title

Test Plan: unit test

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D24729
2014-10-16 16:57:59 -07:00
sdong 6a150c0118 ldb: support --fix_prefix_len
Summary:
ldb to support --fix_prefix_len to allow us to verify more cases.
Also fix a small issue that --bloom_bits might not be applied if --block_size is not given.

Test Plan: run ldb tool against an example DB.

Reviewers: ljin, yhchiang, rven, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24819
2014-10-16 13:42:34 -07:00
fyrz bafbc23baa Filters getting disposed by System.gc before EOL
Previous to this commit Filters passed as parameters to the
BlockTableConfig are disposed before they should be disposed.

Further Smart pointer usage was corrected.

Java holds now the smart pointer to the FilterPolicy correctly
and cares about freeing underlying c++ structures.
2014-10-15 20:59:25 +02:00
Igor Canadi ca250d71a1 Move logging out of mutex
Summary: As title

Test Plan: compiles

Reviewers: sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D24897
2014-10-15 10:56:50 -07:00