Commit Graph

82 Commits

Author SHA1 Message Date
fyrz 5340484266 Built-in comparator(s) in RocksJava
Extended Built-in comparators with ReverseBytewiseComparator.

Reverse key handling is under certain conditions essential. E.g. while
using timestamp versioned data.

As native-comparators were not available using JAVA-API. Both built-in comparators
were exposed via JNI to be set upon database creation time.
2014-09-26 10:35:12 +02:00
Igor Canadi 21ddcf6e4f Remove allow_thread_local
Summary: See https://reviews.facebook.net/D19365

Test Plan: compiles

Reviewers: sdong, yhchiang, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23907
2014-09-24 13:12:16 -07:00
Ankit Gupta 0103b4498c Merge branch 'master' of ssh://github.com/ankgup87/rocksdb 2014-09-24 11:43:57 -07:00
Ankit Gupta 1dfb7bb980 Add block based table config options 2014-09-24 11:43:35 -07:00
Ankit Gupta bfeef94d31 Add rate limiter 2014-09-19 16:11:59 -07:00
Ankit Gupta b93797abc4 Fix build 2014-09-18 22:13:52 -07:00
Yueh-Hsuan Chiang 94e43a1dfe [Java] Fixed 32-bit overflowing issue when converting jlong to size_t
Summary:
Fixed 32-bit overflowing issue when converting jlong to size_t by
capping jlong to std::numeric_limits<size_t>::max().

Test Plan:
make rocksdbjava
make jtest

Reviewers: ankgup87, ljin, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23511
2014-09-17 12:30:06 -07:00
Lei Jin dd641b2117 fix RocksDB java build
Summary: as title

Test Plan: make rocksdbjava

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23193
2014-09-10 12:00:32 -07:00
Stanislau Hlebik 45a5e3ede0 Remove path with arena==nullptr from NewInternalIterator
Summary:
Simply code by removing code path which does not use Arena
from NewInternalIterator

Test Plan:
make all check
make valgrind_check

Reviewers: sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D22395
2014-09-04 17:40:41 -07:00
Lei Jin 66f62e5c78 JNI changes corresponding to BlockBasedTableOptions migration
Summary: as title

Test Plan:
tested on my mac
make rocksdbjava
make jtest

Reviewers: sdong, igor, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21963
2014-08-25 14:22:55 -07:00
Lei Jin 384400128f move block based table related options BlockBasedTableOptions
Summary:
I will move compression related options in a separate diff since this
diff is already pretty lengthy.
I guess I will also need to change JNI accordingly :(

Test Plan: make all check

Reviewers: yhchiang, igor, sdong

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21915
2014-08-25 14:22:05 -07:00
Yueh-Hsuan Chiang 2da53b1e06 [Java] Add purgeOldBackups API
Summary:
1. Check status of CreateNewBackup. If status is not OK, then throw.
2. Add purgeOldBackups API

Test Plan:
make test
make sample

Reviewers: haobo, sdong, zzbennett, swapnilghike, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21753
2014-08-14 10:58:09 -07:00
Yueh-Hsuan Chiang 10fc6c7d25 [Java] Add compaction style to options
Summary:
Add compression type to options

make rocksdbjava
make sample

Reviewers: haobo, yhchiang, sdong, dhruba, rsumbaly, zzbennett, swapnilghike
Reviewed By: yhchiang, sdong
CC: leveldb

Differential Revision: https://reviews.facebook.net/D20463
2014-07-23 15:45:30 -07:00
Yueh-Hsuan Chiang 6e7e3e45ff [Java] Update header inclusion of utilities files
Summary:
Update header inclusion as include/utilities/*.h has been moved to
include/rocksdb/utilities/*.h

Test Plan: make rocksdbjava

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20517
2014-07-23 11:15:14 -07:00
Yueh-Hsuan Chiang d19aa25594 [Java] Add Java support for cache sharding.
Summary:
Add setCacheNumShardBits() and cacheNumShardBits() to Options.  This allows
developers to control the number of shards for the block cache.

Test Plan:
make rocksdbjava
cd java
make db_bench
./jdb_bench.sh --cache_size=1048576 --cache_numshardbits=6

Reviewers: sdong, ljin, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19347
2014-07-22 09:48:37 -07:00
Yueh-Hsuan Chiang 80a94d0517 [Java] Fixed the Java compilation error caused by PlainTableFactory API change.
Summary: Fixed the Java compilation error caused by PlainTableFactory API change.

Test Plan:
make rocksdbjava
make jdb_bench
cd java;./jdb_bench.sh

Reviewers: sdong, ljin, ankgup87, swapnilghike, zzbennett, rsumbaly, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20295
2014-07-21 09:19:28 -07:00
Ankit Gupta 0a4d930264 Caching methodId and fieldId is fine: v2 2014-07-09 09:09:08 -07:00
Ankit Gupta b6caaea9d3 Caching methodId and fieldId is fine 2014-07-09 09:06:40 -07:00
Ankit Gupta 4216ca36ae Class IDs and method IDs should not be cached 2014-07-07 21:45:02 -07:00
Ankit Gupta 97bfcd6a16 Update doc of options.cc 2014-07-07 12:26:06 -07:00
Ankit Gupta 4efca96bb3 Change enum type from int to byte 2014-07-07 12:18:55 -07:00
Ankit Gupta 54d7a2c0c5 Add compression type to options 2014-07-07 09:58:54 -07:00
Yueh-Hsuan Chiang e72b02e3c2 [Java] Add basic Java binding for rocksdb::Env.
Summary: Add basic Java binding for rocksdb::Env.

Test Plan:
make rocksdbjava
make jtest
cd java
./jdb_bench.sh --max_background_compactions=1
./jdb_bench.sh --max_background_compactions=10

Reviewers: sdong, ankgup87, haobo

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18903
2014-06-05 17:09:25 -07:00
Ankit Gupta fa1b62cabd Merge remote-tracking branch 'upstream/master' 2014-05-29 18:33:11 -07:00
Yueh-Hsuan Chiang ab3e566699 [Java] Generalize dis-own native handle and refine dispose framework.
Summary:
1. Move disOwnNativeHandle() function from RocksDB to RocksObject
to allow other RocksObject to use disOwnNativeHandle() when its
ownership of native handle has been transferred.

2. RocksObject now has an abstract implementation of dispose(),
which does the following two things.  First, it checks whether
both isOwningNativeHandle() and isInitialized() return true.
If so, it will call the protected abstract function dispose0(),
which all the subclasses of RocksObject should implement.  Second,
it sets nativeHandle_ = 0.  This redesign ensure all subclasses
of RocksObject have the same dispose behavior.

3. All subclasses of RocksObject now should implement dispose0()
instead of dispose(), and dispose0() will be called only when
isInitialized() returns true.

Test Plan:
make rocksdbjava
make jtest

Reviewers: dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett, haobo

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18801
2014-05-28 18:16:29 -07:00
Ankit Gupta d271cc5b4e Treat negative values as no limit 2014-05-21 07:54:22 -07:00
Ankit Gupta e87973cde1 Removing code from portal.h for setting handle of RestoreOptions and RestoreBackupableDB 2014-05-18 22:40:48 -07:00
Ankit Gupta b830bb937f Add newline at end of file 2014-05-13 22:34:29 -07:00
Ankit Gupta cf0b773a29 Add more options to backupable DB 2014-05-13 22:22:21 -07:00
Ankit Gupta 036e323f7a Add RestoreBackupableDB and RestoreOptions 2014-05-13 22:20:58 -07:00
Yueh-Hsuan Chiang 2082a7d745 [Java] Temporary set the number of BG threads based on the number of BG compactions.
Summary:
Before the Java binding for Env is ready, Java developers have no way to
control the number of background threads.  This diff provides a temporary
solution where RocksDB.setMaxBackgroundCompactions() will affect the
number of background threads.

Note that once Env is ready.  Changes made in this diff should be reverted.

Test Plan:
make rocksdbjava
make jtest
make jdb_bench
java/jdb_bench.sh

Reviewers: haobo, sdong

Reviewed By: sdong

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18681
2014-05-13 12:28:47 -07:00
Yueh-Hsuan Chiang d14581f936 [Java] Rename org.rocksdb.Iterator to org.rocksdb.RocksIterator.
Summary:
Renamed Iterator to RocksIterator to avoid potential confliction with
the Java built-in Iterator.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, dhruba, sdong, ankgup87

Reviewed By: sdong

CC: leveldb, rsumbaly, swapnilghike, zzbennett

Differential Revision: https://reviews.facebook.net/D18615
2014-05-12 11:02:25 -07:00
Yueh-Hsuan Chiang 61955a0dda [Java] Add RocksObject, the base class of all java objects with a c++ pointer.
Summary:
Add RocksObject, the base class of all java objects which has a c++ pointer.
While the finalizer of a RocksObject will release its c++ resource, it is
suggested to call its RocksObject.dispose() to manually release its
c++ resource.

Existing RocksDB java classes are now extending RocksObject.

Test Plan:
make rocksdbjava
make jtest
make jdb_bench

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18411
2014-05-01 01:44:46 -07:00
Ankit Gupta 65fba4b984 Merge branch 'master' of https://github.com/facebook/rocksdb 2014-04-29 07:53:29 -07:00
Yueh-Hsuan Chiang 9895465c6a [Java] Fixed compile error due to the removal of ReadOptions.prefix_seek, minor improvement on DbBenchmark.java. 2014-04-27 21:49:45 -07:00
Ankit Gupta 13b5632fdd Merge branch 'master' of https://github.com/facebook/rocksdb 2014-04-25 22:41:04 -07:00
Ankit Gupta 9e0ab427b8 Fix bug + formatting 2014-04-25 22:39:33 -07:00
Ankit Gupta 3f8b4129ef Fix formatting 2014-04-25 20:59:16 -07:00
Ankit Gupta e96e71becf Multi-get impl and formatting 2014-04-25 13:57:20 -07:00
Yueh-Hsuan Chiang 05979493d6 [Java] Enable filluniquerandom, readseq, BloomFilter, and 70+ command-line options to DbBenchmark.java
Summary:
* Add filluniquerandom
* Add readseq, implemented using iterator.
* Realize most command-line-arguments from db_bench.cc (70+).
* Some code are commented out as some of the options in Options
  not yet have Java bindings.
* Add default option to DbBenchmark.
* RocksDB will now take the ownership of all c++ raw-pointers from Options, which includes a c++ raw-pointer for Filter.

Test Plan: ./jdb_bench.sh --db=/tmp/rocksjava-bench/db --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=134217728 --max_bytes_for_level_base=1073741824 --disable_wal=0 --wal_dir=/tmp/rocksjava-bench/wal --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=filluniquerandom,readseq,readrandom --use_existing_db=0 --threads=4

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18267
2014-04-24 13:43:57 -07:00
Yueh-Hsuan Chiang d8fe006056 [Java] Add Java bindings and Java tests for 30+ rocksdb::Options.
Summary:
Add Java bindings and Java tests for 30+ rocksdb::Options.  Codes are
machine-genearted based on include/rocksdb/options.h with manual-polishment.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18273
2014-04-23 20:54:35 -07:00
Ankit Gupta 2214fd8a15 Refactor filter impl 2014-04-22 08:58:43 -07:00
Ankit Gupta 89cb481aa1 Fix doc 2014-04-22 00:09:40 -07:00
Ankit Gupta 677b0d6d3f Refactor filter impl 2014-04-22 00:04:56 -07:00
Ankit Gupta 5e797cf0dd Change filter implementation 2014-04-21 23:56:19 -07:00
Ankit Gupta cea2be20b6 Fix formatting 2014-04-21 20:27:09 -07:00
Ankit Gupta dc4b27ac48 Add bloom filters 2014-04-21 20:25:30 -07:00
Yueh-Hsuan Chiang e316af5f16 [Java] Add Java binding and Java test for ReadOptions.
Summary: Add Java binding and test for rocksdb::ReadOptions.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18129
2014-04-21 15:52:59 -07:00
Yueh-Hsuan Chiang ef8b8a8ef6 [Java] Add Java bindings for memtables and sst format.
Summary:
Add Java bindings for memtables and sst format.  Specifically,
add two abstract Java classses --- MemTableConfig and SstFormatConfig.
Each MemTable / SST implementation should has its own config class
extends MemTableConfig / SstFormatConfig respectively and pass it
to Options via setMemTableConfig / setSstConfig.

Test Plan:
make rocksdbjava
make jdb_test
make jdb_bench
java/jdb_bench.sh \
  --benchmarks=fillseq,readrandom,readwhilewriting \
  --memtablerep=hash_skiplist \
  --use_plain_table=1 \
  --key_size=20 \
  --prefix_size=12 \
  --value_size=100 \
  --cache_size=17179869184 \
  --disable_wal=0 \
  --sync=0 \

Reviewers: haobo, ankgup87, sdong

Reviewed By: haobo

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17997
2014-04-21 15:40:46 -07:00
Ankit Gupta bbdd550b66 Remove getIterator function from portal 2014-04-19 23:17:42 -07:00