Commit Graph

847 Commits

Author SHA1 Message Date
Yueh-Hsuan Chiang 7fe247080f Update HISTORY.md for RocksJava 2014-11-14 11:23:00 -08:00
Igor Canadi 25f273027b Fix iOS compile with -Wshorten-64-to-32
Summary: So iOS size_t is 32-bit, so we need to static_cast<size_t> any uint64_t :(

Test Plan: TARGET_OS=IOS make static_lib

Reviewers: dhruba, ljin, yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28743
2014-11-13 14:39:30 -05:00
fyrz d50c68e3a5 [RocksJava] JavaDoc cleanup warnings with Java8
Java8 is more restrictive than Java7 with generating
JavaDocs. This commit resolves current existing Java8
warnings.
2014-11-12 20:51:04 +01:00
fyrz 079d942ea8 [RocksJava] Code-cleanup + Java7 warnings removed 2014-11-12 20:42:25 +01:00
fyrz 9a255b95f0 [RocksJava] Sample and Default value
- RocksDB ColumnFamilySample adjusted to C++ sample.
- DefaultColumnFamily is available now as constant in RocksDB.
2014-11-12 19:49:13 +01:00
fyrz 9d2ba21361 [RocksJava] Incorporated review comments 2014-11-12 19:28:26 +01:00
fyrz fa9cfc65f3 [RocksJava] Integrated Review comments from yhchiang in D28023 2014-11-12 19:28:25 +01:00
fyrz 75010d2084 [RocksJava] ColumnFamily custom Options API extension
*********************
                   ***************************
                 ******** ************* ********
                ********   ***********   ********
               ********     *********     ********
              *************************************
              *************************************
              *************************************
               ******     ***       ***     ******
                ******    ***  ***  ***    ******
                 ******        ***        ******
                   ***************************
                      *********************
2014-11-12 19:28:24 +01:00
fyrz 0345c2156f [RocksJava] Extend Options with ColumnFamilyOptions implementation ColumnFamilyOptions implementation with tests
[RocksJava] Extended ColumnFamilyTest

Summary: Options Refactoring split part 3

Test Plan:
make rocksdbjava
make jtest

Reviewers: yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D28023
2014-11-12 19:28:23 +01:00
Igor Canadi 767777c2bd Turn on -Wshorten-64-to-32 and fix all the errors
Summary:
We need to turn on -Wshorten-64-to-32 for mobile. See D1671432 (internal phabricator) for details.

This diff turns on the warning flag and fixes all the errors. There were also some interesting errors that I might call bugs, especially in plain table. Going forward, I think it makes sense to have this flag turned on and be very very careful when converting 64-bit to 32-bit variables.

Test Plan: compiles

Reviewers: ljin, rven, yhchiang, sdong

Reviewed By: yhchiang

Subscribers: bobbaldwin, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D28689
2014-11-11 16:47:22 -05:00
fyrz fc6fcbab9e [RocksJava] Flush functionality
RocksJava now supports also flush functionality of
RocksDB.
2014-11-11 07:47:38 +01:00
fyrz 8e5547f64f [RocksJava] Makefile restructured 2014-11-11 07:47:38 +01:00
Adam Retter d904fbbb0b Addresed comments from code review https://reviews.facebook.net/D27567 2014-11-09 16:10:11 +00:00
Adam Retter eeb9cf6c42 Test for WriteBatchHandler 2014-11-09 16:10:11 +00:00
Adam Retter 8fb4751d50 Iterator support for Write Batches 2014-11-09 16:10:11 +00:00
fyrz c4bf07c245 [RocksJava] -WShadow improvements
Minor corrections to resolve -WShadow build problems with RocksJava code.
2014-11-06 23:14:48 +01:00
Eugene Su 59d5497989 suppress JDK8 errors for #385 2014-11-06 16:25:53 +08:00
Yueh-Hsuan Chiang 71783f6524 Merge pull request #377 from fyrz/RocksJava-KeyMayExist
[RocksJava] KeyMayExist w/o ColumnFamilies
2014-11-04 11:05:19 -08:00
fyrz 94e31ac227 [RocksJava] Extend Options with DBOptions implementation [RocksJava] Included DBOptionsTest and refactored OptionsTest
Summary: Options refactoring - Split Part2

Test Plan:
make rocksdbjava
make jtest

Reviewers: yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D28017
2014-11-03 00:13:57 +01:00
fyrz b060d30065 [RocksJava] Build fix after options refactoring 2014-11-02 23:48:40 +01:00
fyrz 27129c739f [RocksJava] KeyMayExist w/o ColumnFamilies 2014-11-01 01:02:12 +01:00
Yueh-Hsuan Chiang ccaf1aa7cb Merge pull request #372 from fyrz/RocksJava-CF-Merge-Hardening
[RocksJava] Merge with ColumnFamilies & Hardening CFHandle
2014-10-30 16:49:40 -07:00
fyrz 85b04ca765 [RocksJava] Review comments - reformatted MergeTest 2014-10-31 00:41:04 +01:00
fyrz df7abb4e8d [RocksJava] Integrated code review comments
- Added TODO comments for disabled methods
2014-10-31 00:06:52 +01:00
fyrz 171be0ed55 Merge with ColumnFamilies & Hardening CFHandle
Summary:
ColumnFamilyHandles face the same problem as RocksIterator previously
so used methods were also applied for ColumnFamilyHandles.

Another problem with CF was that Options passed to CFs were
always filled with default values. To enable Merge, all parts
of the database must share the same merge functionality which
is not possible using default values. So from now on every
CF will inherit from db options.

Changes to RocksDB:
- merge can now take also a cfhandle

Changes to MergeTest:
- Corrected formatting
- Included also GC tests
- Extended tests to cover CF related parts
- Corrected paths to cleanup properly within the test process
- Reduced verbosity of the test

Test Plan:
make rocksdbjava
make jtest

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D27999
2014-10-30 22:32:04 +01:00
fyrz 39464a990f [RocksJava] Options Refactoring 3.6
Summary:
Options extends now two interfaces DBOptionsInterface
and ColumnFamilyOptionsInterface. There are also further
improvements to the Options bindings:

Optimize methods were ported to Java. (OptimizeForPointLookup,
OptimizeLevelCompaction, OptimizeUniversalCompaction).

To align BuiltinComparator with every other Enum it was moved to
a separate file.

Test Plan:
make rocksdbjava
make jtest
2014-10-30 22:26:39 +01:00
fyrz c73d13bb81 [RocksJava] Integrate review comments from yhchiang 2014-10-30 20:54:35 +01:00
fyrz b011e201fa Integrated review comments by ankgup87
- Added tests
- Minor code-style changes
2014-10-30 20:54:35 +01:00
fyrz 2c1bd8846f BlockBasedTableConfig & PlainTableConfig enhancements
Summary:
BlockBasedTableConfig
- ported Checksum
- ported IndexType

PlainTableConfig
- added missing options
- added EncodingType

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D26595
2014-10-30 20:54:35 +01:00
fyrz f7c9730696 [RocksJava] Integrated review comments
Parameter types for BackupID are now aligned to int.
2014-10-28 18:42:30 +01:00
fyrz 7e12ae5a21 [RocksJava] - BackupInfos & Restore-/BackupableDB enhancements
Summary:
- BackupableDB deleteBackup method
- BackupableDB purgeOldBackups bugfix
- BackupInfos now available in Restorable-/BackupableDB
- Extended BackupableDBTest to cover more of the currently implemented functionality.

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D27027
2014-10-28 18:42:05 +01:00
Yueh-Hsuan Chiang 75d7e2c374 Merge pull request #369 from fyrz/Small-Fix
[RocksJava] Minor correction to the previous pull request merge
2014-10-27 16:42:46 -07:00
Yueh-Hsuan Chiang db52419cf0 Merge pull request #290 from vladb38/master
Added merge operators to RocksJava
2014-10-27 16:41:33 -07:00
fyrz 45e756f04a [RocksJava] Minor correction to the previous pull request merge 2014-10-28 00:07:53 +01:00
fyrz b08c39e14f [RocksJava] RocksIterator: Assert for valid RocksDB instance & documentation 2014-10-27 23:53:27 +01:00
fyrz 56ef2caaa5 [RocksJava] - Hardening RocksIterator
RocksIterator will sometimes Sigsegv on dispose. Mainly thats related
to dispose order. If the related RocksDB instance is freed beforehand
RocksIterator.dispose() will fail.

Within this commit there is a major change to RocksIterator. RocksIterator
will hold a private reference to the RocksDB instance which created the
RocksIterator. So even if RocksDB is freed in the same GC cycle the
RocksIterator instances will be freed prior to related RocksDB instances.

Another aspect targets the dispose logic if the RocksDB is freed previously
and already gc`ed. On dispose of a RocksIterator the dispose logic will check
if the RocksDB instance points to an initialized DB. If not the dispose logic
will not perform any further action.

The crash can be reproduced by using the related test provided within this
commit.

Related information: This relates to @adamretter`s facebook rocksdb-dev group
post about SigSegv on RocksIterator.dispose().
2014-10-27 23:20:12 +01:00
Yueh-Hsuan Chiang 179c23021e Merge pull request #351 from fyrz/RocksJava_Snapshot_Support
[RocksJava] Support Snapshots
2014-10-27 14:50:22 -07:00
fyrz 679a9671fe RocksJava Fix after MutableCFOptions change. 2014-10-27 21:56:25 +01:00
Vlad Balan a04929aa49 fixed conflict in java/Makefile 2014-10-27 10:27:02 -07:00
fyrz a1bae76c87 Integrated changes due to review bei ankgup87 2014-10-26 13:27:43 +01:00
fyrz b8ce526487 [RocksJava] Support Snapshots
Summary:
Snapshots integration into RocksJava. Added support for the following functionalities:

- getSnapshot
- releaseSnapshot
- ReadOptions support to set a Snapshot
- ReadOptions support to retrieve Snapshot
- SnapshotTest

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24801
2014-10-25 11:10:19 +02:00
Yueh-Hsuan Chiang bc3bc4bc2f Merge pull request #357 from fyrz/JavaTest-Fix
Fix issues introduced by latest Java additions
2014-10-24 17:08:10 -07:00
fyrz bd4fbaee37 Fixed cross platform build after introducing Java-7 dependencies 2014-10-23 21:10:49 +02:00
fyrz 1eb545721d Fix incorrectly merged Java - Makefile 2014-10-23 21:10:49 +02:00
fyrz 9aa9668a83 [RocksJava] Memtables update to 3.6
- Adjusted HashLinkedList to 3.6.0
- Adjusted SkipList to 3.6.0
- Introduced a memtable test
2014-10-23 21:10:17 +02:00
Adam Retter a6fb7f312d Fix code review comments raised in https://reviews.facebook.net/D22779 2014-10-21 15:52:28 +01:00
Adam Retter c63494fb61 Tests for ComparatorOptions, Comparator and DirectComparator, and by
proxy we also exercise Slice and DirectSlice
2014-10-21 15:52:27 +01:00
Adam Retter 5e25274110 Fix code style problems identified by lint 2014-10-21 15:52:27 +01:00
Adam Retter 25641bfc9c Fix to memory dealocation when creating a slice from a byte buffer 2014-10-21 15:52:27 +01:00
Adam Retter fc12cb83f2 Add locking to comparator jni callback methods which must be thread-safe 2014-10-21 15:52:27 +01:00
Adam Retter d6fe8dacc8 Feature - Implement Java API for Comparator and Slice. Allows use of
either byte[] or DirectByteBuffer for accessing underlying data.
2014-10-21 15:52:27 +01:00
Vlad Balan dbcfe27d6c Merge branch 'master' of https://github.com/facebook/rocksdb 2014-10-17 15:44:52 -07:00
fyrz bafbc23baa Filters getting disposed by System.gc before EOL
Previous to this commit Filters passed as parameters to the
BlockTableConfig are disposed before they should be disposed.

Further Smart pointer usage was corrected.

Java holds now the smart pointer to the FilterPolicy correctly
and cares about freeing underlying c++ structures.
2014-10-15 20:59:25 +02:00
fyrz 1b97934a2c Options correction 2014-10-14 07:33:57 +02:00
Vlad Balan 2ef3ed86f3 Integrated feedback from ankgup87
Test Plan: tested using unit tests

Reviewers: ankgup87

Differential Revision: https://reviews.facebook.net/D24573
2014-10-13 15:37:29 -07:00
Vlad Balan a40ce219b9 Adding merge functions to RocksDBJava
Summary:
Added support for the merge operation to RocksJava.
You can specify a merge function to be used on the current database.
The merge function can either be one of the functions defined in
utilities/merge_operators.h, which can be specified through its
corresponding name, or a user-created function that needs to be
encapsulated in a JNI object in order to be used. Examples are
provided for both use cases.

Test Plan: There are unit test in MergeTest.java

Reviewers: ankgup87

Subscribers: vladb38

Differential Revision: https://reviews.facebook.net/D24525
2014-10-13 14:40:16 -07:00
fyrz ee28f431d0 With the last commit a Test was accidentally disabled. This commit solves this. 2014-10-13 21:12:42 +02:00
fyrz 18004d2f20 [RocksJava] Column family support
This commit includes the support for the following functionalities:

 - Single Get/Put operations
 - WriteBatch operations
 - Single iterator functionality
 - Open database with column families
 - Open database with column families Read/Only
 - Create column family
 - Drop column family
 - Properties of column families
 - Listing of column families
 - Fully backwards comptabile implementation
 - Multi Iterator support
 - MultiGet
 - KeyMayExist
 - Option to create missing column families on open

In addition there is are two new Tests:

 - Test of ColumnFamily functionality
 - Test of Read only feature to open subsets of column families
 - Basic test to test the KeyMayExist feature

What is not supported currently using RocksJava:

 - Custom ColumnFamilyOptions

The following targets work as expected:

 - make rocksdbjava
 - make jtest

Test environment: Ubuntu 14.04(LTS, x64), Java 1.7.0_65(OpenJDK IcedTea 2.5.2), g++ 4.8.2, kernel 3.13.0-35-generix
2014-10-13 10:34:52 +02:00
Yueh-Hsuan Chiang 5908d08a0b Merge pull request #336 from fyrz/32BitRocksJavaBug
32 Bit RocksJava Fix for overflowing jlongs
2014-10-12 19:48:54 -07:00
fyrz 4f5a687254 32-Bit RocksJava resolution for jlong overflows
Summary:
This pull request solves the jlong overflow problem on 32-Bit machines as described in https://github.com/facebook/rocksdb/issues/278:

1. There is a new org.rocksdb.test.PlatformRandomHelper to assist in getting random values. For 32 Bit the getLong method is overriden by xpromaches code above. For 64 Bit it behaves as is.
2. The detection should be cross-platform (Windows is supported though it is not ported completely yet).
3. Every JNI method which sets jlong values must check if the value fits into size_t. If it overflows size_t a InvalidArgument Status object will be returned. If its ok a OK Status will be returned.
4. Setters which have this check will throw a RocksDBException if its no OK Status.

Additionally some other parts of code were corrected using the wrong type casts.

Test Plan:
make rocksdbjava
make jtest

Differential Revision: https://reviews.facebook.net/D24531
2014-10-12 22:42:18 +02:00
fyrz 16d2ebdbcf Minor adjustment to prevent two warnings 2014-10-12 22:41:59 +02:00
fyrz 70294c9114 JavaDoc improvements on RocksJava
Added some more documentation improvements and readability
improvements.
2014-10-12 22:41:59 +02:00
fyrz ced6129574 Improved JavaDoc 2014-10-07 22:21:34 +02:00
fyrz 5e43155b3f RocksJava should support not only BlockBased Bloomfilter 2014-10-07 22:21:34 +02:00
fyrz 4f272408c5 RocksJava Makefile includes incorrect paths to version.h 2014-10-07 20:43:04 +02:00
ankgup87 daab6dc511 Merge pull request #318 from criccomini/master
Build RocksDB JNI cross-platform fat jar
2014-10-07 09:53:20 -07:00
Chris Riccomini 1e5a52815b update release readme 2014-10-06 08:24:51 -07:00
Chris Riccomini 2a4d6e7960 merge master to resolve merge conflicts 2014-10-06 08:22:55 -07:00
fyrz 22c64be439 Cross platform fix for Java benchmark shell script. 2014-10-05 10:46:55 +02:00
ankgup87 fcd13a77ad Merge pull request #334 from fyrz/JavaDoc-Cleanup
JavaDoc fixes & enhancements RocksJava
2014-10-03 14:47:30 -07:00
fyrz 05204bb119 Lint changes 2014-10-03 21:43:47 +02:00
fyrz da8ff9ff89 Fixed Findbugs issues
- BackupableDB missing call to super.finalize(major)
- WriteBatchTest inefficient String usage(minor)
- RocksDB local dead variable store(medium)
2014-10-03 12:20:40 +02:00
fyrz a5757ff3c2 Listing of changes
- JavaDoc readability of RocksObject JavaDoc
- JavaDoc improvements BlockBasedTableConfig, GenericRateLimiterConfig, RocksDB
- JavaDoc improvements MemTableConfig
- JavaDoc improvements RocksObject
- JavaDoc improvements GenericRateLimiterConfig
- JavaDoc improvements ReadOptions
- JavaDoc improvements RateLimiterConfig
- JavaDoc improvements RestoreOptions
- JavaDoc improvements RestoreBackupableDB
- JavaDoc improvements BlockBasedTableConfig
- JavaDoc improvements Options
- JavaDoc improvements BackupableDB and BackupableDBOptions
2014-10-03 11:59:54 +02:00
Yueh-Hsuan Chiang df3373fbf7 [Java] Fix compile error on DbBenchmark.java
Summary:
Fix compile error on DbBenchmark.java

Test Plan:
make rocksdbjava
make jdb_bench
2014-10-03 02:14:43 -07:00
Yueh-Hsuan Chiang 0e516a75da Fix lint errors in java/rocksjni/options.cc
Summary:
Fix lint errors in java/rocksjni/options.cc

Test Plan:
make rocksdbjava
2014-10-03 00:10:58 -07:00
Chris Riccomini e869fc6a88 remove proper javadoc directory 2014-10-02 15:46:49 -07:00
Chris Riccomini 99744e0c4b bump version to 3.6 2014-10-02 14:59:20 -07:00
Chris Riccomini 45d526e226 singular javadoc directory 2014-10-02 14:57:18 -07:00
Chris Riccomini 378f321da2 merge master to resolve merge conflicts 2014-10-02 14:52:10 -07:00
Chris Riccomini c832f1644a add not about updating pom version and rename pom to be unversioned 2014-10-02 14:42:49 -07:00
Chris Riccomini 2d72f7807f update release docs in java 2014-10-02 14:26:52 -07:00
Chris Riccomini 2e80124982 add javadoc and sources targets for sonatype 2014-10-02 13:57:54 -07:00
Chris Riccomini df08a2d03c add single rocksdbjni pom 2014-10-02 13:47:07 -07:00
Chris Riccomini fd2545c80a add maven publication target and instructions 2014-10-02 13:29:47 -07:00
fyrz 0173541778 FilterTest 2014-10-02 21:17:48 +02:00
fyrz d410b39d59 BlockBasedTableConfig Filter policy support RocksJava
As proposed by yhchiang the filter can now be set in Java for
a BlockBasedTableConfig instance.
2014-10-02 21:17:48 +02:00
Chris Riccomini d0916f452f add major minor micro version to java jars 2014-10-02 11:07:45 -07:00
fyrz b3343fdeac resolution for java build problem introduced by 5ec53f3edf 2014-10-02 09:25:07 +02:00
Chris Riccomini 3a0d498a36 rebase master 2014-09-30 14:40:04 -07:00
Chris Riccomini 0b923f0f9a add centos 5.6 build instead of ubuntu. 2014-09-29 16:58:16 -07:00
Chris Riccomini 6a64ea6171 add note about java 7 2014-09-29 10:57:38 -07:00
Chris Riccomini c4519c777f fix mis-named jar in JNI loader 2014-09-29 10:42:00 -07:00
Chris Riccomini b8e26615aa since we're not sharing folders with the vm, copy built .so files and jars back to host system. 2014-09-26 18:27:32 -07:00
Chris Riccomini 4e735bb7f9 Rsync files to VM rather than sync folders, since sync folders was causing clock skew and confusig make. 2014-09-26 15:41:28 -07:00
Chris Riccomini 82a8f43ccc Document RELEASE.mdgit status 2014-09-26 14:58:33 -07:00
Chris Riccomini 9db13987b1 Update RocksDB's Java bindings to support multiple native RocksDB builds in the same Jar file. Cross build RocksDB for linux32 and linux64 using Vagrant. Build a cross-platform fat jar that contains osx, linux32, and linux64 RocksDB static builds. 2014-09-26 13:57:12 -07:00
fyrz 5340484266 Built-in comparator(s) in RocksJava
Extended Built-in comparators with ReverseBytewiseComparator.

Reverse key handling is under certain conditions essential. E.g. while
using timestamp versioned data.

As native-comparators were not available using JAVA-API. Both built-in comparators
were exposed via JNI to be set upon database creation time.
2014-09-26 10:35:12 +02:00
Igor Canadi 21ddcf6e4f Remove allow_thread_local
Summary: See https://reviews.facebook.net/D19365

Test Plan: compiles

Reviewers: sdong, yhchiang, ljin

Reviewed By: ljin

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23907
2014-09-24 13:12:16 -07:00
Ankit Gupta 611e286b9d Merge branch 'master' of https://github.com/facebook/rocksdb 2014-09-24 11:44:23 -07:00
Ankit Gupta 0103b4498c Merge branch 'master' of ssh://github.com/ankgup87/rocksdb 2014-09-24 11:43:57 -07:00
Ankit Gupta 1dfb7bb980 Add block based table config options 2014-09-24 11:43:35 -07:00
Naveen ba6d660f6d Resolving merge conflict 2014-09-23 16:00:54 -07:00
Naveen 51eeaf65e2 Addressing review comments 2014-09-23 15:24:31 -07:00
Naveen fd7d3fe604 Addressing review comments (adding a env variable to override temp directory) 2014-09-23 15:24:19 -07:00
Naveen cf7ace8865 Addressing review comments 2014-09-23 15:23:57 -07:00
Ankit Gupta bfeef94d31 Add rate limiter 2014-09-19 16:11:59 -07:00
Ankit Gupta b93797abc4 Fix build 2014-09-18 22:13:52 -07:00
Yueh-Hsuan Chiang adae3ca1fe [Java] Fix JNI link error caused by the removal of options.db_stats_log_interval
Summary: Fix JNI link error caused by the removal of options.db_stats_log_interval in https://reviews.facebook.net/D21915.

Test Plan:
make rocksdbjava
make jtest

Reviewers: ljin, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23505
2014-09-18 21:09:44 -07:00
Yueh-Hsuan Chiang 94e43a1dfe [Java] Fixed 32-bit overflowing issue when converting jlong to size_t
Summary:
Fixed 32-bit overflowing issue when converting jlong to size_t by
capping jlong to std::numeric_limits<size_t>::max().

Test Plan:
make rocksdbjava
make jtest

Reviewers: ankgup87, ljin, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23511
2014-09-17 12:30:06 -07:00
Yueh-Hsuan Chiang acb9348ff3 [Java] Include WriteBatch into RocksDBSample.java, fix how DbBenchmark.java handles WriteBatch.
Summary:
Include WriteBatch into RocksDBSample.java, fix how DbBenchmark.java handles WriteBatch.
Previously DbBenchmark.java does not use WriteBatch when benchmarks is set to fillbatch.

Test Plan:
make rocksdbjava -j32
make jtest
make jdb_bench
cd java
./jdb_bench.sh --benchmarks=fillbatch

Reviewers: naveenatceg, ljin, sdong, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D22983
2014-09-15 12:34:12 -07:00
Lei Jin dd641b2117 fix RocksDB java build
Summary: as title

Test Plan: make rocksdbjava

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D23193
2014-09-10 12:00:32 -07:00
Naveen 1d284db213 Addressing review comments 2014-09-08 17:44:52 -07:00
Stanislau Hlebik 45a5e3ede0 Remove path with arena==nullptr from NewInternalIterator
Summary:
Simply code by removing code path which does not use Arena
from NewInternalIterator

Test Plan:
make all check
make valgrind_check

Reviewers: sdong

Reviewed By: sdong

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D22395
2014-09-04 17:40:41 -07:00
Lei Jin 66f62e5c78 JNI changes corresponding to BlockBasedTableOptions migration
Summary: as title

Test Plan:
tested on my mac
make rocksdbjava
make jtest

Reviewers: sdong, igor, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21963
2014-08-25 14:22:55 -07:00
Lei Jin 384400128f move block based table related options BlockBasedTableOptions
Summary:
I will move compression related options in a separate diff since this
diff is already pretty lengthy.
I guess I will also need to change JNI accordingly :(

Test Plan: make all check

Reviewers: yhchiang, igor, sdong

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21915
2014-08-25 14:22:05 -07:00
Naveen 343e98a7d1 Reverting import change 2014-08-18 14:08:10 -07:00
Naveen ddb8039e3a RocksDB static build
Make file changes to download and build the dependencies
.Load the shared library when RocksDB is initialized
2014-08-18 14:04:37 -07:00
Yueh-Hsuan Chiang 2da53b1e06 [Java] Add purgeOldBackups API
Summary:
1. Check status of CreateNewBackup. If status is not OK, then throw.
2. Add purgeOldBackups API

Test Plan:
make test
make sample

Reviewers: haobo, sdong, zzbennett, swapnilghike, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D21753
2014-08-14 10:58:09 -07:00
Yueh-Hsuan Chiang 10fc6c7d25 [Java] Add compaction style to options
Summary:
Add compression type to options

make rocksdbjava
make sample

Reviewers: haobo, yhchiang, sdong, dhruba, rsumbaly, zzbennett, swapnilghike
Reviewed By: yhchiang, sdong
CC: leveldb

Differential Revision: https://reviews.facebook.net/D20463
2014-07-23 15:45:30 -07:00
Yueh-Hsuan Chiang 6e7e3e45ff [Java] Update header inclusion of utilities files
Summary:
Update header inclusion as include/utilities/*.h has been moved to
include/rocksdb/utilities/*.h

Test Plan: make rocksdbjava

Reviewers: ljin, sdong, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20517
2014-07-23 11:15:14 -07:00
Ankit Gupta 25682d1596 [Java] Optimize statistics collector, improve object dependency in RocksObjects
Summary:
This diff merges pull request #208.  Contributor: ankgup87

[Java] Optimize statistics collector
* Optimize statistics collector by collecting statistics of multiple DBs in a single thread rather than starting up a new thread for each DB.
* Also, fix packaging of jnilib file on OS_X platform.
* Diff review: https://reviews.facebook.net/D20265

[Java] Add documentation on interdependency of dispose call of RocksObjects
* Remove transferCppRawPointersOwnershipFrom function.
  - This function was setting opt.filter_ and thus filter_ to be null. This way there is no
    one holding reference for filter object and can thus be GC'd which is not the intention.
    Replaced it with storeOptionsInstace which stores options instance. Options class
    internally holds Filter instance. Thus when Options is GC'd, filter reference
    will be GC'd automatically.
* Added documentation explaining interdependency of Filter, Options and DB.
* Diff review: https://reviews.facebook.net/D20379

Test Plan:
described in their diff reviews

Reviewers:  haobo sdong swapnilghike zzbennett rsumbaly yhchiang

Reviewed by: yhchiang
2014-07-22 14:52:50 -07:00
Yueh-Hsuan Chiang d19aa25594 [Java] Add Java support for cache sharding.
Summary:
Add setCacheNumShardBits() and cacheNumShardBits() to Options.  This allows
developers to control the number of shards for the block cache.

Test Plan:
make rocksdbjava
cd java
make db_bench
./jdb_bench.sh --cache_size=1048576 --cache_numshardbits=6

Reviewers: sdong, ljin, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19347
2014-07-22 09:48:37 -07:00
Yueh-Hsuan Chiang ae7743f226 Fixed some make and linking issues of RocksDBJava
Summary:
Fixed some make and linking issues of RocksDBJava. Specifically:
* Add JAVA_LDFLAGS, which does not include gflags
* rocksdbjava library now uses JAVA_LDFLAGS instead of LDFLAGS
* java/Makefile now includes build_config.mk
* rearrange make rocksdbjava workflow to ensure the library file is correctly
  included in the jar file.

Test Plan:
make rocksdbjava
make jdb_bench
java/jdb_bench.sh

Reviewers: dhruba, swapnilghike, zzbennett, rsumbaly, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20289
2014-07-21 22:41:54 -07:00
Yueh-Hsuan Chiang 80a94d0517 [Java] Fixed the Java compilation error caused by PlainTableFactory API change.
Summary: Fixed the Java compilation error caused by PlainTableFactory API change.

Test Plan:
make rocksdbjava
make jdb_bench
cd java;./jdb_bench.sh

Reviewers: sdong, ljin, ankgup87, swapnilghike, zzbennett, rsumbaly, igor

Reviewed By: igor

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D20295
2014-07-21 09:19:28 -07:00
Ankit Gupta 0a4d930264 Caching methodId and fieldId is fine: v2 2014-07-09 09:09:08 -07:00
Ankit Gupta b6caaea9d3 Caching methodId and fieldId is fine 2014-07-09 09:06:40 -07:00
Ankit Gupta 21e522673f Revert wrong commit 2014-07-09 09:06:20 -07:00
Ankit Gupta 05bd545000 Caching methodId and fieldId is fine 2014-07-09 09:02:05 -07:00
Ankit Gupta 4216ca36ae Class IDs and method IDs should not be cached 2014-07-07 21:45:02 -07:00
Ankit Gupta 637744ce15 Package .so file with JNI jar 2014-07-07 20:50:21 -07:00
Ankit Gupta 97bfcd6a16 Update doc of options.cc 2014-07-07 12:26:06 -07:00
Ankit Gupta c9cfb88256 Remove doc from options class 2014-07-07 12:21:15 -07:00
Ankit Gupta 4efca96bb3 Change enum type from int to byte 2014-07-07 12:18:55 -07:00
Ankit Gupta e8c592c625 Add newline 2014-07-07 10:25:27 -07:00
Ankit Gupta 47f0cf6d38 Add docs 2014-07-07 10:06:28 -07:00
Ankit Gupta da12f9ec4e Remove .so package with JNI 2014-07-07 10:01:28 -07:00
Ankit Gupta 54d7a2c0c5 Add compression type to options 2014-07-07 09:58:54 -07:00
Ankit Gupta 6b835c6009 Merge branch 'master' of https://github.com/facebook/rocksdb 2014-07-03 14:13:46 -07:00
Ankit Gupta 70aa243fa1 Package .so file with .jar 2014-07-03 14:13:05 -07:00
Yueh-Hsuan Chiang f799c8be5f [Java] Correct the library loading for zlib in RocksJava.
Summary:
Correct the library loading for zlib in RocksJava: zlib should
be loaded by loadLibrary("z") instead of loadLibrary("zlib").

Test Plan:
make rocksdbjava
cd java
make db_bench
./jdb_bench.sh --compression_type=zlib

Reviewers: sdong, ljin, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19341
2014-06-29 12:35:47 -07:00
Ankit Gupta b2625a4cc1 Add java docs for StatsCollector constructor and StatsCollectorCallback thread safety 2014-06-29 07:52:27 -07:00
Ankit Gupta abf9920386 Merge branch 'master' of https://github.com/facebook/rocksdb 2014-06-28 11:22:33 -07:00
Yueh-Hsuan Chiang 85f9bb4ef4 [Java] Enable compression_ratio option in DbBenchmark.java
Summary:
Enable the random values in Java DB Bench to be generated based
on the compression_ratio specified in the command-line arguments.

Test Plan:
make rocksdbjava
java/jdb_bench.sh

Reviewers: sdong, ankgup87, haobo

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19101
2014-06-24 15:12:36 -06:00
Ankit Gupta 4938032549 Fix line width 2014-06-22 13:31:10 -07:00
Ankit Gupta 718029fc38 Arc lint fixes 2014-06-22 13:30:26 -07:00
Ankit Gupta e0ebea6cc2 Add doc and end of line 2014-06-22 13:27:22 -07:00
Ankit Gupta 143ee5a1c1 Add stats collector 2014-06-22 13:23:10 -07:00
Yueh-Hsuan Chiang 403636842a [Java] Add HISTORY-JAVA.md and include both C++ and Java history files in .jar.
Summary:
Add HISTORY-JAVA.md and include both C++ and Java history files in .jar file
that describes important chagnes of RocksJava.

Test Plan: make rocksdbjava and make sure HISTORY.md is inside the .jar file

Reviewers: haobo, sdong, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19185
2014-06-20 23:18:25 -07:00
Yueh-Hsuan Chiang 21e18ffa73 [Java] Improve the Java Doc of RocksObject
Summary:
Improve the Java Doc of RocksObject, which explains more about the life-cycle
of a native handle.

Test Plan: no code change

Reviewers: haobo, sdong, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D19179
2014-06-20 23:16:54 -07:00
Yueh-Hsuan Chiang 3e701a654f [Java] Improve documentation for RocksEnv and its C++ resource.
Summary:
Improve documentation for RocksEnv and its C++ resource.  Specifically,
the result of RocksEnv::Default() is a singleton, and the ownership
of its c++ resource belongs to rocksdb c++.  As a result, calling
dispose() of the return value of RocksDB::Default() will be no-op.

Test Plan: no code change.

Reviewers: haobo, ankgup87

Reviewed By: ankgup87

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18945
2014-06-07 17:27:03 -07:00
Yueh-Hsuan Chiang e72b02e3c2 [Java] Add basic Java binding for rocksdb::Env.
Summary: Add basic Java binding for rocksdb::Env.

Test Plan:
make rocksdbjava
make jtest
cd java
./jdb_bench.sh --max_background_compactions=1
./jdb_bench.sh --max_background_compactions=10

Reviewers: sdong, ankgup87, haobo

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18903
2014-06-05 17:09:25 -07:00
Ankit Gupta 1a6c1a5ddd Fix build 2014-06-05 13:34:38 +01:00
Ankit Gupta 2fa0a993b8 Fix build 2014-06-05 13:25:08 +01:00
Ankit Gupta fa1b62cabd Merge remote-tracking branch 'upstream/master' 2014-05-29 18:33:11 -07:00
Yueh-Hsuan Chiang ab3e566699 [Java] Generalize dis-own native handle and refine dispose framework.
Summary:
1. Move disOwnNativeHandle() function from RocksDB to RocksObject
to allow other RocksObject to use disOwnNativeHandle() when its
ownership of native handle has been transferred.

2. RocksObject now has an abstract implementation of dispose(),
which does the following two things.  First, it checks whether
both isOwningNativeHandle() and isInitialized() return true.
If so, it will call the protected abstract function dispose0(),
which all the subclasses of RocksObject should implement.  Second,
it sets nativeHandle_ = 0.  This redesign ensure all subclasses
of RocksObject have the same dispose behavior.

3. All subclasses of RocksObject now should implement dispose0()
instead of dispose(), and dispose0() will be called only when
isInitialized() returns true.

Test Plan:
make rocksdbjava
make jtest

Reviewers: dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett, haobo

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18801
2014-05-28 18:16:29 -07:00
Yueh-Hsuan Chiang 663627abf3 Merge pull request #153 from bpot/fix_jni_segfault
Segfault when using BackupableDB with Java bindings
2014-05-27 14:07:22 -07:00
Yueh-Hsuan Chiang 8e41e54e1b [Java] Makes DbBenchmark takes 0 and 1 as boolean values.
Summary:
Originally, the boolean arguments in Java DB benchmark only takes
true and false as valid input.  This diff allows boolean arguments
to be set using 0 or 1.

Test Plan:
make rocksdbjava
make jdb_bench
java/jdb_bench.sh --db=/tmp/path-not-exist --use_existing_db=1
java/jdb_bench.sh --db=/tmp/path-not-exist --use_existing_db=0
java/jdb_bench.sh --db=/tmp/path-not-exist --use_existing_db=1

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

Subscribers: leveldb

Differential Revision: https://reviews.facebook.net/D18687
2014-05-27 13:52:44 -07:00
Bob Potter 05dd018353 BackupableDB: don't keep a reference to the RocksDB object now that we have disOwnNativeObject 2014-05-27 13:54:02 -05:00
Bob Potter 5fe176d8f6 Java Bindings: prevent segfault from double delete
Give BackupableDB sole responsibility of the native
object to prevent RocksDB from also trying to delete
it.
2014-05-21 18:47:18 -05:00
Ankit Gupta d271cc5b4e Treat negative values as no limit 2014-05-21 07:54:22 -07:00
Ankit Gupta e87973cde1 Removing code from portal.h for setting handle of RestoreOptions and RestoreBackupableDB 2014-05-18 22:40:48 -07:00
Ankit Gupta 29eef1f87d Add javadoc for BackupableDBOptions constructor params 2014-05-13 22:40:01 -07:00
Ankit Gupta b830bb937f Add newline at end of file 2014-05-13 22:34:29 -07:00
Ankit Gupta c92425e0ba Merge branch 'master' of https://github.com/facebook/rocksdb 2014-05-13 22:23:04 -07:00
Ankit Gupta cf0b773a29 Add more options to backupable DB 2014-05-13 22:22:21 -07:00
Ankit Gupta 036e323f7a Add RestoreBackupableDB and RestoreOptions 2014-05-13 22:20:58 -07:00
Yueh-Hsuan Chiang 9a0e3ab7ec Merge branch 'master' of github.com:facebook/rocksdb into show-reads 2014-05-13 16:45:54 -07:00
Yueh-Hsuan Chiang e883407ead [Java] Refined the output of Java DbBenchmark. 2014-05-13 16:44:53 -07:00
Yueh-Hsuan Chiang e30dec938d [Java] Fixed a bug in Java DB Benchmark where random reads does not consider full key range.
Summary: Fixed a bug in Java DB Benchmark where random reads does not consider full key range.

Test Plan:
make rocksdbjava
make jdb_bench
cd java
jdb_bench.sh --db=/tmp/rocksdb-test --benchmarks=fillseq --use_existing_db=false --num=100000
jdb_bench.sh --db=/tmp/rocksdb-test --benchmarks=readrandom --use_existing_db=true --num=100000 --reads=1000000

Reviewers: haobo, sdong

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18693
2014-05-13 13:19:50 -07:00
Yueh-Hsuan Chiang 93f2643656 [Java] Make read benchmarks print out found / not-found information.
Summary: Make read benchmarks print out found / not-found information.

Test Plan:
make rocksdbjava
make jdb_bench
cd java
./jdb_bench.sh

Reviewers: haobo, sdong

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18699
2014-05-13 13:11:26 -07:00
Yueh-Hsuan Chiang 2082a7d745 [Java] Temporary set the number of BG threads based on the number of BG compactions.
Summary:
Before the Java binding for Env is ready, Java developers have no way to
control the number of background threads.  This diff provides a temporary
solution where RocksDB.setMaxBackgroundCompactions() will affect the
number of background threads.

Note that once Env is ready.  Changes made in this diff should be reverted.

Test Plan:
make rocksdbjava
make jtest
make jdb_bench
java/jdb_bench.sh

Reviewers: haobo, sdong

Reviewed By: sdong

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18681
2014-05-13 12:28:47 -07:00
Yueh-Hsuan Chiang d14581f936 [Java] Rename org.rocksdb.Iterator to org.rocksdb.RocksIterator.
Summary:
Renamed Iterator to RocksIterator to avoid potential confliction with
the Java built-in Iterator.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, dhruba, sdong, ankgup87

Reviewed By: sdong

CC: leveldb, rsumbaly, swapnilghike, zzbennett

Differential Revision: https://reviews.facebook.net/D18615
2014-05-12 11:02:25 -07:00
Yueh-Hsuan Chiang 61955a0dda [Java] Add RocksObject, the base class of all java objects with a c++ pointer.
Summary:
Add RocksObject, the base class of all java objects which has a c++ pointer.
While the finalizer of a RocksObject will release its c++ resource, it is
suggested to call its RocksObject.dispose() to manually release its
c++ resource.

Existing RocksDB java classes are now extending RocksObject.

Test Plan:
make rocksdbjava
make jtest
make jdb_bench

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18411
2014-05-01 01:44:46 -07:00
Yueh-Hsuan Chiang a3ab998ef9 [Java] Add static method RocksDB.loadLibrary()
Summary:
Add static method RocksDB.loadLibrary() which loads necessary library
files.

Test Plan:
make rocksdbjava
make jtest
make jdb_bench
java/jdb_bench.sh

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: ankgup87

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18375
2014-04-30 11:54:02 -07:00
Ankit Gupta d5cb6e7bb5 Assert on key size in multiget 2014-04-29 08:02:52 -07:00
Ankit Gupta 65fba4b984 Merge branch 'master' of https://github.com/facebook/rocksdb 2014-04-29 07:53:29 -07:00
Yueh-Hsuan Chiang 9895465c6a [Java] Fixed compile error due to the removal of ReadOptions.prefix_seek, minor improvement on DbBenchmark.java. 2014-04-27 21:49:45 -07:00
Ankit Gupta 13b5632fdd Merge branch 'master' of https://github.com/facebook/rocksdb 2014-04-25 22:41:04 -07:00
Ankit Gupta 9e0ab427b8 Fix bug + formatting 2014-04-25 22:39:33 -07:00
Ankit Gupta 3f8b4129ef Fix formatting 2014-04-25 20:59:16 -07:00
Ankit Gupta e96e71becf Multi-get impl and formatting 2014-04-25 13:57:20 -07:00
Yueh-Hsuan Chiang a82c492a95 [Java] Fixed some errors in the sample and benchmark code. 2014-04-24 22:15:25 -07:00
Yueh-Hsuan Chiang e537a840c2 [Java] Fixed a compile error caused in some Java version. 2014-04-24 16:17:00 -07:00
Yueh-Hsuan Chiang 3e0b93b5da [Java] Fixed a compile error. 2014-04-24 15:37:27 -07:00
Yueh-Hsuan Chiang 05979493d6 [Java] Enable filluniquerandom, readseq, BloomFilter, and 70+ command-line options to DbBenchmark.java
Summary:
* Add filluniquerandom
* Add readseq, implemented using iterator.
* Realize most command-line-arguments from db_bench.cc (70+).
* Some code are commented out as some of the options in Options
  not yet have Java bindings.
* Add default option to DbBenchmark.
* RocksDB will now take the ownership of all c++ raw-pointers from Options, which includes a c++ raw-pointer for Filter.

Test Plan: ./jdb_bench.sh --db=/tmp/rocksjava-bench/db --num_levels=6 --key_size=20 --prefix_size=20 --keys_per_prefix=0 --value_size=100 --block_size=4096 --cache_size=17179869184 --cache_numshardbits=6 --compression_ratio=1 --min_level_to_compress=-1 --disable_seek_compaction=1 --hard_rate_limit=2 --write_buffer_size=134217728 --max_write_buffer_number=2 --level0_file_num_compaction_trigger=8 --target_file_size_base=134217728 --max_bytes_for_level_base=1073741824 --disable_wal=0 --wal_dir=/tmp/rocksjava-bench/wal --sync=0 --disable_data_sync=1 --verify_checksum=1 --delete_obsolete_files_period_micros=314572800 --max_grandparent_overlap_factor=10 --max_background_compactions=4 --max_background_flushes=0 --level0_slowdown_writes_trigger=16 --level0_stop_writes_trigger=24 --statistics=0 --stats_per_interval=0 --stats_interval=1048576 --histogram=0 --use_plain_table=1 --open_files=-1 --mmap_read=1 --mmap_write=0 --memtablerep=prefix_hash --bloom_bits=10 --bloom_locality=1 --perf_level=0 --benchmarks=filluniquerandom,readseq,readrandom --use_existing_db=0 --threads=4

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18267
2014-04-24 13:43:57 -07:00
Yueh-Hsuan Chiang d8fe006056 [Java] Add Java bindings and Java tests for 30+ rocksdb::Options.
Summary:
Add Java bindings and Java tests for 30+ rocksdb::Options.  Codes are
machine-genearted based on include/rocksdb/options.h with manual-polishment.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18273
2014-04-23 20:54:35 -07:00
Ankit Gupta dd9f6f0a31 Fix formatting 2014-04-22 10:51:39 -07:00
Ankit Gupta 7a5106fbea Add doc 2014-04-22 09:01:57 -07:00
Ankit Gupta 2214fd8a15 Refactor filter impl 2014-04-22 08:58:43 -07:00
Ankit Gupta 89cb481aa1 Fix doc 2014-04-22 00:09:40 -07:00
Ankit Gupta 677b0d6d3f Refactor filter impl 2014-04-22 00:04:56 -07:00
Ankit Gupta 5e797cf0dd Change filter implementation 2014-04-21 23:56:19 -07:00
Ankit Gupta cea2be20b6 Fix formatting 2014-04-21 20:27:09 -07:00
Ankit Gupta dc4b27ac48 Add bloom filters 2014-04-21 20:25:30 -07:00
Yueh-Hsuan Chiang e316af5f16 [Java] Add Java binding and Java test for ReadOptions.
Summary: Add Java binding and test for rocksdb::ReadOptions.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, dhruba, sdong, ankgup87, rsumbaly, swapnilghike, zzbennett

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18129
2014-04-21 15:52:59 -07:00
Yueh-Hsuan Chiang ef8b8a8ef6 [Java] Add Java bindings for memtables and sst format.
Summary:
Add Java bindings for memtables and sst format.  Specifically,
add two abstract Java classses --- MemTableConfig and SstFormatConfig.
Each MemTable / SST implementation should has its own config class
extends MemTableConfig / SstFormatConfig respectively and pass it
to Options via setMemTableConfig / setSstConfig.

Test Plan:
make rocksdbjava
make jdb_test
make jdb_bench
java/jdb_bench.sh \
  --benchmarks=fillseq,readrandom,readwhilewriting \
  --memtablerep=hash_skiplist \
  --use_plain_table=1 \
  --key_size=20 \
  --prefix_size=12 \
  --value_size=100 \
  --cache_size=17179869184 \
  --disable_wal=0 \
  --sync=0 \

Reviewers: haobo, ankgup87, sdong

Reviewed By: haobo

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17997
2014-04-21 15:40:46 -07:00
Ankit Gupta bbdd550b66 Remove getIterator function from portal 2014-04-19 23:17:42 -07:00
Ankit Gupta 1574e0c41a Add doc 2014-04-19 13:21:06 -07:00
Ankit Gupta 06b590dd7c Add doc 2014-04-19 13:13:01 -07:00
Ankit Gupta dc28a726c1 Add doc + refactor + fix formatting 2014-04-19 13:05:21 -07:00
Ankit Gupta 1d6c1e018f Add more iterator JNI bindings 2014-04-19 12:55:28 -07:00
Ankit Gupta eda398491a Add more iterator functions 2014-04-19 03:35:01 -07:00
Ankit Gupta 5bbeefaa49 Adding iterator JNI binding 2014-04-19 03:26:22 -07:00
Ankit Gupta 686fdea811 Fix formatting issues 2014-04-18 10:48:48 -07:00
Ankit Gupta ebd85e8f3a Fix build 2014-04-18 10:47:03 -07:00
Ankit Gupta dc291f5bf0 Merge branch 'master' of https://github.com/facebook/rocksdb
Conflicts:
	Makefile
	java/Makefile
	java/org/rocksdb/Options.java
	java/rocksjni/portal.h
2014-04-18 10:32:14 -07:00
Yueh-Hsuan Chiang 9b2a0939cf [Java] Add Java bindings for 30 options for rocksdb::DBOptions.
Summary:
1. Add Java bindings for 30 options for rocksdb::DBOptions.
2. Add org.rocksdb.test.OptionsTest
3. Codes are semi-auto generated, JavaDocs are manually polished.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D18015
2014-04-18 01:14:29 -07:00
Yueh-Hsuan Chiang bb6fd15a6e [Java] Add a basic binding and test for BackupableDB and StackableDB.
Summary:
Add a skeleton binding and test for BackupableDB which shows that BackupableDB
and RocksDB can share the same JNI calls.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17793
2014-04-17 17:28:51 -07:00
Ankit Gupta 136e4efb84 Add doc 2014-04-16 22:02:46 -07:00
Ankit Gupta d160595d97 Add doc 2014-04-16 22:00:44 -07:00
Ankit Gupta d3b44f072f Fix white spaces 2014-04-16 21:55:15 -07:00
Ankit Gupta 5108ed7a2f Add fixed classes 2014-04-16 21:39:29 -07:00
Ankit Gupta 320ae72e17 Add histogramType for statistics 2014-04-16 21:38:33 -07:00
Ankit Gupta 144066a8ff Deleted statisticsType 2014-04-15 18:08:37 -07:00
Ankit Gupta b18d914389 Change statisticsType to tickerType 2014-04-15 17:37:50 -07:00
Ankit Gupta 3723d08290 Add newline at end of file 2014-04-14 22:16:10 -07:00
Ankit Gupta 98421774b6 remove unused header 2014-04-14 21:14:29 -07:00
Ankit Gupta 8756a3210f Remove unused headers 2014-04-14 21:11:03 -07:00
Ankit Gupta c199e0e2a4 Add statistics 2014-04-14 21:08:04 -07:00
Ankit Gupta a044398260 Add statistics object 2014-04-14 21:06:13 -07:00
Yueh-Hsuan Chiang 51f4b5090d [Java] Add SizeUnit in org.rocksdb.util to store const like KB, GB.
Summary:
* Add a class SizeUnit to store frequently used consts.  Currently
  it has KB, MB, GB, TB, and PB.

* Change the parameter type of Options.writeBufferSize and Options.blockSize
  from int to long.

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17703
2014-04-14 14:03:43 -07:00
Yueh-Hsuan Chiang 31e7e7fe84 [JAVA] Add java binding for Options.block_cache.
Summary:
Add java bindings for Options.block_cache and allow DbBenchmark to
set cache_size.

Test Plan:
make rocksdbjava
make jtest
make jdb_Bench

Reviewers: haobo, sdong, ankgup87

Reviewed By: ankgup87

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17481
2014-04-14 13:42:36 -07:00
Yueh-Hsuan Chiang 2885ad9b77 [JNI] Each set function of Options / WriteOptions now returns its option instance.
Summary:
Make each set function of Options / WriteOptions return its option instance.
Java developers can now easier specify each option like the following:

    options.setCreateIfMissing(true)
        .setWriteBufferSize(8 * 1024)
        .setMaxWriteBufferNumber(3)
        .setDisableSeekCompaction(true)
        .setBlockSize(64 * 1024)
        .setMaxBackgroundCompactions(10);

Test Plan:
make rocksdbjava
make jtest

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17661
2014-04-14 12:48:50 -07:00
Yueh-Hsuan Chiang dfe2d2f3a2 [JNI] Add readwhilewriting to Java db_bench
Summary: Add readwhilewriting to Java db_bench

Test Plan:
make jni
make jdb_bench

Reviewers: haobo, ankgup87, sdong, dhruba

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17619
2014-04-10 16:12:04 -07:00
Yueh-Hsuan Chiang 0f5cbcd798 [JNI] Add an initial benchmark for java binding for rocksdb.
Summary:
* Add a benchmark for java binding for rocksdb.  The java benchmark
  is a complete rewrite based on the c++ db/db_bench.cc and the
  DbBenchmark in dain's java leveldb.
* Support multithreading.
* 'readseq' is currently not supported as it requires RocksDB Iterator.

* usage:

  --benchmarks
    Comma-separated list of operations to run in the specified order
        Actual benchmarks:
                fillseq    -- write N values in sequential key order in async mode
                fillrandom -- write N values in random key order in async mode
                fillbatch  -- write N/1000 batch where each batch has 1000 values
                              in random key order in sync mode
                fillsync   -- write N/100 values in random key order in sync mode
                fill100K   -- write N/1000 100K values in random order in async mode
                readseq    -- read N times sequentially
                readrandom -- read N times in random order
                readhot    -- read N times in random order from 1% section of DB
        Meta Operations:
                delete     -- delete DB
    DEFAULT: [fillseq, readrandom, fillrandom]

  --compression_ratio
    Arrange to generate values that shrink to this fraction of
        their original size after compression
    DEFAULT: 0.5

  --use_existing_db
    If true, do not destroy the existing database.  If you set this
        flag and also specify a benchmark that wants a fresh database,  that benchmark will fail.
    DEFAULT: false

  --num
    Number of key/values to place in database.
    DEFAULT: 1000000

  --threads
    Number of concurrent threads to run.
    DEFAULT: 1

  --reads
    Number of read operations to do.  If negative, do --nums reads.

  --key_size
    The size of each key in bytes.
    DEFAULT: 16

  --value_size
    The size of each value in bytes.
    DEFAULT: 100

  --write_buffer_size
    Number of bytes to buffer in memtable before compacting
        (initialized to default value by 'main'.)
    DEFAULT: 4194304

  --cache_size
    Number of bytes to use as a cache of uncompressed data.
        Negative means use default settings.
    DEFAULT: -1

  --seed
    Seed base for random number generators.
    DEFAULT: 0

  --db
    Use the db with the following name.
    DEFAULT: /tmp/rocksdbjni-bench

* Add RocksDB.write().

Test Plan: make jbench

Reviewers: haobo, sdong, dhruba, ankgup87

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17433
2014-04-09 00:48:20 -07:00
Yueh-Hsuan Chiang c65448f95a Merge branch 'master' of github.com:facebook/rocksdb into HEAD 2014-04-08 22:21:39 -07:00
Yueh-Hsuan Chiang 5abae2c8ed [JNI] Fixed a column family related compile error.
Summary:
Fixed the following column family related compile error.

./java/rocksjni/write_batch.cc:211:66: error: cannot initialize a parameter of type 'rocksdb::ColumnFamilyMemTables *' with an lvalue of type 'rocksdb::MemTable *'
  rocksdb::Status s = rocksdb::WriteBatchInternal::InsertInto(b, mem, &options);
                                                                   ^~~

Test Plan:
make jni
make jtest

Reviewers: igor, haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17577
2014-04-08 09:54:59 -07:00
Ankit Gupta 8a509717b7 Fix formatting issues 2014-04-08 09:30:35 -07:00
Ankit Gupta 14220442a7 Fix formatting 2014-04-08 09:27:24 -07:00
Ankit Gupta 22d45de2ed Fix formatting errors 2014-04-07 18:32:09 -07:00
Ankit Gupta c990a7645a Add documentation for JNI options 2014-04-06 09:55:48 -07:00
Ankit Gupta 63d74d2bfa Expose JNI layer for these options: block_size, max_write_buffer_number, write_buffer_size, disable_seek_compaction, max_background_compactions. 2014-04-06 09:33:31 -07:00
Yueh-Hsuan Chiang e3511841fa [JNI] Avoid a potential byte-array-copy btw c++ and java in RocksDB.get(byte[], byte[]).
Summary: Avoid a JNI call to GetByteArrayElements, which may introduce a byte-array-copy.

Test Plan: make jtest

Reviewers: haobo, sdong, dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17451
2014-04-03 00:02:01 -07:00
Yueh-Hsuan Chiang 92d2766001 [JNI] Improve the internal interface between java and c++ for basic db operations.
Summary:
Improve the internal interface between java and c++ for basic db operations
by including the RocksDB native handle (i.e., c++ pointer of rocksdb::DB)
as a input parameter of the internal interface.

This improvement reduces one JNI call per db operation from c++.

Test Plan: make test

Reviewers: haobo, sdong, dhruba

Reviewed By: sdong

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17445
2014-04-02 22:23:04 -07:00
Yueh-Hsuan Chiang da0887a3dc [JNI] Add java api and java tests for WriteBatch and WriteOptions, add put() and remove() to RocksDB.
Summary:
* Add java api for rocksdb::WriteBatch and rocksdb::WriteOptions, which are necessary components
  for running benchmark.
* Add java test for org.rocksdb.WriteBatch and org.rocksdb.WriteOptions.
* Add remove() to org.rocksdb.RocksDB, and add put() and remove() to RocksDB which take
  org.rocksdb.WriteOptions.

Test Plan: make jtest

Reviewers: haobo, sdong, dhruba

Reviewed By: sdong

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17373
2014-04-02 14:49:20 -07:00
Yueh-Hsuan Chiang 8c4a3bfa5b Add a java api for rocksdb::Options, currently only supports create_if_missing.
Summary:
* [java] Add a java api for rocksdb::Options, currently only supports create_if_missing.
* [java] Add a test for RocksDBException in RocksDBSample.

Test Plan: make jtest

Reviewers: haobo, sdong

Reviewed By: haobo

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D17385
2014-04-01 16:59:05 -07:00
Yueh-Hsuan Chiang a73383e8ac Minor fix in rocksdb jni library, RocksDB now does not implement Closeable.
Summary:
* [java] RocksDB now does not implement Closeable.
* [java] RocksDB.close() is now synchronized.
* [c++] Fix a bug in rocksjni.cc that does not release a java reference before
        throwing an exception.

Test Plan:
make jni
make jtest

Reviewers: haobo, sdong

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17355
2014-03-31 21:46:10 -07:00
Yueh-Hsuan Chiang 5ec38c3d3e Minor fix in rocksdb jni library.
Summary: Fix a bug in RocksDBSample.java and minor code improvement in rocksjni.cc.

Test Plan:
make jni
make jtest

Reviewers: haobo, sdong

Reviewed By: haobo

CC: leveldb

Differential Revision: https://reviews.facebook.net/D17325
2014-03-29 22:00:52 -07:00
Yueh-Hsuan Chiang 0d463a3685 Add a jni library for rocksdb which supports Open, Get, Put, and Close.
Summary:
This diff contains a simple jni library for rocksdb which supports open, get,
put and closeusing default options (including Options, ReadOptions, and
WriteOptions.)  In the usual case, Java developers can use the c++ rocksdb
library in the way similar to the following:

    RocksDB db = RocksDB.open(path_to_db);
    ...
    db.put("hello".getBytes(), "world".getBytes();
    byte[] value = db.get("hello".getBytes());
    ...
    db.close();

Specifically, this diff has the following major classes:

* RocksDB: a Java wrapper class which forwards the operations
  from the java side to c++ rocksdb library.
* RocksDBException: ncapsulates the error of an operation.
  This exception type is used to describe an internal error from
  the c++ rocksdb library.

This diff also include a simple java sample code calling c++ rocksdb library.

To build the rocksdb jni library, simply run make jni, and make jtest will try to
build and run the sample code.

Note that if the rocksdb is not built with the default glibc that Java uses,
java will try to load the wrong glibc during the run time.  As a result,
the sample code might not work properly during the run time.

Test Plan:
* make jni
* make jtest

Reviewers: haobo, dhruba, sdong, igor, ljin

Reviewed By: dhruba

CC: leveldb, xjin

Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 14:19:21 -07:00
Dhruba Borthakur 0a9f873f4b Removed scribe, thrift and java modules.
Summary: Removed scribe, thrift and java modules.

Test Plan:
make release
make check

Reviewers: emayanke

Reviewed By: emayanke

CC: leveldb

Differential Revision: https://reviews.facebook.net/D13293
2013-10-04 15:36:00 -07:00
Dhruba Borthakur a143ef9b38 Change namespace from leveldb to rocksdb
Summary:
Change namespace from leveldb to rocksdb. This allows a single
application to link in open-source leveldb code as well as
rocksdb code into the same process.

Test Plan: compile rocksdb

Reviewers: emayanke

Reviewed By: emayanke

CC: leveldb

Differential Revision: https://reviews.facebook.net/D13287
2013-10-04 11:59:26 -07:00
Abhishek Kona 41d7809212 Pom changes to make relase 1.5.7 for java.
Summary: Ran ./build_java.sh bump_version 1.5.7

Test Plan: automated change

Reviewers: dhruba

Reviewed By: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D7833
2013-01-10 10:43:43 -08:00
Abhishek Kona 2585979fd7 Release 1.5.6 for Java code + Script to automate it.
Test Plan: it compiles and deploys.

Reviewers: dhruba

CC: leveldb

Differential Revision: https://reviews.facebook.net/D7341
2012-12-17 12:11:11 -08:00
Abhishek Kona f20383f095 Make Java Client compilable.
Summary:
Debug and ported changes from the Open Source Github repo to our repo.
Wrote a script to easy build the java Library. future compiling java lib should just be running this script.

Test Plan: it compiles.

Reviewers: dhruba, leveldb

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D7323
2012-12-12 14:07:52 -08:00
sheki dcd919a0a4 Port various compaction options to Java.
Summary:
Porting various options, mostly related to Multi-threaded compaction to
Java.

Test Plan:
mvn test.
No clear plan on how else test.

Reviewers: dhruba

Reviewed By: dhruba

CC: leveldb, emayanke

Differential Revision: https://reviews.facebook.net/D7221
2012-12-10 10:53:48 -08:00
Thawan Kooburat 696b290821 Add LevelDb's JNI wrapper
Summary: This implement the Java interface by using JNI

Test Plan: compile test

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D5925
2012-10-05 13:13:49 -07:00
Thawan Kooburat fc23714f27 Add LevelDb's Java interface
Summary:
See the wiki below
https://our.intern.facebook.com/intern/wiki/index.php/Database/leveldb/Java

Test Plan: compile test

Reviewers: dhruba

Reviewed By: dhruba

Differential Revision: https://reviews.facebook.net/D5919
2012-10-05 13:11:31 -07:00