dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/java/Makefile

330 lines
12 KiB
Makefile
Raw Normal View History

Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 20:28:25 +00:00
NATIVE_JAVA_CLASSES = \
org.rocksdb.AbstractCompactionFilter\
Added CompactionFilterFactory support to RocksJava Summary: This PR also includes some cleanup, bugfixes and refactoring of the Java API. However these are really pre-cursors on the road to CompactionFilterFactory support. Closes https://github.com/facebook/rocksdb/pull/1241 Differential Revision: D6012778 Pulled By: sagar0 fbshipit-source-id: 0774465940ee99001a78906e4fed4ef57068ad5c
2017-10-12 18:06:51 +00:00
org.rocksdb.AbstractCompactionFilterFactory\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 20:28:25 +00:00
org.rocksdb.AbstractComparator\
Add event listeners to RocksJava (#7425) Summary: Allows adding event listeners in RocksJava. * Adds listeners getter and setter in `Options` and `DBOptions` classes. * Adds `EventListener` Java interface and base class for implementing custom event listener callbacks - `AbstractEventListener`, which has an underlying native callback class implementing C++ `EventListener` class. * `AbstractEventListener` class has mechanism for selectively enabling its callback methods in order to prevent invoking Java method if it is not implemented. This decreases performance cost in case only subset of event listener callback methods is needed - the JNI code for remaining "no-op" callbacks is not executed. * The code is covered by unit tests in `EventListenerTest.java`, there are also tests added for setting/getting listeners field in `OptionsTest.java` and `DBOptionsTest.java`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7425 Reviewed By: pdillinger Differential Revision: D24063390 Pulled By: jay-zhuang fbshipit-source-id: 508c359538983d6b765e70d9989c351794a944ee
2020-10-14 18:32:10 +00:00
org.rocksdb.AbstractEventListener\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.AbstractSlice\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.AbstractTableFilter\
org.rocksdb.AbstractTraceWriter\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 18:22:38 +00:00
org.rocksdb.AbstractTransactionNotifier\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.AbstractWalFilter\
Expose the BackupEngine from the Java API Summary: Merge pull request #665 by adamretter Exposes BackupEngine from C++ to the Java API. Previously only BackupableDB was available Test Plan: BackupEngineTest.java Reviewers: fyrz, igor, ankgup87, yhchiang Reviewed By: yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D42873
2015-08-04 23:07:17 +00:00
org.rocksdb.BackupEngine\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.BackupableDBOptions\
org.rocksdb.BlockBasedTableConfig\
org.rocksdb.BloomFilter\
[RocksJava] Support for stored snapshots Summary: RocksDB supports two ways of saving snapshots. In memory and on disk. The later was added with this pull request to RocksJava. Test Plan: Reviewers: Subscribers:
2014-11-21 20:57:32 +00:00
org.rocksdb.Checkpoint\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 19:04:09 +00:00
org.rocksdb.ClockCache\
Cassandra compaction filter for purge expired columns and rows Summary: Major changes in this PR: * Implement CassandraCompactionFilter to remove expired columns and rows (if all column expired) * Move cassandra related code from utilities/merge_operators/cassandra to utilities/cassandra/* * Switch to use shared_ptr<> from uniqu_ptr for Column membership management in RowValue. Since columns do have multiple owners in Merge and GC process, use shared_ptr helps make RowValue immutable. * Rename cassandra_merge_test to cassandra_functional_test and add two TTL compaction related tests there. Closes https://github.com/facebook/rocksdb/pull/2588 Differential Revision: D5430010 Pulled By: wpc fbshipit-source-id: 9566c21e06de17491d486a68c70f52d501f27687
2017-07-21 21:42:32 +00:00
org.rocksdb.CassandraCompactionFilter\
Create a MergeOperator for Cassandra Row Value Summary: This PR implements the MergeOperator for Cassandra Row Values. Closes https://github.com/facebook/rocksdb/pull/2289 Differential Revision: D5055464 Pulled By: scv119 fbshipit-source-id: 45f276ef8cbc4704279202f6a20c64889bc1adef
2017-06-16 21:12:52 +00:00
org.rocksdb.CassandraValueMergeOperator\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.ColumnFamilyHandle\
[RocksJava] ColumnFamily custom Options API extension ********************* *************************** ******** ************* ******** ******** *********** ******** ******** ********* ******** ************************************* ************************************* ************************************* ****** *** *** ****** ****** *** *** *** ****** ****** *** ****** *************************** *********************
2014-10-31 22:39:14 +00:00
org.rocksdb.ColumnFamilyOptions\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.CompactionJobInfo\
org.rocksdb.CompactionJobStats\
org.rocksdb.CompactionOptions\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 19:04:09 +00:00
org.rocksdb.CompactionOptionsFIFO\
org.rocksdb.CompactionOptionsUniversal\
Add CompactRangeOptions for Java (#4220) Summary: Closes https://github.com/facebook/rocksdb/issues/4195 CompactRangeOptions are available the CPP API, but not in the Java API. This PR adds CompactRangeOptions to the Java API and adds an overloaded compactRange() method. See https://github.com/facebook/rocksdb/issues/4195 for the original discussion. This change supports all fields of CompactRangeOptions, including the required enum converters in the JNI portal. Significant changes: - Make CompactRangeOptions available in the compactRange() for Java. - Deprecate other compactRange() methods that have individual option params, like in the CPP code. - Migrate rocksdb_compactrange_helper() to CompactRangeOptions. - Add Java unit tests for CompactRangeOptions. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4220 Differential Revision: D9380007 Pulled By: sagar0 fbshipit-source-id: 6af6c334f221427f1997b33fb24c3986b092fed6
2018-08-17 17:52:58 +00:00
org.rocksdb.CompactRangeOptions\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.ComparatorOptions\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 19:04:09 +00:00
org.rocksdb.CompressionOptions\
Add a ConfigOptions for use in comparing objects and converting to/from strings (#6389) Summary: The methods in convenience.h are used to compare/convert objects to/from strings. There is a mishmash of parameters in use here with more needed in the future. This PR replaces those parameters with a single structure. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6389 Reviewed By: siying Differential Revision: D21163707 Pulled By: zhichao-cao fbshipit-source-id: f807b4cc7e2b0af3871536b69546b2604dfa81bd
2020-04-22 00:35:28 +00:00
org.rocksdb.ConfigOptions\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.DBOptions\
org.rocksdb.DirectSlice\
[RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava. Changes: - Extraced abstract super class Env - Introduced RocksMemEnv - Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java. - Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35619
2015-03-22 17:30:51 +00:00
org.rocksdb.Env\
Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248
2016-09-29 21:04:41 +00:00
org.rocksdb.EnvOptions\
[RocksJava] Flush functionality RocksJava now supports also flush functionality of RocksDB.
2014-11-09 19:09:39 +00:00
org.rocksdb.FlushOptions\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.Filter\
Replace deprecated RocksDB#addFile with RocksDB#ingestExternalFile Summary: Previously the Java implementation of `RocksDB#addFile` was both incomplete and not inline with the C++ API. Rather than fix it, as I see that `rocksdb::DB::AddFile` is now deprecated in favour of `rocksdb::DB::IngestExternalFile`, I have removed the old broken implementation and implemented `RocksDB#ingestExternalFile`. Closes https://github.com/facebook/rocksdb/issues/2261 Closes https://github.com/facebook/rocksdb/pull/2291 Differential Revision: D5061264 Pulled By: sagar0 fbshipit-source-id: 85df0899fa1b1fc3535175cac4f52353511d4104
2017-05-22 17:22:49 +00:00
org.rocksdb.IngestExternalFileOptions\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.HashLinkedListMemTableConfig\
org.rocksdb.HashSkipListMemTableConfig\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.HdfsEnv\
Implement Java API for ConcurrentTaskLimiter class and compaction_thread_limiter field in ColumnFamilyOptions (#7347) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7347 Test Plan: unit tests included Reviewed By: jay-zhuang Differential Revision: D23592552 Pulled By: pdillinger fbshipit-source-id: 1c3571b6f42bfd0cfd723ff49d01fbc02a1be45b
2020-09-09 19:42:54 +00:00
org.rocksdb.ConcurrentTaskLimiter\
org.rocksdb.ConcurrentTaskLimiterImpl\
RocksJava - JNI Logger callback Summary: Within this commit a new AbstractLogger was introduced which allows to handle log messages at an application level. Log messages are passed up to Java using a JNI callback. This allows a Java-Developer to use common Java APIs for log messages e.g. SLF4J, LOG4J, etc. Within this commit no new dependencies were introduced, which keeps the RocksDB API clean and doesn`t force a developer to use a predefined high-level Java API. Another feature is to dynamically set a custom loggers verbosity at runtime using its public method `setInfoLogLevel` and to retrieve the currently active level using the `infoLogLevel` method. Test Plan: make clean jclean rocksdbjava jtest mvn -f rocksjni.pom package Reviewers: adamretter, ankgup87, yhchiang Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D34755
2015-03-10 22:16:21 +00:00
org.rocksdb.Logger\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 19:04:09 +00:00
org.rocksdb.LRUCache\
RocksJava: memory_util support (#4446) Summary: JNI passthrough for utilities/memory/memory_util.cc sagar0 adamretter Pull Request resolved: https://github.com/facebook/rocksdb/pull/4446 Differential Revision: D10174578 Pulled By: sagar0 fbshipit-source-id: d1d196d771dff22afb7ef7500f308233675696f8
2018-10-08 18:03:34 +00:00
org.rocksdb.MemoryUsageType\
org.rocksdb.MemoryUtil\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.MergeOperator\
Java wrapper for Native Comparators Summary: This is an abstraction for working with custom Comparators implemented in native C++ code from Java. Native code must directly extend `rocksdb::Comparator`. When the native code comparator is compiled into the RocksDB codebase, you can then create a Java Class, and JNI stub to wrap it. Useful if the C++/JNI barrier overhead is too much for your applications comparator performance. An example is provided in `java/rocksjni/native_comparator_wrapper_test.cc` and `java/src/main/java/org/rocksdb/NativeComparatorWrapperTest.java`. Closes https://github.com/facebook/rocksdb/pull/3334 Differential Revision: D7172605 Pulled By: miasantreble fbshipit-source-id: e24b7eb267a3bcb6afa214e0379a1d5e8a2ceabe
2018-03-08 19:16:46 +00:00
org.rocksdb.NativeComparatorWrapper\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 18:22:38 +00:00
org.rocksdb.OptimisticTransactionDB\
org.rocksdb.OptimisticTransactionOptions\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.Options\
Expose LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs in RocksJava Summary: JNI wrappers for LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs. Closes https://github.com/facebook/rocksdb/pull/2898 Differential Revision: D5857934 Pulled By: sagar0 fbshipit-source-id: 68b79e83eab8de9416e3f1fef73e11cf7947e90a
2017-09-22 00:15:27 +00:00
org.rocksdb.OptionsUtil\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.PersistentCache\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.PlainTableConfig\
[RocksJava] Adjusted RateLimiter to 3.10.0 (#1368) Summary: - Deprecated RateLimiterConfig and GenericRateLimiterConfig - Introduced RateLimiter It is now possible to use all C++ related methods also in RocksJava. A noteable method is setBytesPerSecond which can change the allowed number of bytes per second at runtime. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35715
2016-10-07 19:32:21 +00:00
org.rocksdb.RateLimiter\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.ReadOptions\
Java facility to use the RemoveEmptyValueCompactionFilter
2015-07-14 16:48:16 +00:00
org.rocksdb.RemoveEmptyValueCompactionFilter\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.RestoreOptions\
Added CompactionFilterFactory support to RocksJava Summary: This PR also includes some cleanup, bugfixes and refactoring of the Java API. However these are really pre-cursors on the road to CompactionFilterFactory support. Closes https://github.com/facebook/rocksdb/pull/1241 Differential Revision: D6012778 Pulled By: sagar0 fbshipit-source-id: 0774465940ee99001a78906e4fed4ef57068ad5c
2017-10-12 18:06:51 +00:00
org.rocksdb.RocksCallbackObject\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.RocksDB\
org.rocksdb.RocksEnv\
org.rocksdb.RocksIterator\
[RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava. Changes: - Extraced abstract super class Env - Introduced RocksMemEnv - Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java. - Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35619
2015-03-22 17:30:51 +00:00
org.rocksdb.RocksMemEnv\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.SkipListMemTableConfig\
org.rocksdb.Slice\
Added support for SstFileManager to RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/3666 Differential Revision: D7457634 Pulled By: sagar0 fbshipit-source-id: 47741e2ee66e9255c580f4e38cfb86b284c27c2f
2018-04-07 04:22:37 +00:00
org.rocksdb.SstFileManager\
Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248
2016-09-29 21:04:41 +00:00
org.rocksdb.SstFileWriter\
Added support for SstFileReader JNI interface (#5556) Summary: Feature request as per https://github.com/facebook/rocksdb/issues/5538 issue. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5556 Differential Revision: D17219008 fbshipit-source-id: e31f18dec318416eac9dea8213bab31da96e1f3a
2019-09-10 01:11:02 +00:00
org.rocksdb.SstFileReader\
org.rocksdb.SstFileReaderIterator\
SST Partitioner interface that allows to split SST files (#6957) Summary: SST Partitioner interface that allows to split SST files during compactions. It basically instruct compaction to create a new file when needed. When one is using well defined prefixes and prefixed way of defining tables it is good to define also partitioning so that promotion of some SST file does not cover huge key space on next level (worst case complete space). Pull Request resolved: https://github.com/facebook/rocksdb/pull/6957 Reviewed By: ajkr Differential Revision: D22461239 fbshipit-source-id: 9ce07bba08b3ba89c2d45630520368f704d1316e
2020-07-24 20:43:14 +00:00
org.rocksdb.SstPartitionerFactory\
org.rocksdb.SstPartitionerFixedPrefixFactory\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.Statistics\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.ThreadStatus\
org.rocksdb.TimedEnv\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 18:22:38 +00:00
org.rocksdb.Transaction\
org.rocksdb.TransactionDB\
org.rocksdb.TransactionDBOptions\
org.rocksdb.TransactionOptions\
[RocksJava] GetUpdatesSince support
2015-01-25 21:05:29 +00:00
org.rocksdb.TransactionLogIterator\
[RocksJava] TTL-Support
2014-11-17 18:22:44 +00:00
org.rocksdb.TtlDB\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.VectorMemTableConfig\
[RocksJava] Snapshot - GetSequenceNumber Summary: As the C++ part exposes now SequenceNumber retrieval for Snapshots we want this obviously also in the Java API. Test Plan: make rocksdbjava make jtest mvn -f rocksjni.pom test Reviewers: yhchiang, adamretter, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D32571
2015-01-30 20:05:45 +00:00
org.rocksdb.Snapshot\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.StringAppendOperator\
Add UInt64AddOperator to rocksjava (#4448) Summary: Closes https://github.com/facebook/rocksdb/issues/4447 Pull Request resolved: https://github.com/facebook/rocksdb/pull/4448 Differential Revision: D10351852 Pulled By: ajkr fbshipit-source-id: 18287b5190ae0b8153ce425da9a0bdfe1af88c34
2018-10-13 00:33:55 +00:00
org.rocksdb.UInt64AddOperator\
[RocksJava] Makefile restructured
2014-11-09 19:08:35 +00:00
org.rocksdb.WriteBatch\
org.rocksdb.WriteBatch.Handler\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.WriteOptions\
Implement WriteBatchWithIndex in the Java API
2015-01-03 14:52:07 +00:00
org.rocksdb.WriteBatchWithIndex\
Plumb WriteBufferManager through JNI (#4492) Summary: Allow rocks java to explicitly create WriteBufferManager by plumbing it to the native code through JNI. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4492 Differential Revision: D10428506 Pulled By: sagar0 fbshipit-source-id: cd9dd8c2ef745a0303416b44e2080547bdcca1fd
2018-10-17 18:47:51 +00:00
org.rocksdb.WriteBufferManager\
Implement WriteBatchWithIndex in the Java API
2015-01-03 14:52:07 +00:00
org.rocksdb.WBWIRocksIterator
Add more options to backupable DB
2014-05-14 05:22:21 +00:00
Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273)
2016-08-22 18:01:42 +00:00
NATIVE_JAVA_TEST_CLASSES = org.rocksdb.RocksDBExceptionTest\
Java wrapper for Native Comparators Summary: This is an abstraction for working with custom Comparators implemented in native C++ code from Java. Native code must directly extend `rocksdb::Comparator`. When the native code comparator is compiled into the RocksDB codebase, you can then create a Java Class, and JNI stub to wrap it. Useful if the C++/JNI barrier overhead is too much for your applications comparator performance. An example is provided in `java/rocksjni/native_comparator_wrapper_test.cc` and `java/src/main/java/org/rocksdb/NativeComparatorWrapperTest.java`. Closes https://github.com/facebook/rocksdb/pull/3334 Differential Revision: D7172605 Pulled By: miasantreble fbshipit-source-id: e24b7eb267a3bcb6afa214e0379a1d5e8a2ceabe
2018-03-08 19:16:46 +00:00
org.rocksdb.NativeComparatorWrapperTest.NativeStringComparatorWrapper\
Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273)
2016-08-22 18:01:42 +00:00
org.rocksdb.WriteBatchTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.WriteBatchTestInternalHelper
RocksJava Makefile includes incorrect paths to version.h
2014-10-07 18:43:04 +00:00
ROCKSDB_MAJOR = $(shell egrep "ROCKSDB_MAJOR.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3)
ROCKSDB_MINOR = $(shell egrep "ROCKSDB_MINOR.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3)
ROCKSDB_PATCH = $(shell egrep "ROCKSDB_PATCH.[0-9]" ../include/rocksdb/version.h | cut -d ' ' -f 3)
add major minor micro version to java jars
2014-10-02 18:07:45 +00:00
Add a jni library for rocksdb which supports Open, Get, Put, and Close. Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 21:19:21 +00:00
NATIVE_INCLUDE = ./include
Update RocksDB's Java bindings to support multiple native RocksDB builds in the same Jar file. Cross build RocksDB for linux32 and linux64 using Vagrant. Build a cross-platform fat jar that contains osx, linux32, and linux64 RocksDB static builds.
2014-09-26 20:57:12 +00:00
ARCH := $(shell getconf LONG_BIT)
add major minor micro version to java jars
2014-10-02 18:07:45 +00:00
ROCKSDB_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-linux$(ARCH).jar
Update RocksDB's Java bindings to support multiple native RocksDB builds in the same Jar file. Cross build RocksDB for linux32 and linux64 using Vagrant. Build a cross-platform fat jar that contains osx, linux32, and linux64 RocksDB static builds.
2014-09-26 20:57:12 +00:00
ifeq ($(PLATFORM), OS_MACOSX)
add major minor micro version to java jars
2014-10-02 18:07:45 +00:00
ROCKSDB_JAR = rocksdbjni-$(ROCKSDB_MAJOR).$(ROCKSDB_MINOR).$(ROCKSDB_PATCH)-osx.jar
Update RocksDB's Java bindings to support multiple native RocksDB builds in the same Jar file. Cross build RocksDB for linux32 and linux64 using Vagrant. Build a cross-platform fat jar that contains osx, linux32, and linux64 RocksDB static builds.
2014-09-26 20:57:12 +00:00
endif
Add a jni library for rocksdb which supports Open, Get, Put, and Close. Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 21:19:21 +00:00
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 20:28:25 +00:00
JAVA_TESTS = \
org.rocksdb.BackupableDBOptionsTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 06:19:53 +00:00
org.rocksdb.BackupEngineTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.BlockBasedTableConfigTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 20:28:25 +00:00
org.rocksdb.BuiltinComparatorTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 06:19:53 +00:00
org.rocksdb.util.BytewiseComparatorTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 20:28:25 +00:00
org.rocksdb.util.BytewiseComparatorIntTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.CheckPointTest\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 19:04:09 +00:00
org.rocksdb.ClockCacheTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.ColumnFamilyOptionsTest\
org.rocksdb.ColumnFamilyTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.CompactionFilterFactoryTest\
org.rocksdb.CompactionJobInfoTest\
org.rocksdb.CompactionJobStatsTest\
org.rocksdb.CompactionOptionsTest\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 19:04:09 +00:00
org.rocksdb.CompactionOptionsFIFOTest\
org.rocksdb.CompactionOptionsUniversalTest\
org.rocksdb.CompactionPriorityTest\
org.rocksdb.CompactionStopStyleTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.ComparatorOptionsTest\
org.rocksdb.CompressionOptionsTest\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 19:04:09 +00:00
org.rocksdb.CompressionTypesTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.DBOptionsTest\
org.rocksdb.DirectSliceTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 20:28:25 +00:00
org.rocksdb.util.EnvironmentTest\
Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248
2016-09-29 21:04:41 +00:00
org.rocksdb.EnvOptionsTest\
Add event listeners to RocksJava (#7425) Summary: Allows adding event listeners in RocksJava. * Adds listeners getter and setter in `Options` and `DBOptions` classes. * Adds `EventListener` Java interface and base class for implementing custom event listener callbacks - `AbstractEventListener`, which has an underlying native callback class implementing C++ `EventListener` class. * `AbstractEventListener` class has mechanism for selectively enabling its callback methods in order to prevent invoking Java method if it is not implemented. This decreases performance cost in case only subset of event listener callback methods is needed - the JNI code for remaining "no-op" callbacks is not executed. * The code is covered by unit tests in `EventListenerTest.java`, there are also tests added for setting/getting listeners field in `OptionsTest.java` and `DBOptionsTest.java`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/7425 Reviewed By: pdillinger Differential Revision: D24063390 Pulled By: jay-zhuang fbshipit-source-id: 508c359538983d6b765e70d9989c351794a944ee
2020-10-14 18:32:10 +00:00
org.rocksdb.EventListenerTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.HdfsEnvTest\
Replace deprecated RocksDB#addFile with RocksDB#ingestExternalFile Summary: Previously the Java implementation of `RocksDB#addFile` was both incomplete and not inline with the C++ API. Rather than fix it, as I see that `rocksdb::DB::AddFile` is now deprecated in favour of `rocksdb::DB::IngestExternalFile`, I have removed the old broken implementation and implemented `RocksDB#ingestExternalFile`. Closes https://github.com/facebook/rocksdb/issues/2261 Closes https://github.com/facebook/rocksdb/pull/2291 Differential Revision: D5061264 Pulled By: sagar0 fbshipit-source-id: 85df0899fa1b1fc3535175cac4f52353511d4104
2017-05-22 17:22:49 +00:00
org.rocksdb.IngestExternalFileOptionsTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 20:28:25 +00:00
org.rocksdb.util.IntComparatorTest\
org.rocksdb.util.JNIComparatorTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.FilterTest\
org.rocksdb.FlushTest\
org.rocksdb.InfoLogLevelTest\
org.rocksdb.KeyMayExistTest\
Implement Java API for ConcurrentTaskLimiter class and compaction_thread_limiter field in ColumnFamilyOptions (#7347) Summary: as title Pull Request resolved: https://github.com/facebook/rocksdb/pull/7347 Test Plan: unit tests included Reviewed By: jay-zhuang Differential Revision: D23592552 Pulled By: pdillinger fbshipit-source-id: 1c3571b6f42bfd0cfd723ff49d01fbc02a1be45b
2020-09-09 19:42:54 +00:00
org.rocksdb.ConcurrentTaskLimiterTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 06:19:53 +00:00
org.rocksdb.LoggerTest\
Expose LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs in RocksJava Summary: JNI wrappers for LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs. Closes https://github.com/facebook/rocksdb/pull/2898 Differential Revision: D5857934 Pulled By: sagar0 fbshipit-source-id: 68b79e83eab8de9416e3f1fef73e11cf7947e90a
2017-09-22 00:15:27 +00:00
org.rocksdb.LRUCacheTest\
RocksJava: memory_util support (#4446) Summary: JNI passthrough for utilities/memory/memory_util.cc sagar0 adamretter Pull Request resolved: https://github.com/facebook/rocksdb/pull/4446 Differential Revision: D10174578 Pulled By: sagar0 fbshipit-source-id: d1d196d771dff22afb7ef7500f308233675696f8
2018-10-08 18:03:34 +00:00
org.rocksdb.MemoryUtilTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.MemTableTest\
org.rocksdb.MergeTest\
org.rocksdb.MixedOptionsTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 06:19:53 +00:00
org.rocksdb.MutableColumnFamilyOptionsTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.MutableDBOptionsTest\
Java wrapper for Native Comparators Summary: This is an abstraction for working with custom Comparators implemented in native C++ code from Java. Native code must directly extend `rocksdb::Comparator`. When the native code comparator is compiled into the RocksDB codebase, you can then create a Java Class, and JNI stub to wrap it. Useful if the C++/JNI barrier overhead is too much for your applications comparator performance. An example is provided in `java/rocksjni/native_comparator_wrapper_test.cc` and `java/src/main/java/org/rocksdb/NativeComparatorWrapperTest.java`. Closes https://github.com/facebook/rocksdb/pull/3334 Differential Revision: D7172605 Pulled By: miasantreble fbshipit-source-id: e24b7eb267a3bcb6afa214e0379a1d5e8a2ceabe
2018-03-08 19:16:46 +00:00
org.rocksdb.NativeComparatorWrapperTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 06:19:53 +00:00
org.rocksdb.NativeLibraryLoaderTest\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 18:22:38 +00:00
org.rocksdb.OptimisticTransactionTest\
org.rocksdb.OptimisticTransactionDBTest\
org.rocksdb.OptimisticTransactionOptionsTest\
Expose LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs in RocksJava Summary: JNI wrappers for LoadLatestOptions, LoadOptionsFromFile and GetLatestOptionsFileName APIs. Closes https://github.com/facebook/rocksdb/pull/2898 Differential Revision: D5857934 Pulled By: sagar0 fbshipit-source-id: 68b79e83eab8de9416e3f1fef73e11cf7947e90a
2017-09-22 00:15:27 +00:00
org.rocksdb.OptionsUtilTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.OptionsTest\
org.rocksdb.PlainTableConfigTest\
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 00:26:12 +00:00
org.rocksdb.RateLimiterTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.ReadOnlyTest\
org.rocksdb.ReadOptionsTest\
Improve RocksJava Comparator (#6252) Summary: This is a redesign of the API for RocksJava comparators with the aim of improving performance. It also simplifies the class hierarchy. **NOTE**: This breaks backwards compatibility for existing 3rd party Comparators implemented in Java... so we need to consider carefully which release branches this goes into. Previously when implementing a comparator in Java the developer had a choice of subclassing either `DirectComparator` or `Comparator` which would use direct and non-direct byte-buffers resepectively (via `DirectSlice` and `Slice`). In this redesign there we have eliminated the overhead of using the Java Slice classes, and just use `ByteBuffer`s. The `ComparatorOptions` supplied when constructing a Comparator allow you to choose between direct and non-direct byte buffers by setting `useDirect`. In addition, the `ComparatorOptions` now allow you to choose whether a ByteBuffer is reused over multiple comparator calls, by setting `maxReusedBufferSize > 0`. When buffers are reused, ComparatorOptions provides a choice of mutex type by setting `useAdaptiveMutex`. --- [JMH benchmarks previously indicated](https://github.com/facebook/rocksdb/pull/6241#issue-356398306) that the difference between C++ and Java for implementing a comparator was ~7x slowdown in Java. With these changes, when reusing buffers and guarding access to them via mutexes the slowdown is approximately the same. However, these changes offer a new facility to not reuse mutextes, which reduces the slowdown to ~5.5x in Java. We also offer a `thread_local` mechanism for reusing buffers, which reduces slowdown to ~5.2x in Java (closes https://github.com/facebook/rocksdb/pull/4425). These changes also form a good base for further optimisation work such as further JNI lookup caching, and JNI critical. --- These numbers were captured without jemalloc. With jemalloc, the performance improves for all tests, and the Java slowdown reduces to between 4.8x and 5.x. ``` ComparatorBenchmarks.put native_bytewise thrpt 25 124483.795 ± 2032.443 ops/s ComparatorBenchmarks.put native_reverse_bytewise thrpt 25 114414.536 ± 3486.156 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 17228.250 ± 1288.546 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 16035.865 ± 1248.099 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_reused-64_thread-local thrpt 25 21571.500 ± 871.521 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_adaptive-mutex thrpt 25 23613.773 ± 8465.660 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 16768.172 ± 5618.489 ops/s ComparatorBenchmarks.put java_bytewise_direct_reused-64_thread-local thrpt 25 23921.164 ± 8734.742 ops/s ComparatorBenchmarks.put java_bytewise_non-direct_no-reuse thrpt 25 17899.684 ± 839.679 ops/s ComparatorBenchmarks.put java_bytewise_direct_no-reuse thrpt 25 22148.316 ± 1215.527 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_adaptive-mutex thrpt 25 11311.126 ± 820.602 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_non-adaptive-mutex thrpt 25 11421.311 ± 807.210 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_reused-64_thread-local thrpt 25 11554.005 ± 960.556 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_adaptive-mutex thrpt 25 22960.523 ± 1673.421 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_non-adaptive-mutex thrpt 25 18293.317 ± 1434.601 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_reused-64_thread-local thrpt 25 24479.361 ± 2157.306 ops/s ComparatorBenchmarks.put java_reverse_bytewise_non-direct_no-reuse thrpt 25 7942.286 ± 626.170 ops/s ComparatorBenchmarks.put java_reverse_bytewise_direct_no-reuse thrpt 25 11781.955 ± 1019.843 ops/s ``` Pull Request resolved: https://github.com/facebook/rocksdb/pull/6252 Differential Revision: D19331064 Pulled By: pdillinger fbshipit-source-id: 1f3b794e6a14162b2c3ffb943e8c0e64a0c03738
2020-02-03 20:28:25 +00:00
org.rocksdb.util.ReverseBytewiseComparatorIntTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.RocksDBTest\
Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273)
2016-08-22 18:01:42 +00:00
org.rocksdb.RocksDBExceptionTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.DefaultEnvTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.RocksIteratorTest\
[RocksJava] Expose MemEnv in RocksJava Summary: In 3.10 the C++ code was extended with a MemEnv implementation. This is now also available in RocksJava. Changes: - Extraced abstract super class Env - Introduced RocksMemEnv - Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java. - Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent. Test Plan: make rocksdbjava make jtest Reviewers: adamretter, yhchiang, ankgup87 Subscribers: dhruba Differential Revision: https://reviews.facebook.net/D35619
2015-03-22 17:30:51 +00:00
org.rocksdb.RocksMemEnvTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.util.SizeUnitTest\
Add DB::OpenAsSecondary to RocksJava (#7047) Summary: Closes https://github.com/facebook/rocksdb/issues/5852 Pull Request resolved: https://github.com/facebook/rocksdb/pull/7047 Reviewed By: cheng-chang Differential Revision: D22335162 Pulled By: pdillinger fbshipit-source-id: 75f3c524deccea7ebc0ad288da41f1ea81406c1c
2020-07-06 18:47:00 +00:00
org.rocksdb.SecondaryDBTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.SliceTest\
org.rocksdb.SnapshotTest\
Added support for SstFileManager to RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/3666 Differential Revision: D7457634 Pulled By: sagar0 fbshipit-source-id: 47741e2ee66e9255c580f4e38cfb86b284c27c2f
2018-04-07 04:22:37 +00:00
org.rocksdb.SstFileManagerTest\
Add Java API for SstFileWriter Add Java API for SstFileWriter. Closes https://github.com/facebook/rocksdb/issues/1248
2016-09-29 21:04:41 +00:00
org.rocksdb.SstFileWriterTest\
Added support for SstFileReader JNI interface (#5556) Summary: Feature request as per https://github.com/facebook/rocksdb/issues/5538 issue. Pull Request resolved: https://github.com/facebook/rocksdb/pull/5556 Differential Revision: D17219008 fbshipit-source-id: e31f18dec318416eac9dea8213bab31da96e1f3a
2019-09-10 01:11:02 +00:00
org.rocksdb.SstFileReaderTest\
SST Partitioner interface that allows to split SST files (#6957) Summary: SST Partitioner interface that allows to split SST files during compactions. It basically instruct compaction to create a new file when needed. When one is using well defined prefixes and prefixed way of defining tables it is good to define also partitioning so that promotion of some SST file does not cover huge key space on next level (worst case complete space). Pull Request resolved: https://github.com/facebook/rocksdb/pull/6957 Reviewed By: ajkr Differential Revision: D22461239 fbshipit-source-id: 9ce07bba08b3ba89c2d45630520368f704d1316e
2020-07-24 20:43:14 +00:00
org.rocksdb.SstPartitionerTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.TableFilterTest\
org.rocksdb.TimedEnvTest\
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 18:22:38 +00:00
org.rocksdb.TransactionTest\
org.rocksdb.TransactionDBTest\
org.rocksdb.TransactionOptionsTest\
org.rocksdb.TransactionDBOptionsTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.TransactionLogIteratorTest\
org.rocksdb.TtlDBTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 06:19:53 +00:00
org.rocksdb.StatisticsTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.StatisticsCollectorTest\
Add missing functionality to RocksJava (#4833) Summary: This is my latest round of changes to add missing items to RocksJava. More to come in future PRs. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4833 Differential Revision: D14152266 Pulled By: sagar0 fbshipit-source-id: d6cff67e26da06c131491b5cf6911a8cd0db0775
2019-02-22 22:36:38 +00:00
org.rocksdb.WalFilterTest\
Added missing options to RocksJava Summary: This adds almost all missing options to RocksJava Closes https://github.com/facebook/rocksdb/pull/2039 Differential Revision: D4779991 Pulled By: siying fbshipit-source-id: 4a1bf28
2017-03-30 19:04:09 +00:00
org.rocksdb.WALRecoveryModeTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.WriteBatchHandlerTest\
org.rocksdb.WriteBatchTest\
Improve the design and native object management of Stats in RocksJava Summary: Closes https://github.com/facebook/rocksdb/pull/2551 Differential Revision: D5399288 Pulled By: sagar0 fbshipit-source-id: dd3df2ed6cc5ae612db0998ea746cc29fccf568e
2017-07-12 06:19:53 +00:00
org.rocksdb.WriteBatchThreadedTest\
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
org.rocksdb.WriteOptionsTest\
org.rocksdb.WriteBatchWithIndexTest
Moved Java main classes into src/main/java
2015-01-31 22:23:59 +00:00
MAIN_SRC = src/main/java
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
TEST_SRC = src/test/java
OUTPUT = target
Moved Java main classes into src/main/java
2015-01-31 22:23:59 +00:00
MAIN_CLASSES = $(OUTPUT)/classes
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
TEST_CLASSES = $(OUTPUT)/test-classes
Moved Java main classes into src/main/java
2015-01-31 22:23:59 +00:00
JAVADOC = $(OUTPUT)/apidocs
[RocksJava] Test-framework integration
2014-11-02 00:08:41 +00:00
Moved Java Benchmark main classes into benchmark/src/main/java
2015-01-31 22:42:13 +00:00
BENCHMARK_MAIN_SRC = benchmark/src/main/java
BENCHMARK_OUTPUT = benchmark/target
BENCHMARK_MAIN_CLASSES = $(BENCHMARK_OUTPUT)/classes
Moved Java Samples main classes into samples/src/main/java
2015-01-31 23:43:01 +00:00
SAMPLES_MAIN_SRC = samples/src/main/java
SAMPLES_OUTPUT = samples/target
SAMPLES_MAIN_CLASSES = $(SAMPLES_OUTPUT)/classes
Update test lib versions and maven plugin versions
2015-01-31 23:21:19 +00:00
JAVA_TEST_LIBDIR = test-libs
JAVA_JUNIT_JAR = $(JAVA_TEST_LIBDIR)/junit-4.12.jar
JAVA_HAMCR_JAR = $(JAVA_TEST_LIBDIR)/hamcrest-core-1.3.jar
JAVA_MOCKITO_JAR = $(JAVA_TEST_LIBDIR)/mockito-all-1.10.19.jar
JAVA_CGLIB_JAR = $(JAVA_TEST_LIBDIR)/cglib-2.2.2.jar
JAVA_ASSERTJ_JAR = $(JAVA_TEST_LIBDIR)/assertj-core-1.7.1.jar
JAVA_TESTCLASSPATH = $(JAVA_JUNIT_JAR):$(JAVA_HAMCR_JAR):$(JAVA_MOCKITO_JAR):$(JAVA_CGLIB_JAR):$(JAVA_ASSERTJ_JAR)
[RocksJava] Test-framework integration
2014-11-02 00:08:41 +00:00
Resolve Java test dependencies from local maven repo if present
2015-07-14 17:39:22 +00:00
MVN_LOCAL = ~/.m2/repository
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 00:26:12 +00:00
# Set the default JAVA_ARGS to "" for DEBUG_LEVEL=0
Force Java version on Travis CI (#6512) Summary: In the `.travis.yml` file the `jdk: openjdk7` element is ignored when `language: cpp`. So whatever version of the JDK that was installed in the Travis container was used - typically JDK 11. To ensure our RocksJava builds are working, we now instead install and use OpenJDK 8. Ideally we would use OpenJDK 7, as RocksJava supports Java 7, but many of the newer Travis containers don't support Java 7, so Java 8 is the next best thing. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6512 Differential Revision: D20388296 Pulled By: pdillinger fbshipit-source-id: 8bbe6b59b70cfab7fe81ff63867d907fefdd2df1
2020-03-12 19:22:03 +00:00
JAVA_ARGS ?=
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 00:26:12 +00:00
Force Java version on Travis CI (#6512) Summary: In the `.travis.yml` file the `jdk: openjdk7` element is ignored when `language: cpp`. So whatever version of the JDK that was installed in the Travis container was used - typically JDK 11. To ensure our RocksJava builds are working, we now instead install and use OpenJDK 8. Ideally we would use OpenJDK 7, as RocksJava supports Java 7, but many of the newer Travis containers don't support Java 7, so Java 8 is the next best thing. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6512 Differential Revision: D20388296 Pulled By: pdillinger fbshipit-source-id: 8bbe6b59b70cfab7fe81ff63867d907fefdd2df1
2020-03-12 19:22:03 +00:00
JAVAC_ARGS ?=
Add JAVAC_ARGS as Makefile variable Summary: Set to "-Xlint:deprecation -Xlint:unchecked" for non-zero DEBUG_LEVEL Shows up a few warnings for later perusal however passes ok. Closes https://github.com/facebook/rocksdb/pull/2557 Differential Revision: D5408104 Pulled By: sagar0 fbshipit-source-id: 02908602e09b46d33b0dfc0ffd4b11954db03c53
2017-07-12 19:04:55 +00:00
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 00:26:12 +00:00
# When debugging add -Xcheck:jni to the java args
ifneq ($(DEBUG_LEVEL),0)
Force Java version on Travis CI (#6512) Summary: In the `.travis.yml` file the `jdk: openjdk7` element is ignored when `language: cpp`. So whatever version of the JDK that was installed in the Travis container was used - typically JDK 11. To ensure our RocksJava builds are working, we now instead install and use OpenJDK 8. Ideally we would use OpenJDK 7, as RocksJava supports Java 7, but many of the newer Travis containers don't support Java 7, so Java 8 is the next best thing. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6512 Differential Revision: D20388296 Pulled By: pdillinger fbshipit-source-id: 8bbe6b59b70cfab7fe81ff63867d907fefdd2df1
2020-03-12 19:22:03 +00:00
JAVA_ARGS += -ea -Xcheck:jni
JAVAC_ARGS += -Xlint:deprecation -Xlint:unchecked
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 00:26:12 +00:00
endif
Use an Amazon S3 bucket for downloading deps (#6526) Summary: After we had a lot of failures with maven.org downloads, we wanted an alternative location for downloading binary dependencies. Hosting them through github would have been good in terms of organizational and network dependencies, but that approach seems to be awkward (fake releases, so would need a 'rocksdb-deps' repo) and strangely complicated for Facebook policy on open source repositories. This commit moves the downloads (that are not officially hosted by others on github) from my personal rocksdb fork to an S3 bucket owned by the Facebook RocksDB AWS account. Facebook employees can access this through an internal tool, and we should be able to grant permission to outside collaborators. Assuming this works out, I will back-port to older branches to stabilize their CI testing as well. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6526 Test Plan: CI Differential Revision: D20430130 Pulled By: pdillinger fbshipit-source-id: df52394a65e0a57942db3039bdaade8a4d520cb2
2020-03-13 20:36:45 +00:00
# Using a Facebook AWS account for S3 storage. (maven.org has a history
# of failing in Travis builds.)
DEPS_URL?=https://rocksdb-deps.s3-us-west-2.amazonaws.com/jars
AIX and Solaris Sparc Support Summary: Replacement of #2147 The change was squashed due to a lot of conflicts. Closes https://github.com/facebook/rocksdb/pull/2194 Differential Revision: D4929799 Pulled By: siying fbshipit-source-id: 5cd49c254737a1d5ac13f3c035f128e86524c581
2017-04-22 03:41:37 +00:00
Reduce the need to re-download dependencies (#6318) Summary: Both changes are related to RocksJava: 1. Allow dependencies that are already present on the host system due to Maven to be reused in Docker builds. 2. Extend the `make clean-not-downloaded` target to RocksJava, so that libraries needed as dependencies for the test suite are not deleted and re-downloaded unnecessarily. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6318 Differential Revision: D19608742 Pulled By: pdillinger fbshipit-source-id: 25e25649e3e3212b537ac4512b40e2e53dc02ae7
2020-01-29 16:00:16 +00:00
clean: clean-not-downloaded clean-downloaded
clean-not-downloaded:
$(AM_V_at)rm -rf $(NATIVE_INCLUDE)
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_at)rm -rf $(OUTPUT)
$(AM_V_at)rm -rf $(BENCHMARK_OUTPUT)
$(AM_V_at)rm -rf $(SAMPLES_OUTPUT)
[RocksJava] Make cleanup - Clean Target - Remove JNI includes on clean - Remove target folder generated by Maven - Remove shared object - Remove jar
2014-11-19 20:15:01 +00:00
Reduce the need to re-download dependencies (#6318) Summary: Both changes are related to RocksJava: 1. Allow dependencies that are already present on the host system due to Maven to be reused in Docker builds. 2. Extend the `make clean-not-downloaded` target to RocksJava, so that libraries needed as dependencies for the test suite are not deleted and re-downloaded unnecessarily. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6318 Differential Revision: D19608742 Pulled By: pdillinger fbshipit-source-id: 25e25649e3e3212b537ac4512b40e2e53dc02ae7
2020-01-29 16:00:16 +00:00
clean-downloaded:
$(AM_V_at)rm -rf $(JAVA_TEST_LIBDIR)
Add a jni library for rocksdb which supports Open, Get, Put, and Close. Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 21:19:21 +00:00
Fix java makefile dependencies Summary: Fix dependencies in java makefile, to avoid java build failure. Test Plan: run "make rocksdbjava" multiple times. Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64023
2016-09-16 17:54:31 +00:00
javadocs: java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_GEN)mkdir -p $(JAVADOC)
$(AM_V_at)javadoc -d $(JAVADOC) -sourcepath $(MAIN_SRC) -subpackages org
add javadoc and sources targets for sonatype
2014-10-02 20:57:54 +00:00
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
javalib: java java_test javadocs
[RocksJava] JavaDoc is executed too often Previous to this commit too much targets got dependencies on javadocs target. Introduced one additional target "javalib" which resolves that situation. JavaDoc will now be generated once while executing a task with prefix "rocksdbjava".
2014-11-19 20:21:21 +00:00
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
java:
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_GEN)mkdir -p $(MAIN_CLASSES)
Switch Travis to Xenial build (#4789) Summary: I think this should now also run on Travis's new virtualised infrastructure which affords more memory and CPU. We also need to think about migrating from travis-ci.org to travis-ci.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4789 Differential Revision: D15856272 fbshipit-source-id: 10b41d21924e8a362bc9646a63ccd1a5dfc437c6
2019-06-17 17:15:58 +00:00
ifeq ($(shell java -version 2>&1 | grep 1.7.0 > /dev/null; printf $$?), 0)
Add JAVAC_ARGS as Makefile variable Summary: Set to "-Xlint:deprecation -Xlint:unchecked" for non-zero DEBUG_LEVEL Shows up a few warnings for later perusal however passes ok. Closes https://github.com/facebook/rocksdb/pull/2557 Differential Revision: D5408104 Pulled By: sagar0 fbshipit-source-id: 02908602e09b46d33b0dfc0ffd4b11954db03c53
2017-07-12 19:04:55 +00:00
$(AM_V_at)javac $(JAVAC_ARGS) -d $(MAIN_CLASSES)\
Moved Java main classes into src/main/java
2015-01-31 22:23:59 +00:00
$(MAIN_SRC)/org/rocksdb/util/*.java\
$(MAIN_SRC)/org/rocksdb/*.java
Switch Travis to Xenial build (#4789) Summary: I think this should now also run on Travis's new virtualised infrastructure which affords more memory and CPU. We also need to think about migrating from travis-ci.org to travis-ci.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4789 Differential Revision: D15856272 fbshipit-source-id: 10b41d21924e8a362bc9646a63ccd1a5dfc437c6
2019-06-17 17:15:58 +00:00
else
$(AM_V_at)javac $(JAVAC_ARGS) -h $(NATIVE_INCLUDE) -d $(MAIN_CLASSES)\
$(MAIN_SRC)/org/rocksdb/util/*.java\
$(MAIN_SRC)/org/rocksdb/*.java
endif
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_at)@cp ../HISTORY.md ./HISTORY-CPP.md
$(AM_V_at)@rm -f ./HISTORY-CPP.md
Switch Travis to Xenial build (#4789) Summary: I think this should now also run on Travis's new virtualised infrastructure which affords more memory and CPU. We also need to think about migrating from travis-ci.org to travis-ci.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4789 Differential Revision: D15856272 fbshipit-source-id: 10b41d21924e8a362bc9646a63ccd1a5dfc437c6
2019-06-17 17:15:58 +00:00
ifeq ($(shell java -version 2>&1 | grep 1.7.0 > /dev/null; printf $$?), 0)
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_at)javah -cp $(MAIN_CLASSES) -d $(NATIVE_INCLUDE) -jni $(NATIVE_JAVA_CLASSES)
Switch Travis to Xenial build (#4789) Summary: I think this should now also run on Travis's new virtualised infrastructure which affords more memory and CPU. We also need to think about migrating from travis-ci.org to travis-ci.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4789 Differential Revision: D15856272 fbshipit-source-id: 10b41d21924e8a362bc9646a63ccd1a5dfc437c6
2019-06-17 17:15:58 +00:00
endif
Add a jni library for rocksdb which supports Open, Get, Put, and Close. Summary: This diff contains a simple jni library for rocksdb which supports open, get, put and closeusing default options (including Options, ReadOptions, and WriteOptions.) In the usual case, Java developers can use the c++ rocksdb library in the way similar to the following: RocksDB db = RocksDB.open(path_to_db); ... db.put("hello".getBytes(), "world".getBytes(); byte[] value = db.get("hello".getBytes()); ... db.close(); Specifically, this diff has the following major classes: * RocksDB: a Java wrapper class which forwards the operations from the java side to c++ rocksdb library. * RocksDBException: ncapsulates the error of an operation. This exception type is used to describe an internal error from the c++ rocksdb library. This diff also include a simple java sample code calling c++ rocksdb library. To build the rocksdb jni library, simply run make jni, and make jtest will try to build and run the sample code. Note that if the rocksdb is not built with the default glibc that Java uses, java will try to load the wrong glibc during the run time. As a result, the sample code might not work properly during the run time. Test Plan: * make jni * make jtest Reviewers: haobo, dhruba, sdong, igor, ljin Reviewed By: dhruba CC: leveldb, xjin Differential Revision: https://reviews.facebook.net/D17109
2014-03-28 21:19:21 +00:00
[JNI] Add an initial benchmark for java binding for rocksdb. Summary: * Add a benchmark for java binding for rocksdb. The java benchmark is a complete rewrite based on the c++ db/db_bench.cc and the DbBenchmark in dain's java leveldb. * Support multithreading. * 'readseq' is currently not supported as it requires RocksDB Iterator. * usage: --benchmarks Comma-separated list of operations to run in the specified order Actual benchmarks: fillseq -- write N values in sequential key order in async mode fillrandom -- write N values in random key order in async mode fillbatch -- write N/1000 batch where each batch has 1000 values in random key order in sync mode fillsync -- write N/100 values in random key order in sync mode fill100K -- write N/1000 100K values in random order in async mode readseq -- read N times sequentially readrandom -- read N times in random order readhot -- read N times in random order from 1% section of DB Meta Operations: delete -- delete DB DEFAULT: [fillseq, readrandom, fillrandom] --compression_ratio Arrange to generate values that shrink to this fraction of their original size after compression DEFAULT: 0.5 --use_existing_db If true, do not destroy the existing database. If you set this flag and also specify a benchmark that wants a fresh database, that benchmark will fail. DEFAULT: false --num Number of key/values to place in database. DEFAULT: 1000000 --threads Number of concurrent threads to run. DEFAULT: 1 --reads Number of read operations to do. If negative, do --nums reads. --key_size The size of each key in bytes. DEFAULT: 16 --value_size The size of each value in bytes. DEFAULT: 100 --write_buffer_size Number of bytes to buffer in memtable before compacting (initialized to default value by 'main'.) DEFAULT: 4194304 --cache_size Number of bytes to use as a cache of uncompressed data. Negative means use default settings. DEFAULT: -1 --seed Seed base for random number generators. DEFAULT: 0 --db Use the db with the following name. DEFAULT: /tmp/rocksdbjni-bench * Add RocksDB.write(). Test Plan: make jbench Reviewers: haobo, sdong, dhruba, ankgup87 Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17433
2014-04-09 07:48:20 +00:00
sample: java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
Add JAVAC_ARGS as Makefile variable Summary: Set to "-Xlint:deprecation -Xlint:unchecked" for non-zero DEBUG_LEVEL Shows up a few warnings for later perusal however passes ok. Closes https://github.com/facebook/rocksdb/pull/2557 Differential Revision: D5408104 Pulled By: sagar0 fbshipit-source-id: 02908602e09b46d33b0dfc0ffd4b11954db03c53
2017-07-12 19:04:55 +00:00
$(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/RocksDBSample.java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni_not_found
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 00:26:12 +00:00
java $(JAVA_ARGS) -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) RocksDBSample /tmp/rocksdbjni
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni_not_found
[JNI] Add java api and java tests for WriteBatch and WriteOptions, add put() and remove() to RocksDB. Summary: * Add java api for rocksdb::WriteBatch and rocksdb::WriteOptions, which are necessary components for running benchmark. * Add java test for org.rocksdb.WriteBatch and org.rocksdb.WriteOptions. * Add remove() to org.rocksdb.RocksDB, and add put() and remove() to RocksDB which take org.rocksdb.WriteOptions. Test Plan: make jtest Reviewers: haobo, sdong, dhruba Reviewed By: sdong CC: leveldb Differential Revision: https://reviews.facebook.net/D17373
2014-04-02 20:14:55 +00:00
[RocksJava] Incorporated review comments
2014-11-08 17:58:35 +00:00
column_family_sample: java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
Add JAVAC_ARGS as Makefile variable Summary: Set to "-Xlint:deprecation -Xlint:unchecked" for non-zero DEBUG_LEVEL Shows up a few warnings for later perusal however passes ok. Closes https://github.com/facebook/rocksdb/pull/2557 Differential Revision: D5408104 Pulled By: sagar0 fbshipit-source-id: 02908602e09b46d33b0dfc0ffd4b11954db03c53
2017-07-12 19:04:55 +00:00
$(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/RocksDBColumnFamilySample.java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 00:26:12 +00:00
java $(JAVA_ARGS) -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) RocksDBColumnFamilySample /tmp/rocksdbjni
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_at)@rm -rf /tmp/rocksdbjni
[RocksJava] Incorporated review comments
2014-11-08 17:58:35 +00:00
Add TransactionDB and OptimisticTransactionDB to the Java API Summary: Closes https://github.com/facebook/rocksdb/issues/697 Closes https://github.com/facebook/rocksdb/issues/1151 Closes https://github.com/facebook/rocksdb/pull/1298 Differential Revision: D7131402 Pulled By: sagar0 fbshipit-source-id: bcd34ce95ed88cc641786089ff4232df7b2f089f
2018-03-02 18:22:38 +00:00
transaction_sample: java
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
$(AM_V_at)javac -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/TransactionSample.java
$(AM_V_at)@rm -rf /tmp/rocksdbjni
java -ea -Xcheck:jni -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) TransactionSample /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni
optimistic_transaction_sample: java
$(AM_V_GEN)mkdir -p $(SAMPLES_MAIN_CLASSES)
$(AM_V_at)javac -cp $(MAIN_CLASSES) -d $(SAMPLES_MAIN_CLASSES) $(SAMPLES_MAIN_SRC)/OptimisticTransactionSample.java
$(AM_V_at)@rm -rf /tmp/rocksdbjni
java -ea -Xcheck:jni -Djava.library.path=target -cp $(MAIN_CLASSES):$(SAMPLES_MAIN_CLASSES) OptimisticTransactionSample /tmp/rocksdbjni
$(AM_V_at)@rm -rf /tmp/rocksdbjni
[RocksJava] Test-framework integration
2014-11-02 00:08:41 +00:00
resolve_test_deps:
java dependencies test -s -> use test -d Summary: To correct a build process where the JAVA_TEST_LIBDIR is a symlink to a cache directory. Test -s (size 0) on symlinks returns true, resulting in a mkdir over the top of the symlink resulting in failure. As a solution -d checks if it is a directory (or the symlink refers to a directory), which works in the case of real directories and symlinks to directories. Trivial I know but it was really easy for me to use a symlink here to prevent frequent downloads in a CI environment. Thanks for your consideration. Closes https://github.com/facebook/rocksdb/pull/1917 Differential Revision: D4612263 Pulled By: siying fbshipit-source-id: 4d458f8e1760068cdd6b5eae4bce6e12c400df41
2017-04-12 22:04:41 +00:00
test -d "$(JAVA_TEST_LIBDIR)" || mkdir -p "$(JAVA_TEST_LIBDIR)"
Don't download from (unreliable) maven.org (#6348) Summary: I set up a mirror of our Java deps on github so we can download them through github URLs rather than maven.org, which is proving terribly unreliable from Travis builds. Also sanitized calls to curl, so they are easier to read and appropriately fail on download failure. Pull Request resolved: https://github.com/facebook/rocksdb/pull/6348 Test Plan: CI Differential Revision: D19633621 Pulled By: pdillinger fbshipit-source-id: 7eb3f730953db2ead758dc94039c040f406790f3
2020-01-30 19:00:08 +00:00
test -s "$(JAVA_JUNIT_JAR)" || cp $(MVN_LOCAL)/junit/junit/4.12/junit-4.12.jar $(JAVA_TEST_LIBDIR) || curl --fail --insecure --output $(JAVA_JUNIT_JAR) --location $(DEPS_URL)/junit-4.12.jar
test -s "$(JAVA_HAMCR_JAR)" || cp $(MVN_LOCAL)/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar $(JAVA_TEST_LIBDIR) || curl --fail --insecure --output $(JAVA_HAMCR_JAR) --location $(DEPS_URL)/hamcrest-core-1.3.jar
test -s "$(JAVA_MOCKITO_JAR)" || cp $(MVN_LOCAL)/org/mockito/mockito-all/1.10.19/mockito-all-1.10.19.jar $(JAVA_TEST_LIBDIR) || curl --fail --insecure --output "$(JAVA_MOCKITO_JAR)" --location $(DEPS_URL)/mockito-all-1.10.19.jar
test -s "$(JAVA_CGLIB_JAR)" || cp $(MVN_LOCAL)/cglib/cglib/2.2.2/cglib-2.2.2.jar $(JAVA_TEST_LIBDIR) || curl --fail --insecure --output "$(JAVA_CGLIB_JAR)" --location $(DEPS_URL)/cglib-2.2.2.jar
test -s "$(JAVA_ASSERTJ_JAR)" || cp $(MVN_LOCAL)/org/assertj/assertj-core/1.7.1/assertj-core-1.7.1.jar $(JAVA_TEST_LIBDIR) || curl --fail --insecure --output "$(JAVA_ASSERTJ_JAR)" --location $(DEPS_URL)/assertj-core-1.7.1.jar
[RocksJava] Test-framework integration
2014-11-02 00:08:41 +00:00
Fix java makefile dependencies Summary: Fix dependencies in java makefile, to avoid java build failure. Test Plan: run "make rocksdbjava" multiple times. Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64023
2016-09-16 17:54:31 +00:00
java_test: java resolve_test_deps
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_GEN)mkdir -p $(TEST_CLASSES)
Switch Travis to Xenial build (#4789) Summary: I think this should now also run on Travis's new virtualised infrastructure which affords more memory and CPU. We also need to think about migrating from travis-ci.org to travis-ci.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4789 Differential Revision: D15856272 fbshipit-source-id: 10b41d21924e8a362bc9646a63ccd1a5dfc437c6
2019-06-17 17:15:58 +00:00
ifeq ($(shell java -version 2>&1|grep 1.7.0 >/dev/null; printf $$?),0)
Add JAVAC_ARGS as Makefile variable Summary: Set to "-Xlint:deprecation -Xlint:unchecked" for non-zero DEBUG_LEVEL Shows up a few warnings for later perusal however passes ok. Closes https://github.com/facebook/rocksdb/pull/2557 Differential Revision: D5408104 Pulled By: sagar0 fbshipit-source-id: 02908602e09b46d33b0dfc0ffd4b11954db03c53
2017-07-12 19:04:55 +00:00
$(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES):$(JAVA_TESTCLASSPATH) -d $(TEST_CLASSES)\
Moved Java main classes into src/main/java
2015-01-31 22:23:59 +00:00
$(TEST_SRC)/org/rocksdb/test/*.java\
$(TEST_SRC)/org/rocksdb/util/*.java\
$(TEST_SRC)/org/rocksdb/*.java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_at)javah -cp $(MAIN_CLASSES):$(TEST_CLASSES) -d $(NATIVE_INCLUDE) -jni $(NATIVE_JAVA_TEST_CLASSES)
Switch Travis to Xenial build (#4789) Summary: I think this should now also run on Travis's new virtualised infrastructure which affords more memory and CPU. We also need to think about migrating from travis-ci.org to travis-ci.com. Pull Request resolved: https://github.com/facebook/rocksdb/pull/4789 Differential Revision: D15856272 fbshipit-source-id: 10b41d21924e8a362bc9646a63ccd1a5dfc437c6
2019-06-17 17:15:58 +00:00
else
$(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES):$(JAVA_TESTCLASSPATH) -h $(NATIVE_INCLUDE) -d $(TEST_CLASSES)\
$(TEST_SRC)/org/rocksdb/test/*.java\
$(TEST_SRC)/org/rocksdb/util/*.java\
$(TEST_SRC)/org/rocksdb/*.java
endif
Moved Java test classes into src/test/java
2015-01-31 19:43:09 +00:00
Fix java makefile dependencies Summary: Fix dependencies in java makefile, to avoid java build failure. Test Plan: run "make rocksdbjava" multiple times. Reviewers: IslamAbdelRahman, sdong, yhchiang Reviewed By: yhchiang Subscribers: andrewkr, dhruba, leveldb Differential Revision: https://reviews.facebook.net/D64023
2016-09-16 17:54:31 +00:00
test: java java_test run_test
More granular steps in the Makefile, can help with running all or single Java tests (and with ASAN build - https://github.com/facebook/rocksdb/wiki/JNI-Debugging) (#1237)
2016-07-29 19:55:54 +00:00
run_test:
Fixed various memory leaks and Java 8 JNI Compatibility Summary: I have manually audited the entire RocksJava code base. Sorry for the large pull-request, I have broken it down into many small atomic commits though. My initial intention was to fix the warnings that appear when running RocksJava on Java 8 with `-Xcheck:jni`, for example when running `make jtest` you would see many errors similar to: ``` WARNING in native method: JNI call made without checking exceptions when required to from CallObjectMethod WARNING in native method: JNI call made without checking exceptions when required to from CallVoidMethod WARNING in native method: JNI call made without checking exceptions when required to from CallStaticVoidMethod ... ``` A few of those warnings still remain, however they seem to come directly from the JVM and are not directly related to RocksJava; I am in contact with the OpenJDK hostpot-dev mailing list about these - http://mail.openjdk.java.net/pipermail/hotspot-dev/2017-February/025981.html. As a result of fixing these, I realised we were not r Closes https://github.com/facebook/rocksdb/pull/1890 Differential Revision: D4591758 Pulled By: siying fbshipit-source-id: 7f7fdf4
2017-02-28 00:26:12 +00:00
java $(JAVA_ARGS) -Djava.library.path=target -cp "$(MAIN_CLASSES):$(TEST_CLASSES):$(JAVA_TESTCLASSPATH):target/*" org.rocksdb.test.RocksJunitRunner $(JAVA_TESTS)
[JNI] Add an initial benchmark for java binding for rocksdb. Summary: * Add a benchmark for java binding for rocksdb. The java benchmark is a complete rewrite based on the c++ db/db_bench.cc and the DbBenchmark in dain's java leveldb. * Support multithreading. * 'readseq' is currently not supported as it requires RocksDB Iterator. * usage: --benchmarks Comma-separated list of operations to run in the specified order Actual benchmarks: fillseq -- write N values in sequential key order in async mode fillrandom -- write N values in random key order in async mode fillbatch -- write N/1000 batch where each batch has 1000 values in random key order in sync mode fillsync -- write N/100 values in random key order in sync mode fill100K -- write N/1000 100K values in random order in async mode readseq -- read N times sequentially readrandom -- read N times in random order readhot -- read N times in random order from 1% section of DB Meta Operations: delete -- delete DB DEFAULT: [fillseq, readrandom, fillrandom] --compression_ratio Arrange to generate values that shrink to this fraction of their original size after compression DEFAULT: 0.5 --use_existing_db If true, do not destroy the existing database. If you set this flag and also specify a benchmark that wants a fresh database, that benchmark will fail. DEFAULT: false --num Number of key/values to place in database. DEFAULT: 1000000 --threads Number of concurrent threads to run. DEFAULT: 1 --reads Number of read operations to do. If negative, do --nums reads. --key_size The size of each key in bytes. DEFAULT: 16 --value_size The size of each value in bytes. DEFAULT: 100 --write_buffer_size Number of bytes to buffer in memtable before compacting (initialized to default value by 'main'.) DEFAULT: 4194304 --cache_size Number of bytes to use as a cache of uncompressed data. Negative means use default settings. DEFAULT: -1 --seed Seed base for random number generators. DEFAULT: 0 --db Use the db with the following name. DEFAULT: /tmp/rocksdbjni-bench * Add RocksDB.write(). Test Plan: make jbench Reviewers: haobo, sdong, dhruba, ankgup87 Reviewed By: haobo CC: leveldb Differential Revision: https://reviews.facebook.net/D17433
2014-04-09 07:48:20 +00:00
db_bench: java
Fix the verbosity issue in Java makefile Summary: When running `make rocksdbjava V=1`, it currently does not print out the original command but something odd instead: mkdir -p jl/db This patch tries to fix this issue. Test Plan: make rocksdbjava V=1 Reviewers: igor, sdong, IslamAbdelRahman, meyering Reviewed By: meyering Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 19:38:17 +00:00
$(AM_V_GEN)mkdir -p $(BENCHMARK_MAIN_CLASSES)
Add JAVAC_ARGS as Makefile variable Summary: Set to "-Xlint:deprecation -Xlint:unchecked" for non-zero DEBUG_LEVEL Shows up a few warnings for later perusal however passes ok. Closes https://github.com/facebook/rocksdb/pull/2557 Differential Revision: D5408104 Pulled By: sagar0 fbshipit-source-id: 02908602e09b46d33b0dfc0ffd4b11954db03c53
2017-07-12 19:04:55 +00:00
$(AM_V_at)javac $(JAVAC_ARGS) -cp $(MAIN_CLASSES) -d $(BENCHMARK_MAIN_CLASSES) $(BENCHMARK_MAIN_SRC)/org/rocksdb/benchmark/*.java