Commit graph

709 commits

Author SHA1 Message Date
Yi Wu a958c26435 Rename jvalue to jval in rocksjni
Summary: jvalue shadows a global name in <jni.h>. Rename it to jval to fix java build.

Test Plan:
    JAVA_HOME=/usr/local/jdk-7u10-64 make rocksdbjava -j64

Reviewers: adamretter, yhchiang, IslamAbdelRahman

Reviewed By: yhchiang, IslamAbdelRahman

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D63981
2016-09-14 13:12:55 -07:00
xh931076284 3639f32889 Fix bug in UnScSigned-off-by: xh931076284 <931076284@qq.com> (#1336)
Fix HdfsEnv::UnSchedule() API error
2016-09-14 10:17:34 -07:00
Yi Wu 81747f1be6 Refactor MutableCFOptions
Summary:
* Change constructor of MutableCFOptions to depends only on ColumnFamilyOptions.
* Move `max_subcompactions`, `compaction_options_fifo` and `compaction_pri` to ImmutableCFOptions to make it clear that they are immutable.

Test Plan: existing unit tests.

Reviewers: yhchiang, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D63945
2016-09-13 21:11:59 -07:00
Adam Retter 22d88e24db Allow an offset as well as a length to be specified for byte[] operations in RocksJava JNI (#1264)
Test Plan: Execute the Java test suite

Reviewers: yhchiang

Subscribers: andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D61971
2016-09-12 11:51:08 -07:00
Yi Wu a88677d2cf Remove ImmutableCFOptions from public API
Summary: There's no reference to ImmutableCFOptions elsewhere in /include/rocksdb. ImmutableCFOptions was introduced in this commit (5665e5e285) but later its reference in /include/rocksdb/table.h is removed.

Test Plan:
  make all check

Reviewers: IslamAbdelRahman, sdong, yhchiang

Reviewed By: yhchiang

Subscribers: yhchiang, andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D63177
2016-09-02 14:16:31 -07:00
sdong 32149059f9 Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes
Summary: To reduce number of options, merge source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes.

Test Plan: Add two new unit tests. Run all existing tests, including jtest.

Reviewers: yhchiang, igor, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D59829
2016-09-01 14:33:24 -07:00
Adam Retter f85f99bf69 Fix the Windows build of RocksDB Java. Similar to https://github.com/facebook/rocksdb/issues/1220 (#1284) 2016-08-25 10:16:26 -07:00
Adam Retter 817eeb29b4 Add singleDelete to RocksJava (#1275)
* Rename RocksDB#remove -> RocksDB#delete to match C++ API; Added deprecated versions of RocksDB#remove for backwards compatibility.

* Add missing experimental feature RocksDB#singleDelete
2016-08-22 11:02:31 -07:00
Adam Retter ffdf6eee19 Add Status to RocksDBException so that meaningful function result Status from the C++ API isn't lost (#1273) 2016-08-22 11:01:42 -07:00
Alexander Jipa 0b63f51fbc fixes 1215: execute_process(COMMAND mkdir ${DIR}) fails to create a directory with cmake on Windows (#1219)
CMake - Use a platform-neutral mkdir function
2016-08-18 00:59:22 +01:00
Islam AbdelRahman dd7a748cff Fix java build
Summary: Fix the java build

Test Plan: make rocksdbjava -j64

Reviewers: yhchiang, sdong

Reviewed By: sdong

Subscribers: andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D62097
2016-08-15 14:03:33 -07:00
Adam Retter d367555027 Added further Java API options for controlling concurrent writes 2016-08-11 16:33:20 -07:00
Adam Retter ebdfe34cc4 Exposed further Java API options for controlling compaction 2016-08-11 16:33:20 -07:00
Adam Retter f4d986364e Added SetOptions support to RocksJava (#1243)
* [refactor] Split Java ColumnFamilyOptions into mutable and immutable and implement any missing immutable options

* [feature] Implement RocksDB#setOptions
2016-08-06 12:03:47 -07:00
Adam Retter 9253767a6a Correct geHistogramData() -> getHistogramData() (#1257) 2016-08-04 20:16:16 -07:00
Adam Retter 9ae92f50b2 More granular steps in the Makefile, can help with running all or single Java tests (and with ASAN build - https://github.com/facebook/rocksdb/wiki/JNI-Debugging) (#1237) 2016-07-29 12:55:54 -07:00
Islam AbdelRahman 7c01d65344 [Fix Java] Remove duplicate cases in LoggerJniCallback::Logv
Summary: Remove duplicate cases in LoggerJniCallback::Logv and fix complication error

Test Plan: make jclean rocksdbjava jtest -j4

Reviewers: adamretter, andrewkr, sdong

Reviewed By: sdong

Subscribers: andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D61335
2016-07-29 12:47:15 -07:00
Adam Retter 8796934af7 Added missing Java ReadOptions settings (#1109)
* Minor reorganisation

* Java API - Added missing ReadOptions settings
2016-07-29 12:12:03 -07:00
Adam Retter 811ee21112 Bugfix to ensure that logging can be achieved from threads that are not known to the JVM (#1106) 2016-07-28 16:54:06 -07:00
Alexander Jipa ac0d93b08f fixes 1217: rocksdbjni javac and javah execute_processes fail on windows (#1218)
picking execute_process command based on OS
2016-07-27 00:18:49 +01:00
Islam AbdelRahman 8745f013ff [Fix java build] Stop using non standard std::make_unique
Summary: std::make_unique is not standard and not always available, remove it

Test Plan: Run "make clean jclean rocksdbjava jtest -j8" on my mac

Reviewers: yhchiang, yiwu, sdong, andrewkr

Reviewed By: andrewkr

Subscribers: andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D61143
2016-07-26 13:18:31 -07:00
Alexander Jipa ee8bf2e41f fixes 1228: rockdbjni loadLibraryFromJarToTemp fails when file is already present (#1232)
overriding existing library in tmp folder
2016-07-24 11:19:00 +01:00
Alexander Jipa 12767b3130 fixes 1220: rocksjni build fails on Windows due to variable-size array declaration (#1223)
* fixes 1220: rocksjni build fails on Windows due to variable-size array declaration

using new keyword to create variable-size arrays in order to satisfy most of the compilers

* fixes 1220: rocksjni build fails on Windows due to variable-size array declaration

using unique_ptr keyword to create variable-size arrays in order to satisfy most of the compilers
2016-07-22 10:17:10 -07:00
Alexander Jipa 515b11ffab fixes #1210: rocksdb/java/CMakeLists.txt lacks cmake_minimum_required (#1214)
specifying minimum cmake version so that no warning has to be suppressed
2016-07-18 16:37:49 +02:00
Alexander Jipa 876cb8bfb4 fixes #1212: rocksdbjni maven build does not escape slashes in groovy script (#1213)
using gmaven property so that script is not broken by embedding unescaped Windows-style path
2016-07-17 09:43:34 +02:00
sdong 32df9733d1 Add options.write_buffer_manager: control total memtable size across DB instances
Summary: Add option write_buffer_manager to help users control total memory spent on memtables across multiple DB instances.

Test Plan: Add a new unit test.

Reviewers: yhchiang, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: adela, benj, sumeet, muthu, leveldb, andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D59925
2016-07-05 18:11:25 -07:00
sdong 7b79238b65 Deprectate filter_deletes
Summary: filter_deltes is not a frequently used feature. Remove it.

Test Plan: Run all test suites.

Reviewers: igor, yhchiang, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D59427
2016-06-17 10:30:47 -07:00
Adam Retter e3b1e3dfa6 Expose save points in Java WriteBatch and WBWI (#1092)
* Java API - Expose SetSavePoint and RollbackToRestorePoint for WriteBatch and WriteBatchWithIndex

* Minor cleanup
2016-06-14 09:51:14 -07:00
sdong 2a79af1c54 Fix Java Break Related to memtable bloom bits to size ratio change
Summary: Need to change several more places for the change to fix Java tests

Test Plan:
make jtest
under java, run "make db_bench"

Reviewers: yhchiang, andrewkr, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: leveldb, andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D59541
2016-06-10 16:54:24 -07:00
sdong 20699df843 memtable_prefix_bloom_bits -> memtable_prefix_bloom_bits_ratio and deprecate memtable_prefix_bloom_probes
Summary:
memtable_prefix_bloom_probes is not a critical option. Remove it to reduce number of options.
It's easier for users to make mistakes with memtable_prefix_bloom_bits, turn it to memtable_prefix_bloom_bits_ratio

Test Plan: Run all existing tests

Reviewers: yhchiang, igor, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: gunnarku, yoshinorim, MarkCallaghan, leveldb, andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D59199
2016-06-10 12:12:10 -07:00
Adam Retter 972c895c35 Previously WARN level logging became FATAL level logging in the Java API (#1089) 2016-06-07 22:35:57 -07:00
Adam Retter 774a6aa296 Java API - Rename geHistogramData -> getHistogramData (#1107)
Closes https://github.com/facebook/rocksdb/issues/835
2016-06-06 16:04:16 -07:00
Adam Retter a791a2cf2d Java API - Bugfix for native linking of Compaction Filter (#1099) 2016-06-03 10:24:03 -07:00
Adam Retter 72c73cdc8b Java API - Add missing HEADER_LEVEL logging (#1104) 2016-05-06 15:06:12 -07:00
Adam Retter 4d02bfa3a6 Add support for PauseBackgroundWork and ContinueBackgroundWork to the Java API (#1087)
Closes https://github.com/facebook/rocksdb/issues/1071
2016-05-06 15:04:13 -07:00
Andrew Kryczka 9385fd72c0 Delete deprecated backup classes in Java
Summary: Corresponding change to D56331.

Test Plan:
Now build succeeds:

  $ make jclean && make rocksdbjava

Reviewers: yhchiang, IslamAbdelRahman, adamretter

Reviewed By: adamretter

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D56913
2016-04-19 10:32:31 -07:00
Adam Retter 200654067a Merge pull request #1053 from adamretter/benchmark-java-comparator
Benchmark Java comparator vs C++ comparator
2016-04-01 13:53:15 -07:00
Marton Trencseni 9b51987521 Adding pin_l0_filter_and_index_blocks_in_cache feature and related fixes.
Summary:
When a block based table file is opened, if prefetch_index_and_filter is true, it will prefetch the index and filter blocks, putting them into the block cache.
What this feature adds: when a L0 block based table file is opened, if pin_l0_filter_and_index_blocks_in_cache is true in the options (and prefetch_index_and_filter is true), then the filter and index blocks aren't released back to the block cache at the end of BlockBasedTableReader::Open(). Instead the table reader takes ownership of them, hence pinning them, ie. the LRU cache will never push them out. Meanwhile in the table reader, further accesses will not hit the block cache, thus avoiding lock contention.

Test Plan:
'export TEST_TMPDIR=/dev/shm/ && DISABLE_JEMALLOC=1 OPT=-g make all valgrind_check -j32' is OK.
I didn't run the Java tests, I don't have Java set up on my devserver.

Reviewers: sdong

Reviewed By: sdong

Subscribers: andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D56133
2016-04-01 10:42:39 -07:00
Adam Retter 188bb2e7ad Fix formatting identified by arc lint 2016-03-01 16:00:45 +00:00
Adam Retter 0f2d2fcff6 Refactored tests to use try-with-resources 2016-03-01 16:00:45 +00:00
Adam Retter f8e02c7825 Deprecate org.rocksdb.AbstractNativeReference#dispose() and implement java.lang.AutoCloseable 2016-03-01 16:00:45 +00:00
Adam Retter 0f2fdfe23a Fix the javadoc and the formatting of the base-classes for objects with native references 2016-03-01 16:00:44 +00:00
Adam Retter c5af85ecad Fix a memory leak of Slice objects from org.rocksdb.WBWIRocksIterator#entry1 2016-02-10 14:44:39 +00:00
Adam Retter e84137c8ae Remove unnessecary java.util.List expense in JNI 2016-02-10 14:44:39 +00:00
Adam Retter 76e8beeeb9 Pass by pointer from/to Java from JNI not by object 2016-02-10 14:44:39 +00:00
Adam Retter 0e7e6f6e4b Improve Javadoc 2016-02-10 14:44:39 +00:00
Adam Retter 18eb563058 Improve the speed and synchronization around the construction of Java/JNI objects 2016-02-10 14:44:39 +00:00
Baraa Hamodi 21e95811d1 Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
Adam Retter 9ab269ab39 Threaded tests for WriteBatch 2016-02-03 20:15:49 +00:00
Tomas Kolda 52153930d8 Adding support for Windows JNI build - fix Java unit test for release build of JNI DLL 2016-02-02 09:45:18 +01:00
Tomas Kolda e2972803a6 Adding support for Windows JNI build 2016-02-02 09:11:37 +01:00
David Bernard d78c6b28c4 Changes for build on solaris
Makefile adjust paths for solaris build
Makefile enable _GLIBCXX_USE_C99 so that std::to_string is available
db_compaction_test.cc Initialise a variable to avoid a compilation error
db_impl.cc Include <alloca.h>
db_test.cc Include <alloca.h>
Environment.java recognise solaris envrionment
options_bulder.cc Make log unambiguous
geodb_impl.cc Make log and floor unambiguous
2016-01-19 04:45:21 +00:00
sdong b99d4276f2 Fix java test buid broken by 7d87f02799
Summary: 7d87f02799 changed WriteBatch::InsertInto(). Need to change it.

Test Plan: Run it.

Reviewers: yhchiang, ngbronson

Reviewed By: ngbronson

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D52353
2015-12-28 11:51:39 -08:00
charsyam c30b499541 fix typos in comments 2015-12-11 01:54:48 +09:00
Islam AbdelRahman a163cc2d5a Lint everything
Summary:
```
arc2 lint --everything
```

run the linter on the whole code repo to fix exisitng lint issues

Test Plan: make check -j64

Reviewers: sdong, rven, anthony, kradhakrishnan, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D50769
2015-11-16 12:56:21 -08:00
Sage Weil 543c12ab06 options: add recycle_log_file_num option
Signed-off-by: Sage Weil <sage@redhat.com>
2015-10-18 21:21:24 -04:00
sdong 35ad531be3 Seperate InternalIterator from Iterator
Summary:
Separate a new class InternalIterator from class Iterator, when the look-up is done internally, which also means they operate on key with sequence ID and type.

This change will enable potential future optimizations but for now InternalIterator's functions are still the same as Iterator's.
At the same time, separate the cleanup function to a separate class and let both of InternalIterator and Iterator inherit from it.

Test Plan: Run all existing tests.

Reviewers: igor, yhchiang, anthony, kradhakrishnan, IslamAbdelRahman, rven

Reviewed By: rven

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D48549
2015-10-13 15:32:13 -07:00
dmittendorf 1ddd91cd2c Fixed packaging of java crossbuild jar by forcing all compiled binaries to be output to the java/target directory. The uber crossbuild jar is then assembled within the java/target directory. 2015-10-12 20:29:58 -04:00
James Lent 9e819d0963 Modify the way java static builds are done so that:
1) There is no need to download and install the compression libraries twice
just to get access to their header files during the compile phase.
2) Ensure that the compression library headers files used during the
compile phase are the same ones used to build the static library that
is linked into the library.
2015-10-12 18:24:51 -04:00
James Lent 5a7222782a Ensure that the compression libraries are statically linked into dynamic libraries included
in the Java jar.  Also build the linux libraries using the portable flag to fix a problem with
the linux32 build and improve the general portability of the RocksDB dynamic libraries.
==> linux32: util/crc32c.cc:318:39: error: ‘_mm_crc32_u64’ was not declared in this scope
2015-10-09 14:41:40 -04:00
Vivek Nelamangala a52888ed00 Install snappy headers to standard locations using yum, so that build_tools/build_detect_platform sets -DSNAPPY flag to g++ . Current jars of rocksdb do no have snappy compression avaliable . 2015-10-08 14:52:22 -07:00
Yueh-Hsuan Chiang ff57c6511d [RocksJava] Fix test failure of InfoLogLevelTest
Summary:
In patch https://reviews.facebook.net/D47067, we change the
log level of the initial database information to header level.
As a result, even when the InfoLogLevel is set to Fatal, the
LOG file of a newly opened rocksdb instance will not be empty.
However, the current InfoLogLevelTest expect it should be empty.

This patch fixes this issue by enabling InfoLogLevelTest to
ignore the Log header.

Test Plan: make jtest

Reviewers: fyrz, anthony, IslamAbdelRahman, sdong, adamretter

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D47229
2015-09-21 18:26:21 -07:00
Adam Retter 7d937a0901 Exit with non-zero status if one or more Java tests fail 2015-09-18 15:26:01 +01:00
Yueh-Hsuan Chiang 2e8e7eb391 Fix the verbosity issue in Java makefile
Summary:
When running `make rocksdbjava V=1`, it currently does
not print out the original command but something odd instead:

    mkdir -p jl/db

This patch tries to fix this issue.

Test Plan: make rocksdbjava V=1

Reviewers: igor, sdong, IslamAbdelRahman, meyering

Reviewed By: meyering

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D47061
2015-09-16 12:38:17 -07:00
Yueh-Hsuan Chiang 0e6e5472e1 Fixed a compile warning in rocksjni/loggerjnicallback.cc
Summary:
This patch fixes the following compile warning.
    java/rocksjni/loggerjnicallback.cc: In constructor ‘rocksdb::LoggerJniCallback::LoggerJniCallback(JNIEnv*, jobject)’:
    java/rocksjni/loggerjnicallback.cc:19:14: warning: unused variable ‘rs’ [-Wunused-variable]
       const jint rs = env->GetJavaVM(&m_jvm);
                  ^

Test Plan: make rocksdbjavastaticrelease

Reviewers: sdong, anthony, IslamAbdelRahman, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D45981
2015-09-02 15:07:44 -07:00
Yueh-Hsuan Chiang aad0572f83 Fixed the build issue of rocksdbjavastaticrelease
Summary: This patch fixed couple build issues of rocksdbjavastaticrelease.

Test Plan: make rocksdbjavastaticrelease

Reviewers: sdong, anthony, IslamAbdelRahman, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D45915
2015-09-01 11:07:43 -07:00
Yueh-Hsuan Chiang 40cd91b7fe Fixed compile warning in rocksdbjava
Summary:
Fixed the following compile warning in rocksdbjava:

    java/rocksjni/comparatorjnicallback.cc:20:14: warning: unused variable ‘rs’ [-Wunused-variable]
       const jint rs = env->GetJavaVM(&m_jvm);
                  ^
    java/rocksjni/comparatorjnicallback.cc: In member function ‘JNIEnv* rocksdb::BaseComparatorJniCallback::getJniEnv() const’:
    java/rocksjni/comparatorjnicallback.cc:45:8: warning: unused variable ‘rs’ [-Wunused-variable]
       jint rs = m_jvm->AttachCurrentThread(reinterpret_cast<void **>(&env), NULL);
            ^
    java/rocksjni/loggerjnicallback.cc: In constructor ‘rocksdb::LoggerJniCallback::LoggerJniCallback(JNIEnv*, jobject)’:
    java/rocksjni/loggerjnicallback.cc:19:14: warning: unused variable ‘rs’ [-Wunused-variable]
       const jint rs = env->GetJavaVM(&m_jvm);
                  ^
    java/rocksjni/loggerjnicallback.cc: In member function ‘JNIEnv* rocksdb::LoggerJniCallback::getJniEnv() const’:
    java/rocksjni/loggerjnicallback.cc:33:8: warning: unused variable ‘rs’ [-Wunused-variable]
       jint rs = m_jvm->AttachCurrentThread(reinterpret_cast<void **>(&env), NULL);
            ^

Test Plan: make rocksdbjava -j32

Reviewers: sdong, anthony, IslamAbdelRahman, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D45891
2015-08-31 18:36:17 -07:00
Yueh-Hsuan Chiang 102ac118b2 Update JAVA-HISTORY.md for v3.13
Summary: Update JAVA-HISTORY.md for v3.13

Test Plan: no code change.

Reviewers: igor, anthony, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43539
2015-08-04 18:12:58 -07:00
Yueh-Hsuan Chiang f5d072ae6c Fixed RocksJava test failure of shouldSetTestCappedPrefixExtractor
Summary:
Fixed RocksJava test failure of shouldSetTestCappedPrefixExtractor
by adding the missing native implementation of
useCappedPrefixExtractor.

Test Plan:
make jclean
make rocksdbjava -j32
make jtest

Reviewers: igor, anthony, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43551
2015-08-04 17:01:08 -07:00
Yueh-Hsuan Chiang f39cbcb0a5 Merge pull request #654 from adamretter/remove-emptyvalue-compactionfilter
RemoveEmptyValueCompactionFilter
2015-08-04 16:54:14 -07:00
ashishn f0b5bcc7b5 add support for capped prefix extractor in java 2015-08-04 16:46:23 -07:00
Yueh-Hsuan Chiang ce21afd205 Expose the BackupEngine from the Java API
Summary:
Merge pull request #665 by adamretter

Exposes BackupEngine from C++ to the Java API. Previously only BackupableDB was available

Test Plan: BackupEngineTest.java

Reviewers: fyrz, igor, ankgup87, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D42873
2015-08-04 16:08:54 -07:00
Andres Noetzli 6a82fba75f Add missing hashCode() implementation
Summary:
Whenever a Java class implements equals(), it has to implement hashCode(), otherwise
there might be weird behavior when inserting instances of the class in a hash map for
example. This adds two missing hashCode() implementations and extends tests to test
the hashCode() implementations.

Test Plan: make jtest

Reviewers: rven, igor, sdong, yhchiang

Reviewed By: yhchiang

Subscribers: anthony, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43017
2015-07-24 17:07:19 -07:00
Andres Noetzli f73c801432 Fixing Java tests.
Summary:
While working on https://reviews.facebook.net/D43017 , I realized
that some Java tests are failing due to a deprecated option.
This patch removes the offending tests, adds @Deprecated annotations
to the Java interface and removes the corresponding functions in
rocksjni

Test Plan: make jtest (all tests are passing now)

Reviewers: rven, igor, sdong, anthony, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D43035
2015-07-24 16:56:26 -07:00
Igor Canadi 35ca59364c Don't let flushes preempt compactions
Summary:
When we first started, max_background_flushes was 0 by default and compaction thread was executing flushes (since there was no flush thread). Then, we switched the default max_background_flushes to 1. However, we still support the case where there is no flush thread and flushes are done in compaction. This is making our code a bit more complicated. By not supporting this use-case we can make our code simpler.

We have a special case that when you set max_background_flushes to 0, we
schedule the flush to execute on the compaction thread.

Test Plan: make check (there might be some unit tests that depend on this behavior)

Reviewers: IslamAbdelRahman, yhchiang, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D41931
2015-07-17 12:02:52 -07:00
agiardullo 79373c372d Fix ROCKSDB_WARNING
Summary:
ROCKSDB_WARNING is only defined if either ROCKSDB_PLATFORM_POSIX or OS_WIN is defined.  This works well for building rocksdb with its own build scripts.  But this won't work when an outside project(like mongodb) doesn't define ROCKSDB_PLATFORM_POSIX.

This fix defines ROCKSDB_WARNING for all platforms.  No idea if its defined correctly on non-posix,non-windows platforms but this is no worse that the current situation where this macro is missing on unexpected platforms.

This fix should hopefully fix anyone whose build broke now that we've switched from using #warning to Pragma (to support windows).  Unfortunately, while mongo-rocks compiles, it ignores the Pragma and doesn't print a warning.  I have not been able to figure out a way to implement this portably on all platforms.

Of course, an alternate solution would be to just get rid of ROCKSDB_WARNING and live with include file redirects indefinitely.  Thoughts?

Test Plan: build rocks, build mongorocks

Reviewers: igor, kradhakrishnan, IslamAbdelRahman, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D42477
2015-07-17 11:04:55 -07:00
Adam Retter 91bf1b80ef Java facility to use the RemoveEmptyValueCompactionFilter 2015-07-16 11:50:11 +01:00
Adam Retter 3d00271e40 The ability to specify a compaction filter via the Java API 2015-07-16 11:50:10 +01:00
Jörg Maier 05e194158c Merge pull request #639 from cleaton/setMaxTableFileSize
Add JNI interface to set max_table_files_size for FIFO compaction
2015-07-14 23:23:06 +02:00
Adam Retter 9d22a97379 Resolve Java test dependencies from local maven repo if present 2015-07-14 18:42:33 +01:00
Yueh-Hsuan Chiang 03d433ee65 [RocksJava] Fixed test failures
Summary:
The option bottommost_level_compaction was introduced lately.
This option breaks the Java API behavior. To prevent the library
from doing so we set that option to a fixed value in Java.

In future we are going to remove that portion and replace the
hardcoded options using a more flexible way.

Fixed bug introduced by WriteBatchWithIndex Patch

Lately icanadi changed the behavior of WriteBatchWithIndex.
See commit: 821cff114e

This commit solves problems introduced by above mentioned commit.

Test Plan:
make rocksdbjava
make jtest

Reviewers: adamretter, ankgup87, yhchiang

Reviewed By: yhchiang

Subscribers: igor, dhruba

Differential Revision: https://reviews.facebook.net/D40647
2015-07-01 23:22:03 -07:00
Yueh-Hsuan Chiang c00948d5e1 [RocksJava] Fix test failure of compactRangeToLevel
Summary:
Rewrite Java tests compactRangeToLevel and compactRangeToLevelColumnFamily
to make them more deterministic and robust.

Test Plan:
make rocksdbjava
make jtest

Reviewers: anthony, fyrz, adamretter, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D40941
2015-07-01 16:03:56 -07:00
Jesper Lundgren dda74111ae add setMaxTableFilesSize Options unit test 2015-06-20 16:24:09 +08:00
Jesper Lundgren d62b6ed838 add setMaxTableFilesSize to JNI interface 2015-06-20 16:23:22 +08:00
Islam AbdelRahman 12e030a992 Use CompactRangeOptions for CompactRange
Summary:
This diff update DB::CompactRange to use RangeCompactionOptions instead of using multiple parameters
Old CompactRange is still available but deprecated

Test Plan:
make all check
make rocksdbjava
USE_CLANG=1 make all
OPT=-DROCKSDB_LITE make release

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D40209
2015-06-17 14:36:14 -07:00
agiardullo dc9d70de65 Optimistic Transactions
Summary: Optimistic transactions supporting begin/commit/rollback semantics.  Currently relies on checking the memtable to determine if there are any collisions at commit time.  Not yet implemented would be a way of enuring the memtable has some minimum amount of history so that we won't fail to commit when the memtable is empty.  You should probably start with transaction.h to get an overview of what is currently supported.

Test Plan: Added a new test, but still need to look into stress testing.

Reviewers: yhchiang, igor, rven, sdong

Reviewed By: sdong

Subscribers: adamretter, MarkCallaghan, leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D33435
2015-05-29 14:36:35 -07:00
agiardullo c815351038 Support saving history in memtable_list
Summary:
For transactions, we are using the memtables to validate that there are no write conflicts.  But after flushing, we don't have any memtables, and transactions could fail to commit.  So we want to someone keep around some extra history to use for conflict checking.  In addition, we want to provide a way to increase the size of this history if too many transactions fail to commit.

After chatting with people, it seems like everyone prefers just using Memtables to store this history (instead of a separate history structure).  It seems like the best place for this is abstracted inside the memtable_list.  I decide to create a separate list in MemtableListVersion as using the same list complicated the flush/installalflushresults logic too much.

This diff adds a new parameter to control how much memtable history to keep around after flushing.  However, it sounds like people aren't too fond of adding new parameters.  So I am making the default size of flushed+not-flushed memtables be set to max_write_buffers.  This should not change the maximum amount of memory used, but make it more likely we're using closer the the limit.  (We are now postponing deleting flushed memtables until the max_write_buffer limit is reached).  So while we might use more memory on average, we are still obeying the limit set (and you could argue it's better to go ahead and use up memory now instead of waiting for a write stall to happen to test this limit).

However, if people are opposed to this default behavior, we can easily set it to 0 and require this parameter be set in order to use transactions.

Test Plan: Added a xfunc test to play around with setting different values of this parameter in all tests.  Added testing in memtablelist_test and planning on adding more testing here.

Reviewers: sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D37443
2015-05-28 16:34:24 -07:00
Adam Retter 197f01b7b8 Bugfix remove deprecated option use which was removed in previous commit 019ecd1932 2015-04-30 23:16:04 +01:00
Adam Retter aa094e8ea3 Fix conversion from nano-seconds to milli-seconds and seconds 2015-04-30 23:15:33 +01:00
fyrz 019ecd1932 [RocksJava] Remove deprecated methods
Summary:
- Removed deprecated ColumnFamilyDescript constructor methods
- Removed deprecated skipLogErrorOnRecovery methods
- Removed deprecated tableCacheRemoveScanCountLimit methods

Test Plan:
make rocksdbjava
make jtest

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D37191
2015-04-16 00:31:43 +02:00
fyrz 5b7131c751 [RocksJava] Removed deprecated skipLogErrorOnRecovery methods.
As annonunced these options are not used anymore so these are
removed from code.
2015-04-15 23:37:24 +02:00
fyrz 566f652716 [RocksJava] Removed deprecated ColumnFamilyDescriptor methods
As announced previously removed methods are obsolete and will
be replaced by its byte array pendants.
2015-04-15 23:31:59 +02:00
fyrz 582c4b0f74 [RocksJava] Fix RateLimiter Tests in 3.10 2015-04-15 23:19:41 +02:00
Yueh-Hsuan Chiang 727684bf97 Fixed a typo in RocksDBSample.java
Summary:
Fixed a typo in RocksDBSample.java

Test Plan:
make clean
make rocksdbjava -j32
make jtest
2015-03-25 11:09:30 -07:00
Yueh-Hsuan Chiang 2d417e52df Update HISTORY.md for 3.10.0
Summary: Update HISTORY.md for 3.10.0

Test Plan: no code chagne.

Reviewers: sdong, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35871
2015-03-24 16:39:39 -07:00
fyrz 8f104ae5e0 [RocksJava] Maven build fix 2015-03-24 22:52:30 +01:00
Yueh-Hsuan Chiang afc51649e2 Merge pull request #546 from fyrz/RocksJava-MemEnv
[RocksJava] Expose MemEnv in RocksJava
2015-03-24 14:01:12 -07:00
Yueh-Hsuan Chiang 6284eef4c0 Merge pull request #545 from fyrz/RocksJava-Level-Compression
[RocksJava] Add compression per level to API
2015-03-24 12:14:09 -07:00
Yueh-Hsuan Chiang 8d8656243e Merge pull request #551 from fyrz/RocksJava-JavaDoc-Fix
[RocksJava] Add missing JavaDoc annotations
2015-03-23 13:25:37 -07:00
fyrz 46443bfa94 [RocksJava] Add missing JavaDoc annotations 2015-03-23 21:17:20 +01:00
fyrz 864b7e88f4 [RocksJava] Java sample bugfix
One of the latest commits broke the sample code. This
resolves the introduced bug.
2015-03-23 21:03:28 +01:00
fyrz 2b2394cbbb [RocksJava] DBBenchmark option for RocksMemEnv
Extended DBBenchmark with option to enable RocksMemEnv.
2015-03-23 18:59:32 +01:00
fyrz fd8804f979 [RocksJava] Expose MemEnv in RocksJava
Summary:
In 3.10 the C++ code was extended with a MemEnv implementation. This
is now also available in RocksJava.

Changes:
- Extraced abstract super class Env
- Introduced RocksMemEnv
- Remove unnecessary disposeInternal method. The disposal of the default environment is managed by C++ so there needs to be no disposeInternal method in Java.
- Introduced a RocksMemEnvTest, which is aligned with the C++ equivalent.

Test Plan:
make rocksdbjava
make jtest

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35619
2015-03-23 18:59:31 +01:00
fyrz 004b89fba4 [RocksJava] Add compression per level to API
Summary:
RocksDB offers the possibility to set different compression types
on a per level basis. This shall be also available using RocksJava.

Test Plan:
make rocksdbjava
make jtest

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35577
2015-03-23 18:58:56 +01:00
Adam Retter 51da3aab4a Merge pull request #536 from fyrz/RocksJava-32-Bit-adjustment
[RocksJava] 32-Bit adjustments
2015-03-22 23:33:35 +00:00
fyrz 39f4271be0 [RocksJava] Enhanced Logger comment
Added information about performance penalties using a custom
logger implementation.
2015-03-19 22:01:00 +01:00
fyrz 5615e23d8c [RocksJava] 32-Bit adjustments
Summary:
Before this change overflowing size_t values led to a checked Exception.

After that change:

size_t overflows on 32-Bit architecture throw now an IllegalArgumentException,
which removes the necessity for a developer to catch these Exceptions explicitly.

This is especially an advantage for developers targeting 64-Bit systems because
it is not necessary anymore to catch exceptions which are never thrown on a 64-Bit
system.

Test Plan:
make clean jclean rocksdbjava jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34923
2015-03-19 20:45:55 +01:00
Adam Retter 836bcc2f00 Merge pull request #532 from fyrz/RocksJava-LevelCompactionDynamicLevelBytes
[RocksJava] Added LevelCompactionDynamicLevelBytes to Options
2015-03-19 12:51:40 +00:00
Yueh-Hsuan Chiang f7ed654641 Fix RocksJava test failure due to deprecation of table_cache_remove_scan_count_limit
Summary:
table_cache_remove_scan_count_limit is marked as deprecated in RocksDB C++.
(see rocksdb/options.h).  This patch made necessary changes on RocksJava
side.

Test Plan:
make rocksdbjava -j32
make jtest

Reviewers: rven, igor, fyrz, adamretter, ankgup87, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35355
2015-03-18 15:40:27 -07:00
Yueh-Hsuan Chiang 652db51a31 Fix compilation error in rocksjni/write_batch_test.cc
Summary: Fix compilation error in rocksjni/write_batch_test.cc

Test Plan: make rocksdbjava -j32

Reviewers: sdong, igor, rven

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D35343
2015-03-18 15:10:57 -07:00
fyrz c345d1ee88 [RocksJava] Integrated changes for D34857 2015-03-18 21:38:17 +01:00
fyrz 12350115da [RocksJava] Added LevelCompactionDynamicLevelBytes to Options
Summary: Added LevelCompactionDynamicLevelBytes to Options.

Test Plan:
make clean jclean rocksdbjava jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, ankgup87, yhchiang

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34857
2015-03-18 21:38:17 +01:00
Igor Canadi c88ff4ca76 Deprecate removeScanCountLimit in NewLRUCache
Summary: It is no longer used by the implementation, so we should also remove it from the public API.

Test Plan: make check

Reviewers: sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D34971
2015-03-17 15:04:37 -07:00
Adam Retter eafa1bfc3c Merge pull request #529 from fyrz/RocksJava-Logger
[RockJava] Custom logger addition
2015-03-16 10:38:15 +00:00
fyrz 04778a94c5 [RocksJava] OptimizeFiltersForHits
Summary: Added optimize_filters_for_hits option.

Test Plan:
make clean jclean rocksdbjava jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D35013
2015-03-14 21:00:15 +01:00
fyrz 57f2a00c6f RocksJava - JNI Logger callback
Summary:
Within this commit a new AbstractLogger was introduced
which allows to handle log messages at an application level.

Log messages are passed up to Java using a JNI callback.

This allows a Java-Developer to use common Java APIs for log
messages e.g. SLF4J, LOG4J, etc. Within this commit no new
dependencies were introduced, which keeps the RocksDB API clean
and doesn`t force a developer to use a predefined high-level Java API.

Another feature is to dynamically set a custom loggers verbosity at
runtime using its public method `setInfoLogLevel` and to retrieve
the currently active level using the `infoLogLevel` method.

Test Plan:
make clean jclean rocksdbjava jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, ankgup87, yhchiang

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34755
2015-03-14 20:57:18 +01:00
fyrz 814627af3d [RocksJava] Change log level at runtime in custom logger
This commit introduces the possibility to change the log level
of a used custom logger at runtime.
2015-03-14 20:57:18 +01:00
fyrz a3bd4142f2 [RocksJava] Logging JNI callback
Within this commit a new AbstractLogger was introduced
which pushes info log messages all the way up to Java.
2015-03-14 20:57:18 +01:00
Adam Retter 756532daf5 Merge pull request #524 from fyrz/RocksJava-Test-Fix
[RocksJava] Fix JTest issues with enabled assertions.

Closes https://github.com/facebook/rocksdb/issues/523
2015-03-13 12:45:04 +00:00
Adam Retter 47a2b3a406 Merge pull request #534 from fyrz/RocksJava-Fix-BrokenJacocoReport
[RocksJava] Fix broken jacoco report
2015-03-13 12:39:36 +00:00
fyrz 2dc636f62b [RocksJava] Fix broken jacoco report
With the last folder layout change a bug was introduced which
prevents Jacoco from working correctly.
2015-03-12 21:28:58 +01:00
fyrz f210b0f6c5 [RocksJava] Fix NativeLibraryLoader
- Resolve problem while using a temporary data folder
- Fix test
2015-03-12 20:13:09 +01:00
fyrz 0d13bbe272 RocksJava] Fix ColumnFamily tests
Summary:
Cleaned up some tests regarding disposal order and tests
which were failing when C++ assertions were enabled.

Test Plan:
- Enable C++ Assertions (remove e.g. -DNDebug in rocksdbjava target)
- make rocksdbjava jtest

Reviewers: adamretter, ankgup87, yhchiang

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D34569
2015-03-05 21:13:46 +01:00
fyrz 67533809fd [RocksJava] Fixed CompactionTest 2015-03-04 22:45:18 +01:00
fyrz 1b7b997b88 [RocksJava] Remove MaxValue from Statistics 2015-03-04 22:45:17 +01:00
fyrz f862b38124 [RocksJava] Fix cleanup in tests 2015-03-04 22:45:16 +01:00
fyrz a01b592597 [RocksJava] DefaultColumnFamily Memory Fix
In the current implementation DefaultColumnFamily will not disown
the native handle. As the database handles the lease on the native
handle this should be the case.
2015-03-04 22:45:16 +01:00
fyrz 22c73d15b2 [RocksJava] Fix ColumnFamily tests
Cleanup column family memory cleanup
2015-03-04 22:45:15 +01:00
fyrz 9fcf1a7b00 [RocksJava] RocksJava Testcases
- NativeLibraryLoader Test
- EnvironmentTest Bugfix
2015-02-26 22:35:26 +01:00
Adam Retter b74ad66320 Merge pull request #508 from fyrz/RocksJava-Final-Patch
[RocksJava] Final usage correction
2015-02-26 13:40:05 +00:00
fyrz 8c76844742 [RocksJava] Updated TickerTypes and Histogram to 3.10
Updated tickers and histogram types to 3.10.
2015-02-25 23:34:10 +01:00
fyrz 7f0c77cb31 [RocksJava] Integrated changes from D33417 2015-02-25 21:55:32 +01:00
fyrz 819e787bb9 [RocksJava] Final usage correction
Summary:
Introduced final keyword to parameters with immutable values and classes
which should not be derived.

Test Plan:
make rocksdbjava
make jtest

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33417
2015-02-25 21:55:31 +01:00
Adam Retter d85993998c Merge pull request #506 from fyrz/RocksJava-Raw-Use
[RocksJava] Raw use of parametrized class
2015-02-24 20:11:42 +00:00
Adam Retter 30e93c9b9c Merge pull request #505 from fyrz/RocksJava-Redundant-Modifier
[RocksJava] Redundant access-modifier in interfaces
2015-02-24 20:03:00 +00:00
fyrz 6a0e737d9d [RocksJava] Raw use of parametrized class
Resolved raw use of parametrized class issues.
2015-02-12 18:23:25 +01:00
fyrz 4397012707 [RocksJava] Redundant access-modifier in interfaces
Removed redundant access modifier in interfaces.
2015-02-12 17:58:53 +01:00
fyrz eaf39568e5 [RocksJava] FlushOptions Correction
SetWaitForFlush did not set flush properly.
2015-02-12 17:31:38 +01:00
fyrz 5e8e453d59 [RocksJava] Integrated changes from D33165 2015-02-10 21:12:27 +01:00
fyrz 677d02427f [RocksJava] CF Name shall handle bytes correctly
Summary:
Bytes are currently misinterpreted by the Java if the
byte array contains zero bytes within its content. For Strings
thats usually not useful. As the Java API allows every kind
of byte array values it might be the case that zero padding might
happen.

Test Plan:
make rocksdbjava
make jtest

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D33165
2015-02-10 21:00:00 +01:00
Yueh-Hsuan Chiang bbb52b21f3 Merge pull request #483 from adamretter/restructure-java-build
Restructure java build
2015-02-09 14:53:11 +08:00
Adam Retter f33f3955ee Moved Java Samples main classes into samples/src/main/java 2015-02-01 20:05:13 +00:00
Adam Retter ad325517fc Update test lib versions and maven plugin versions 2015-02-01 20:05:10 +00:00
Adam Retter d6187d07bb Maven can now build a standard project layout 2015-02-01 20:05:10 +00:00
Adam Retter 157768890c Moved Java Benchmark main classes into benchmark/src/main/java 2015-02-01 20:05:10 +00:00
Adam Retter dd8d5471ea Adjustment to NativeLibraryLoader to allow native library to be loaded
from either java.library.path or from extracting from the Jar. Means
that the test in the build do not need to rely on the Jar, useful when
creating similar builds (and executing tests) from Maven
2015-02-01 20:05:10 +00:00
Adam Retter 353db6daef Moved Java main classes into src/main/java 2015-02-01 20:04:51 +00:00
Adam Retter 98cb501bc0 Moved Java test classes into src/test/java 2015-02-01 19:30:31 +00:00
fyrz 7479a62a7a Release.md - Remove version change instrcution
The version change instruction is obsolete with the change
that maven pulls versioning information from version.h.
2015-02-01 14:32:51 +01:00
Jörg Maier 4a4e4279f0 Update HISTORY-JAVA.md 2015-01-31 16:13:06 +01:00
fyrz ca52a67cfb [RocksJava] Deprecate setSkipLogErrorOnRecovery
- see: 62ad0a9b19
2015-01-31 15:59:57 +01:00
fyrz cb5c3159f0 [RocksJava] Snapshot - GetSequenceNumber
Summary:
As the C++ part exposes now SequenceNumber retrieval
for Snapshots we want this obviously also in the Java API.

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom test

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32571
2015-01-31 15:50:34 +01:00
fyrz 391f85fc82 [RocksJava] Incorporated changes for D32151 2015-01-31 15:15:49 +01:00
fyrz 68cd93b873 [RocksJava] GetUpdatesSince support
Summary:
This differential describes further changes to the Java-API

New methods:

* GetUpdatesSince
* GetLatestSequenceNumber
* EnableFileDeletions
* DisableFileDeletions

This pull requests depends on: https://github.com/facebook/rocksdb/pull/472

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom package

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32151
2015-01-31 15:15:48 +01:00
fyrz caedd40ddd [RocksJava] Adjusted auto pointer 2015-01-31 15:15:47 +01:00
fyrz b39006e3db [RocksJava] enable/disable File deletions 2015-01-31 15:15:47 +01:00
fyrz 9a456fba20 [RocksJava] GetUpdatesSince support 2015-01-31 15:15:46 +01:00
fyrz 939bb36597 [RocksJava] Fix ColumnFamily name alloc in TTL DB
While fixing the RocksDB ColumnFamily name the TTL DB
wasn`t touched. This commit resolves this.
2015-01-31 14:43:21 +01:00
fyrz cc0d8be011 [RocksJava] Integrated review comments (D32145) 2015-01-28 21:54:01 +01:00
fyrz ca2b00277e [RocksJava] Cleanup portal.h & tests
Summary:
Simple Java Native Objects usually are represented using
the same functionality but within different classes.

With this commit a template class was introduced to remove
the redundant impelementation to a certain extent.

[RocksJava] Removed todo comment in portal.h

As jclass instances shall not be cached, both
todos are obsolete and can be removed.

[RocksJava] Add missing test to Makefile

[RocksJava] Added tests for uncovered methods

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32145
2015-01-27 21:14:48 +01:00
fyrz f8dc5c459f [RocksJava] Add missing test to Makefile 2015-01-27 21:14:48 +01:00
fyrz b3c1331488 [RocksJava] Removed todo comment in portal.h
As jclass instances shall not be cached, both
todos are obsolete and can be removed.
2015-01-27 21:14:47 +01:00
fyrz 7ffcc457ff [RocksJava] Cleanup portal.h
Simple Java Native Objects usually are represented using
the same functionality but within different classes.

With this commit a template class was introduced to remove
the redundant impelementation to a certain extent.
2015-01-27 21:14:47 +01:00
fyrz e61f38e5a0 [RocksJava] Fix native library loader
Summary:
Prior to this the native library loader instance didn`t
care about a state. So if library loading was called multiple
times, multiple copies of the shared object were put into
the tmp folder and loaded into the JVM.

This changed within this commit to the following behavior:

- library loading is now synchronized
- library is loaded within the first call
- if loading was successful the library loaded sets a flag
- every subsequent call checks for a boolean flag indicating if there was
  already a successful attempt

Test Plan:
- Execute example and watch tmp folder while the example is running

- After this patch only one shared object will be in the tmp folder

Usual tests:
- make rocksdbjava jtest
- mvn -f rocksjni.pom package

Reviewers: adamretter, ankgup87, yhchiang

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32133
2015-01-24 16:55:28 +01:00
fyrz bef7821f07 [RocksJava] ReadOptions support in Iterators
The methods:

- newIterator
- iterators

support now also ReadOptions. That allows a user of the Java API
to retrieve RocksIterator instances on a snapshot.
2015-01-23 08:02:55 +01:00
Yueh-Hsuan Chiang 912c52e825 Merge pull request #465 from fyrz/RocksJava-BlockBasedTable-FormatVersion
[RocksJava] Format version support in BlockBasedTableConfig
2015-01-22 16:00:42 -08:00
fyrz dd53428f8b Incorporated review comments
- added spaces between operators
- removed obsolete text in JavaDoc
2015-01-22 23:47:31 +01:00
fyrz 908258a4f2 [RocksJava] BlockBasedTableConfig 3.10
- Added support for format version in BlockBasedTableConfig
2015-01-22 23:46:38 +01:00
fyrz 2efe228499 [RocksJava] Incorporated changes for D31809 2015-01-22 23:37:45 +01:00
fyrz e204a5a16c [RocksJava] ColumnFamily name JNI correction
Summary:
Previous to this commit there was a problem with unterminated
String usage as jByteArrays are not zero terminated.

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom package

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D31809
2015-01-22 23:29:45 +01:00
fyrz 96264784d9 [RocksJava] ColumnFamily name JNI correction
Previous to this commit there was a problem with unterminated
String usage as jByteArrays are not zero terminated.
2015-01-22 23:29:45 +01:00
fyrz 4ffe0be414 [RocksJava] Integrated changes for D31449 2015-01-19 22:19:58 +01:00
fyrz e828567541 [RocksJava] Integrated changes from D31449 2015-01-17 23:52:43 +01:00
fyrz 859c54a03d [RocksJava] TTL-Support
Summary:
TTLDB Support exposed in Java-API. It is now
possible to open a datbase using the RocksDB time
to live feature.

Test Plan:
make rocksdbjava
make test
mvn -f rocksjni.pom package

@Adam please test mac osx compile

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba, adam

Differential Revision: https://reviews.facebook.net/D31449
2015-01-17 23:28:06 +01:00
fyrz 5ff8aec4db [RocksJava] TTL Support 2015-01-17 23:28:06 +01:00
fyrz ca47da9e63 [RocksJava] TTL-Support 2015-01-17 23:28:05 +01:00
fyrz ea25ff7158 [RocksJava] Integrated proposed simplificiation 2015-01-17 01:22:29 +01:00
fyrz d68e83c356 [RocksJava] DirectSlice String termination fix
DirectSlice fix for non terminated String copy. This lead sometimes
to problems with DirectSliceTest.
2015-01-17 01:15:49 +01:00
fyrz c75c02e7a2 [RocksJava] WriteBatchWithIndexTest fix
Previous to this commit identiy checks were performed. Now tests are
performed using equals - method as intended.
2015-01-15 21:56:46 +01:00
fyrz c787fb50b8 [RocksJava] JavaDoc errors in Java8
Some of the latest commits included illegal JavaDoc formattings.
2015-01-15 21:56:46 +01:00
Adam Retter 3d246c89cc Abstract duplicate code on key and value slice objects into generic methods 2015-01-14 21:16:06 +00:00
Adam Retter 2d0dd8db3b Implement WBWIRocksIterator for WriteBatchWithIndex in the Java API 2015-01-14 21:16:06 +00:00
Adam Retter de678b288e Abstractions for common iterator behaviour 2015-01-14 21:16:06 +00:00
Adam Retter e01acb3a04 Test for WriteBatchWithIndex#newIterator() 2015-01-14 21:16:06 +00:00
Adam Retter 56f24941ab Simplify the Java API by permitting WriteBatchWithIndex to be provided straight to RocksDB#write 2015-01-14 21:16:06 +00:00
Adam Retter 95d5f98487 Test for RocksDB#write(WriteBatchWithIndex) 2015-01-14 21:16:06 +00:00
Adam Retter ef5b34dee0 Implement WriteBatchWithIndex in the Java API 2015-01-14 21:16:05 +00:00
Adam Retter c6e5545612 Abstractions for common write batch behaviour 2015-01-14 21:16:05 +00:00
Adam Retter be905491bf Test for WriteBatchWithIndex#newIteratorWithBase(org.rocksdb.RocksIterator) 2015-01-14 21:16:05 +00:00
Adam Retter 2241e3f4d5 Extract the interface for a RocksIterator 2015-01-14 21:16:05 +00:00
Adam Retter a8cfa7ace8 Extract the interface for a WriteBatch 2015-01-14 21:16:05 +00:00
Adam Retter 45e43b81df Adds support for db->DefaultColumnFamily() to the Java API 2015-01-14 21:16:05 +00:00
fyrz 23ad5f401a [RocksJava] Incorporated changes for D30525 2015-01-10 20:54:50 +01:00
fyrz 0aab1005f8 [RocksJava] ColumnFamilyDescriptor alignment with listColumnFamilies
Summary:
Previous to this commit ColumnFamilyDescriptor took a String as name for the ColumnFamily name. String is however encoding dependent which is bad because listColumnFamilies returns byte arrays without any encoding information.

All public API call were deprecated and flagged to be removed in 3.10.0

Test Plan:
make rocksdbjava
make test
mvn -f rocksjni.pom package

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D30525
2015-01-10 20:54:49 +01:00
Igor Canadi ded26605f4 Merge pull request #444 from adamretter/java-api-fix
Fix the Java API build on Mac OS X
2014-12-23 15:26:45 +01:00
Adam Retter 98490bccf6 Fix the build on Mac OS X 2014-12-23 14:22:56 +00:00
Lei Jin 5045c43944 add support for nested BlockBasedTableOptions in config string
Summary:
Add support to allow nested config for block-based table factory. The format looks like this:

"write_buffer_size=1024;block_based_table_factory={block_size=4k};max_write_buffer_num=2"

Test Plan: unit test

Reviewers: yhchiang, rven, igor, ljin, jonahcohen

Reviewed By: jonahcohen

Subscribers: jonahcohen, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29223
2014-12-22 16:34:21 -08:00
fyrz 1fed1282ad [RocksJava] Incorporated changes D30081 2014-12-18 22:27:50 +01:00
fyrz 5b9ceef01d [RocksJava] JavaDoc correction 2014-12-18 22:19:57 +01:00
fyrz 5fbba60b6a [RocksJava] Incorporated changes D30081 2014-12-18 22:15:00 +01:00
fyrz b0230d7e09 [RocksJava] Incorporate additions for D30081 2014-12-18 22:05:07 +01:00
fyrz b015ed0ca6 [RocksJava] Slice / DirectSlice improvements
Summary:
- AssertionError when initialized with Non-Direct Buffer
- Tests + coverage for DirectSlice
- Slice sigsegv fixes when initializing from String and byte arrays
- Slice Tests

Test Plan: Run tests without source modifications.

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D30081
2014-12-18 22:05:07 +01:00
Yueh-Hsuan Chiang 4d422db010 Merge pull request #430 from adamretter/increase-parallelism
Added setIncreaseParallelism() to Java API Options
2014-12-18 11:02:13 -08:00
Adam Retter 17e84f2151 Rudimentary test cases for setIncreaseParallelism 2014-12-16 14:32:42 +00:00
Adam Retter eda0dcdd97 Exposed IncreasedParallelism option to Java API as
setIncreasedParallelism
2014-12-16 14:15:41 +00:00
fyrz efc94ceb27 [RocksJava] Incorporated changes for D29283 2014-12-16 07:21:01 +01:00
fyrz 69188ff449 [RocksJava] CompactRange support
Summary: Manual range compaction support in RocksJava.

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D29283
2014-12-16 07:21:01 +01:00
fyrz 48adce77cc [RocksJava] CompactRange support
- manual range compaction support in RocksJava
2014-12-16 07:21:00 +01:00
Yueh-Hsuan Chiang 2871bc7bc8 Merge pull request #422 from fyrz/RocksJava-Quality-Improvements
Rocks java quality improvements
2014-12-05 21:38:05 -08:00
Igor Canadi 815f638cd0 Fix java build 2014-12-03 19:06:57 -08:00
fyrz 3e684aa685 Integrated changes from D29571 2014-12-02 19:56:45 +01:00
fyrz a15169f2e9 Fixed a Lint problem 2014-12-02 09:58:20 +01:00
fyrz b7f9e644cc [RocksJava] Quality improvements
Summary:
- Addressed some FindBugs issues.
- Remove obsolete dbFolder cleanup
- Comparator tests for CF
 - Added AbstractComparatorTest.
 - Fixed a bug in the JNI Part about Java comparators
- Minor test improvements

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D29571
2014-12-02 09:58:19 +01:00
fyrz e002a6122f [RocksJava] Comparator tests for CF
- Added AbstractComparatorTest.
- Fixed a bug in the JNI Part about Java comparators
2014-12-02 09:58:19 +01:00
fyrz 335e6ad5cd [RocksJava] Remove obsolete dbFolder cleanup 2014-12-02 09:58:18 +01:00
fyrz b036804ac1 RocksJava - FindBugs issues
Addressed some FindBugs issues.
2014-12-02 09:58:17 +01:00
fyrz 67cb7ca758 [RocksJava] Fixed MacOS build of RocksJava
There were still some precision loss problems
remainging in RocksJava. This pull request resolve
these.
2014-11-26 20:53:23 +01:00
Yueh-Hsuan Chiang b8136a7d27 Merge pull request #398 from fyrz/RocksJava-CreateCheckPoint
[RocksJava] Support for stored snapshots
2014-11-26 11:40:41 -08:00
Yueh-Hsuan Chiang 533592a27d Merge pull request #401 from fyrz/RocksJava-Sigsegv-MergeOperatorName
[RocksJava] Fixes in MergeOperatorByName
2014-11-26 11:40:13 -08:00
Adam Retter ff0cb90d1c Do not delete Java Fatal Error Log, developers may still want these for
reference
2014-11-26 18:18:04 +00:00
Adam Retter 2a792cd300 There will also be a librocksdbjni-osx.jnilib.dSYM folder on MacOSX
builds to be deleted
2014-11-26 18:17:54 +00:00
fyrz f193deea31 [RocksJava] Addressed comments in D28971 2014-11-25 23:33:42 +01:00
fyrz 94f70a86b9 [RocksJava] Incoroporated changes for D29013 2014-11-25 22:26:03 +01:00
fyrz a280af2a57 [RocksJava] Sigsegv fix for MergerOperatorByName 2014-11-25 22:26:02 +01:00
fyrz fcc2dfd9f9 [RocksJava] Support for stored snapshots
Summary:
RocksDB supports two ways of saving snapshots. In
memory and on disk. The later was added with this
pull request to RocksJava.

Test Plan:

Reviewers:

Subscribers:
2014-11-25 22:25:22 +01:00
fyrz c4765dc10b [RocksJava] Fix precision problem in rocksjni 2014-11-25 20:29:52 +01:00
fyrz c4b65f70fb [RocksJava] Makefile correction
Prevent non exist error message while executing
clean twice.
2014-11-21 23:49:31 +01:00
fyrz faa8d32be0 [RocksJava] Integrated changes from D29019. 2014-11-20 23:34:07 +01:00
fyrz 3d78c7a8cf [RocksJava] Lint adjustments 2014-11-20 22:51:31 +01:00
fyrz d7529b2de9 [RocksJava] Cleanup Backupable implementations
- Correct usage of isInitialized()
- Adjusted JavaDoc
2014-11-20 22:51:30 +01:00
fyrz fa703efb28 [RocksJava] Improved BackupableDBTest
- Splitted methods to meaningful tests
- Added tests for additional functionality
- Covered missing parts
2014-11-20 22:51:30 +01:00
fyrz 24fdc47416 [RocksJava] Backupable/Restorable DB update 3.8.0
- GarbageCollectMethod() available.
- GetCorruptedBackups() available.
2014-11-20 22:51:29 +01:00
fyrz 9972f969ee [RocksJava] BackupableDBOptions alginment + 3.8
- Updated the BackupableDBOptions functionality to 3.8.0.
- Aligned Options implementation with remaining source code.
- Invented test-case.
2014-11-20 22:51:29 +01:00
fyrz e7fcaa4d92 [RocksJava] JavaDoc is executed too often
Previous to this commit too much targets got dependencies
on javadocs target.

Introduced one additional target "javalib" which resolves
that situation. JavaDoc will now be generated once while
executing a task with prefix "rocksdbjava".
2014-11-20 07:28:16 +01:00
fyrz 2cd1794e4f [RocksJava] Make cleanup - Clean Target
- Remove JNI includes on clean
- Remove target folder generated by Maven
- Remove shared object
- Remove jar
2014-11-20 07:21:30 +01:00
fyrz 91ccc8ebef [RocksJava] Integrated changes in D29025
Addressed review comments.
2014-11-18 21:45:12 +01:00
fyrz 5249d0db50 [RocksJava] Convenience methods for Options
RocksDB introduced in 3.7.0 convenience methods
for getting ColumnFamilyOptions and DBOptions
instances from predefined configuration structures.

There is now also a method in RocksJava to load DBOptions
as well as ColumnFamilyOptions from a predefined Properties
based configuration.
2014-11-18 21:43:20 +01:00
Lei Jin 8d3f8f9696 remove all remaining references to cfd->options()
Summary:
The very last reference happens in DBImpl::GetOptions()
I built with both DBImpl::GetOptions() and ColumnFamilyData::options() commented out

Test Plan: make all check

Reviewers: sdong, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D29073
2014-11-18 10:20:10 -08:00
fyrz 8efd4bb424 [RocksJava] Improved comments in RocksDB class
Improved comments in RocksDB getLongProperty methods,
to describe the behavior more detailed.
2014-11-17 21:32:00 +01:00
fyrz 5529c1ad1b [RocksJava] GetIntProperty in RocksDB
Expose GetIntProperty methods to RocksJava. As the integer(64-Bit)
value is no integer in Java the method is aligned with the return
type which is long.
2014-11-17 21:32:00 +01:00
fyrz e97f014b91 [RocksJava] JavaDoc corrections - Java8
This commit solves build problems in Java8 due
to wrong JavaDoc.
2014-11-17 19:27:52 +01:00
Yueh-Hsuan Chiang ec24bd4e6a Merge pull request #402 from adamretter/bugfix-native-library-loader
Use correct classloader in Java NativeLibraryLoader
2014-11-16 16:42:59 -08:00
Yueh-Hsuan Chiang d3c4a0f4ae Improve the comment in InfoLogLevelTest.java
Summary:
Improve the comment in InfoLogLevelTest.java

Test Plan:
make rocksdbjava
2014-11-15 17:05:52 -08:00
Yueh-Hsuan Chiang a77e97c536 Merge pull request #396 from fyrz/RocksJava-LogLevel
[RocksJava] LogLevel support in Options
2014-11-15 17:03:14 -08:00
Adam Retter 585c759cf3 Make sure to use the correct Java classloader for loading the RocksDB
Native Library
2014-11-15 23:42:07 +00:00
Adam Retter c3915abbae Minor tidyup and use Java 7 for file copying 2014-11-15 23:42:07 +00:00
fyrz b8d5e3f08e [RocksJava] MVN Build reads version from version.h 2014-11-15 20:12:04 +01:00
fyrz 8a1bcc39c5 [RocksJava] Bump version to 3.8 in rocksjni.pom 2014-11-15 01:09:54 +01:00
fyrz 4947a0674f [RocksJava] Incorporated review comments D28947 2014-11-15 00:14:36 +01:00
fyrz 07cd3c42a2 [RocksJava] LogLevel support in Options
It's now possible to set a LogLevel in Options and
DBOptions to control LOG verbosity.
2014-11-14 23:40:20 +01:00