Commit graph

3228 commits

Author SHA1 Message Date
Adam Retter d6187d07bb Maven can now build a standard project layout 2015-02-01 20:05:10 +00:00
Adam Retter 157768890c Moved Java Benchmark main classes into benchmark/src/main/java 2015-02-01 20:05:10 +00:00
Adam Retter dd8d5471ea Adjustment to NativeLibraryLoader to allow native library to be loaded
from either java.library.path or from extracting from the Jar. Means
that the test in the build do not need to rely on the Jar, useful when
creating similar builds (and executing tests) from Maven
2015-02-01 20:05:10 +00:00
Adam Retter 353db6daef Moved Java main classes into src/main/java 2015-02-01 20:04:51 +00:00
Adam Retter 98cb501bc0 Moved Java test classes into src/test/java 2015-02-01 19:30:31 +00:00
fyrz 7479a62a7a Release.md - Remove version change instrcution
The version change instruction is obsolete with the change
that maven pulls versioning information from version.h.
2015-02-01 14:32:51 +01:00
Jörg Maier 4a4e4279f0 Update HISTORY-JAVA.md 2015-01-31 16:13:06 +01:00
Adam Retter 384cb6619e Merge pull request #480 from fyrz/RocksJava-Deprecate-SkipLogError
[RocksJava] Deprecate setSkipLogErrorOnRecovery
2015-01-31 15:06:12 +00:00
fyrz ca52a67cfb [RocksJava] Deprecate setSkipLogErrorOnRecovery
- see: 62ad0a9b19
2015-01-31 15:59:57 +01:00
Adam Retter 114d21878d Merge pull request #479 from fyrz/RocksJava-Snapshot-Sequence-Number
[RocksJava] Snapshot - GetSequenceNumber
2015-01-31 14:57:37 +00:00
fyrz cb5c3159f0 [RocksJava] Snapshot - GetSequenceNumber
Summary:
As the C++ part exposes now SequenceNumber retrieval
for Snapshots we want this obviously also in the Java API.

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom test

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32571
2015-01-31 15:50:34 +01:00
Adam Retter ea189b320c Merge pull request #474 from fyrz/RocksJava-GetUpdatesSince
[RocksJava] GetUpdatesSince
2015-01-31 14:41:06 +00:00
fyrz 391f85fc82 [RocksJava] Incorporated changes for D32151 2015-01-31 15:15:49 +01:00
fyrz 68cd93b873 [RocksJava] GetUpdatesSince support
Summary:
This differential describes further changes to the Java-API

New methods:

* GetUpdatesSince
* GetLatestSequenceNumber
* EnableFileDeletions
* DisableFileDeletions

This pull requests depends on: https://github.com/facebook/rocksdb/pull/472

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom package

Reviewers: yhchiang, adamretter, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32151
2015-01-31 15:15:48 +01:00
fyrz caedd40ddd [RocksJava] Adjusted auto pointer 2015-01-31 15:15:47 +01:00
fyrz b39006e3db [RocksJava] enable/disable File deletions 2015-01-31 15:15:47 +01:00
fyrz 9a456fba20 [RocksJava] GetUpdatesSince support 2015-01-31 15:15:46 +01:00
Adam Retter d3a736761b Merge pull request #482 from fyrz/RocksJava-TTL-Fix
[RocksJava] Fix ColumnFamily name alloc in TTL DB
2015-01-31 14:11:13 +00:00
fyrz 939bb36597 [RocksJava] Fix ColumnFamily name alloc in TTL DB
While fixing the RocksDB ColumnFamily name the TTL DB
wasn`t touched. This commit resolves this.
2015-01-31 14:43:21 +01:00
Marko Kevac 86e2a1eeea Allow creating backups from C 2015-01-31 15:47:49 +03:00
sdong db9ed5fdb4 Unaddressed comment in previous diff. Change only in code comments. 2015-01-30 16:07:35 -08:00
sdong 5917de0bae CappedFixTransform: return fixed length prefix, or full key if key is shorter than the fixed length
Summary: Add CappedFixTransform, which is the same as fixed length prefix extractor, except that when slice is shorter than the fixed length, it will use the full key.

Test Plan:
Add a test case for
db_test
options_test
and a new test

Reviewers: yhchiang, rven, igor

Reviewed By: igor

Subscribers: MarkCallaghan, leveldb, dhruba, yoshinorim

Differential Revision: https://reviews.facebook.net/D31887
2015-01-30 16:04:30 -08:00
Igor Canadi 6c6037f60c Expose Snapshot's SequenceNumber
Summary:
Requested here: https://www.facebook.com/groups/rocksdb.dev/permalink/705524519546065/

It might also help with mongo. I don't see a reason why we shouldn't expose this info.

Test Plan: make check

Reviewers: sdong, yhchiang, rven

Reviewed By: rven

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32547
2015-01-29 18:22:43 -08:00
Igor Canadi 2fd8f750ab Compile MemEnv with standard RocksDB library
Summary: This was a feature request by osquery. See task t5617758

Test Plan: compiles and memenv_test runs

Reviewers: yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32115
2015-01-29 16:33:11 -08:00
Igor Canadi 173c52a97f Fix build on older compilers -- emplace() is not available 2015-01-29 13:43:09 -08:00
sdong d07fec3bdc make DBTest.SharedWriteBuffer to pass MockEnv
Summary: DBTest.SharedWriteBuffer uses an Options that doesn't pass CurrentOptions(), so that it doesn't use MockEnv. However, DBTest's constructor uses MockEnv to call DestoryDB() to clean up, causing uncleaned state before it runs.

Test Plan: Run the test modified to make sure they pass default Env and SharedWriteBuffer now passes MockEnv.

Reviewers: rven, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D32475
2015-01-28 16:19:27 -08:00
Igor Canadi 4bdf38b16e Disable FlushSchedule when running TSAN
Summary:
There's a bug in TSAN (or libstdc++?) with std::shared_ptr<> for some reason. In db_test, only FlushSchedule is affected.

See more: https://groups.google.com/forum/#!topic/thread-sanitizer/vz_s-t226Vg

With this change and all other @sdong's and mine diffs, our db_test should be TSAN-clean. I'll move to other tests.

Test Plan: no more flush schedule when running TSAN

Reviewers: yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: sdong, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32469
2015-01-28 15:31:48 -08:00
sdong e84299c769 Fix bug recently introduced in MemFile::Lock()
Summary: This bug fails DBTest.CheckLock

Test Plan: DBTest.CheckLock now passes with MEM_ENV=1.

Reviewers: rven, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D32451
2015-01-28 15:21:39 -08:00
sdong e5aab4c2b2 Fix data race in HashLinkList
Summary:
1) need to do acquire load when read the first entry in the bucket.
2) Make num_entries atomic

Test Plan: Ran DBTest.MultiThreaded with TSAN

Reviewers: yhchiang, rven, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32361
2015-01-28 15:16:52 -08:00
Yueh-Hsuan Chiang 2113ecd3c2 Merge pull request #472 from fyrz/RocksJava-Cleanup
[RocksJava] Cleanup portal.h & tests
2015-01-28 14:37:25 -08:00
sdong 10af17f3d7 fault_injection_test: add a unit test to allow parallel compactions and multiple levels
Summary: Add a new test case in fault_injection_test, which covers parallel compactions and multiple levels. Use MockEnv to run the new test case to speed it up. Improve MockEnv to avoid DestoryDB(), previously failed when deleting lock files.

Test Plan: Run ./fault_injection_test, including valgrind

Reviewers: rven, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D32415
2015-01-28 14:07:25 -08:00
Igor Canadi 0c4d1053df Fix data race #5
Summary: TSAN complained that these are non-atomic reads and writes from different threads.

Test Plan: TSAN no longer complains

Reviewers: yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32409
2015-01-28 13:42:40 -08:00
fyrz cc0d8be011 [RocksJava] Integrated review comments (D32145) 2015-01-28 21:54:01 +01:00
Igor Canadi 5257c9c42c Merge pull request #452 from robertabcd/backupable-mem
Reduce memory footprint in backupable db.
2015-01-28 10:53:46 -08:00
Igor Canadi 560ed402bd [minor] fprintf to stderr instead of stdout in test 2015-01-27 21:00:33 -08:00
Igor Canadi 551a41df37 Merge pull request #476 from alabid/alabid/add-to-simple-example
Added WriteBatch block to simple_example.cc
2015-01-27 18:12:44 -08:00
sdong d2a2b058f0 fault_injection_test: to support file closed after being deleted
Summary: fault_injection_test occasionally fails because file closing can happen after deletion. Improve the test to support it.

Test Plan: I have a new test case I'm working on, where the issue appears almost every time. With the patch, the problem goes away.

Reviewers: rven, yhchiang, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32373
2015-01-27 17:06:47 -08:00
alabid f8f040ccc7 Updated .gitignore to ignore *~ files and example object files 2015-01-27 20:03:57 -05:00
Igor Canadi e8bf2310a0 Remove blob store from the codebase
Summary: We don't have plans to work on this in the short term. If we ever resurrect the project, we can find the code in the history. No need for it to linger around

Test Plan: no test

Reviewers: yhchiang, rven, sdong

Reviewed By: sdong

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D32349
2015-01-27 16:55:33 -08:00
alabid ea7d0b943a Added WriteBatch block to simple_example.cc 2015-01-27 19:37:36 -05:00
Yueh-Hsuan Chiang d6c7300ccf Fixed a compile warning in clang in db/listener_test.cc
Summary: Fixed a compile warning in clang in db/listener_test.cc

Test Plan: make listener_test

Reviewers: oridb

Reviewed By: oridb

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32337
2015-01-27 15:01:04 -08:00
Ori Bernstein f9758e0129 Add compaction listener.
Summary: This adds a listener for compactions, and gives some useful statistics on each compaction pass.

Test Plan: Unit tests.

Reviewers: sdong, igor, rven, yhchiang

Reviewed By: yhchiang

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D31641
2015-01-27 14:44:02 -08:00
sdong e919ecedfc SuperVersion::Unref() to use sequential consistency to decrease ref counting
Summary: I'm not sure the expected results of std::atomic::fetch_sub() when using memory_order_relaxed, and I suspect TSAN complains.

Test Plan: make all check

Reviewers: rven, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D32259
2015-01-27 14:08:08 -08:00
sdong 4c49fedaf1 Use ustricter consistency in thread local operations
Summary:
ThreadSanitizer complains data race of super version and version's destructor with Get(). This patch will fix those warning.

The warning is likely from ColumnFamilyData::ReturnThreadLocalSuperVersion(). With relaxed consistency of CAS, reading the data of the super version can technically happen after swapping it in, enabling the background thread to clean it up.

Test Plan: make all check

Reviewers: rven, igor, yhchiang

Reviewed By: yhchiang

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D32265
2015-01-27 13:56:03 -08:00
sdong 1b43ab58d9 fault_injection_test: add more logging and makes synchronization slightly stronger
Summary:
We see failure of the test in travis but I can't repro it.
Add more logging in failure cases to help us figure out which failure it is.
Also makes synchronization slightly stronger, though there isn't seem to be a problem without it

Test Plan: Run the test

Reviewers: rven, yhchiang, igor

Reviewed By: igor

Subscribers: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D32319
2015-01-27 13:53:17 -08:00
fyrz ca2b00277e [RocksJava] Cleanup portal.h & tests
Summary:
Simple Java Native Objects usually are represented using
the same functionality but within different classes.

With this commit a template class was introduced to remove
the redundant impelementation to a certain extent.

[RocksJava] Removed todo comment in portal.h

As jclass instances shall not be cached, both
todos are obsolete and can be removed.

[RocksJava] Add missing test to Makefile

[RocksJava] Added tests for uncovered methods

Test Plan:
make rocksdbjava
make jtest
mvn -f rocksjni.pom package

Reviewers: adamretter, yhchiang, ankgup87

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32145
2015-01-27 21:14:48 +01:00
fyrz f8dc5c459f [RocksJava] Add missing test to Makefile 2015-01-27 21:14:48 +01:00
fyrz b3c1331488 [RocksJava] Removed todo comment in portal.h
As jclass instances shall not be cached, both
todos are obsolete and can be removed.
2015-01-27 21:14:47 +01:00
fyrz 7ffcc457ff [RocksJava] Cleanup portal.h
Simple Java Native Objects usually are represented using
the same functionality but within different classes.

With this commit a template class was introduced to remove
the redundant impelementation to a certain extent.
2015-01-27 21:14:47 +01:00
Yueh-Hsuan Chiang c4fb83441c Update the comment for the removal of mac-install-gflags.sh
Summary: Update the comment for the removal of mac-install-gflags.sh

Test Plan: n/a

Reviewers: igor

Reviewed By: igor

Subscribers: dhruba

Differential Revision: https://reviews.facebook.net/D32295
2015-01-27 10:30:35 -08:00