Go to file
Levi Tamasi 431e8afba7 Do not explicitly flush blob files when using the integrated BlobDB (#7892)
Summary:
In the original stacked BlobDB implementation, which writes blobs to blob files
immediately and treats blob files as logs, it makes sense to flush the file after
writing each blob to protect against process crashes; however, in the integrated
implementation, which builds blob files in the background jobs, this unnecessarily
reduces performance. This patch fixes this by simply adding a `do_flush` flag to
`BlobLogWriter`, which is set to `true` by the stacked implementation and to `false`
by the new code. Note: the change itself is trivial but the tests needed some work;
since in the new implementation, blobs are now buffered, adding a blob to
`BlobFileBuilder` is no longer guaranteed to result in an actual I/O. Therefore, we can
no longer rely on `FaultInjectionTestEnv` when testing failure cases; instead, we
manipulate the return values of I/O methods directly using `SyncPoint`s.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7892

Test Plan: `make check`

Reviewed By: jay-zhuang

Differential Revision: D26022814

Pulled By: ltamasi

fbshipit-source-id: b3dce419f312137fa70d84cdd9b908fd5d60d8cd
2021-01-25 13:32:33 -08:00
.circleci Small improvements to CircleCI Windows jobs (#7852) 2021-01-11 13:32:07 -08:00
.github/workflows Update clang-format-diff.py (#7609) 2020-11-04 16:09:01 -08:00
buckifier Fix use of positional args in BUCK rules (#7760) 2020-12-09 19:25:31 -08:00
build_tools Migrate away from Travis+Linux+amd64 (#7791) 2020-12-22 00:20:57 -08:00
cache Fix typos in comments (#7687) 2020-11-19 13:32:50 -08:00
cmake Add `find_dependency()` in cmake config file. (#6791) 2020-05-12 21:18:29 -07:00
coverage Find the correct gcov (#6904) 2020-06-01 16:33:05 -07:00
db Do not explicitly flush blob files when using the integrated BlobDB (#7892) 2021-01-25 13:32:33 -08:00
db_stress_tool Increase the txn lock timeout in stress test (#7823) 2020-12-30 20:31:35 -08:00
docs Update github-pages and dependencies (#7850) 2021-01-11 12:48:01 -08:00
env Improvements to Env::GetChildren (#7819) 2021-01-09 09:44:34 -08:00
examples Bring the Configurable options together (#5753) 2020-09-14 17:01:01 -07:00
file Improvements to Env::GetChildren (#7819) 2021-01-09 09:44:34 -08:00
fuzz Update SstFileWriter fuzzer to iterate and check all key-value pairs (#7761) 2020-12-11 16:09:10 -08:00
hdfs fix build with 'USE_HDFS' on windows (#6950) 2020-06-12 16:21:50 -07:00
include/rocksdb Fix various spelling errors still found in code (#7785) 2021-01-15 20:07:39 -08:00
java Fixing Windows build using CMake (#7854) 2021-01-15 17:53:16 -08:00
logging No elide constructors (#7798) 2020-12-23 16:55:53 -08:00
memory slightly improve jemalloc allocator API header (#7592) 2020-10-28 13:47:12 -07:00
memtable Add new API to report dummy entries size in cache in WriteBufferManager (#7837) 2021-01-08 13:26:24 -08:00
monitoring Use mock time for histogram_test (#7799) 2021-01-04 13:12:40 -08:00
options Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
port Improvements to Env::GetChildren (#7819) 2021-01-09 09:44:34 -08:00
table Add more tests to ASSERT_STATUS_CHECKED (3), API change (#7715) 2021-01-06 14:15:02 -08:00
test_util Create a CustomEnv class; Add WinFileSystem; Make LegacyFileSystemWrapper private (#7703) 2021-01-06 10:49:32 -08:00
third-party Fix Compilation on ppc64le using Clang 11 (#7713) 2020-12-01 11:21:44 -08:00
tools Make regression test load options from file for checkpoint (#7864) 2021-01-15 11:16:28 -08:00
trace_replay Add tests in ASSERT_STATUS_CHECKED (#7793) 2020-12-22 10:31:13 -08:00
util workaround race conditions during `PeriodicWorkScheduler` registration (#7888) 2021-01-21 08:50:38 -08:00
utilities Do not explicitly flush blob files when using the integrated BlobDB (#7892) 2021-01-25 13:32:33 -08:00
.clang-format
.gitignore Fuzzing RocksDB (#7685) 2020-11-17 12:56:48 -08:00
.lgtm.yml Create lgtm.yml for LGTM.com C/C++ analysis (#4058) 2018-06-26 12:43:04 -07:00
.travis.yml Cleanup Travis CI config (#7848) 2021-01-11 10:30:28 -08:00
.watchmanconfig Added .watchmanconfig file to rocksdb repo (#5593) 2019-07-19 15:00:33 -07:00
AUTHORS Update RocksDB Authors File 2017-10-18 14:42:10 -07:00
CMakeLists.txt Add artifact suffix to shared library (#7755) 2021-01-21 08:35:03 -08:00
CODE_OF_CONDUCT.md Adopt Contributor Covenant 2019-08-29 23:21:01 -07:00
CONTRIBUTING.md Add Code of Conduct 2017-12-05 18:42:35 -08:00
COPYING
DEFAULT_OPTIONS_HISTORY.md
DUMP_FORMAT.md
HISTORY.md Update HISTORY.md for PR 7888 (#7890) 2021-01-21 14:20:10 -08:00
INSTALL.md Update the version of the dependencies used by the RocksJava static build (#4761) 2018-12-18 20:25:43 -08:00
LANGUAGE-BINDINGS.md Add RestoreDBFromLatestBackup to C API, add new C# package (#7092) 2020-07-08 11:56:41 -07:00
LICENSE.Apache
LICENSE.leveldb
Makefile Update the versions of the test dependencies used for RocksJava (#7805) 2021-01-13 16:01:38 -08:00
README.md Fix the CI badge for ppc64le Jenkins (#7561) 2020-10-16 09:00:56 -07:00
ROCKSDB_LITE.md Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
TARGETS Range Locking: Implementation of range locking (#7506) 2020-12-22 19:12:36 -08:00
USERS.md Add Apache Doris to USERS (#7865) 2021-01-19 15:31:56 -08:00
Vagrantfile Adding CentOS 7 Vagrantfile & build script 2018-02-26 15:27:17 -08:00
WINDOWS_PORT.md #5145 , rename port/dirent.h to port/port_dirent.h to avoid compile err when use port dir as header dir output (#5152) 2019-04-04 11:38:19 -07:00
appveyor.yml Remove 2019 from appveyor (#7038) 2020-06-29 14:31:41 -07:00
defs.bzl Make testpilot recognize that these tests have coverage instrumentation 2020-03-20 11:23:23 -07:00
issue_template.md Add Google Group to Issue Template 2020-01-28 14:40:37 -08:00
src.mk Range Locking: Implementation of range locking (#7506) 2020-12-22 19:12:36 -08:00
thirdparty.inc Fix build jemalloc api (#5470) 2019-06-24 17:40:32 -07:00

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

CircleCI Status TravisCI Status Appveyor Build status PPC64le Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/ and https://rocksdb.slack.com/

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.