Go to file
anand76 fb05b5a652 Force a new manifest file if append to current one fails (#6331)
Summary:
Fix for issue https://github.com/facebook/rocksdb/issues/6316

When an append/sync of the manifest file fails due to an IO error such
as NoSpace, we don't always put the DB in read-only mode. This is true
for flush and compactions, as well as foreground operatons such as column family
add/drop, CompactFiles etc. Subsequent changes to the DB will be
recorded in the same manifest file, which would have a corrupted record
in the middle due to the previous failure. On next DB::Open(), it will
fail to process the full manifest and data will be lost.

To fix this, we reset VersionSet::descriptor_log_ on append/sync
failure, which will force a new manifest file to be written on the next
append.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6331

Test Plan: Add new unit tests in error_handler_test.cc

Differential Revision: D19632951

Pulled By: anand1976

fbshipit-source-id: 68d527cb6e59a94cbbbf9f5a17a7f464381d51e3
2020-01-30 10:56:29 -08:00
buckifier PosixRandomAccessFile::MultiRead() to use I/O uring if supported (#5881) 2019-12-07 20:55:52 -08:00
build_tools Upload DB dir for all crash tests (#6344) 2020-01-29 15:59:07 -08:00
cache Remove key length assertion LRUHandle::CalcTotalCharge (#6115) 2019-12-02 15:00:07 -08:00
cmake cmake: do not build tests for Release build and cleanups (#5916) 2019-12-13 12:48:06 -08:00
coverage Fix interpreter lines for files with python2-only syntax. 2019-07-09 10:51:37 -07:00
db Force a new manifest file if append to current one fails (#6331) 2020-01-30 10:56:29 -08:00
db_stress_tool Add ReadOptions.auto_prefix_mode (#6314) 2020-01-28 14:44:05 -08:00
docs Log warning for high bits/key in legacy Bloom filter (#6312) 2020-01-17 19:37:35 -08:00
env Fix some shadow warning (#6242) 2020-01-08 18:20:13 -08:00
examples Update example of optimistic transaction (#6074) 2020-01-16 14:04:44 -08:00
file Force a new manifest file if append to current one fails (#6331) 2020-01-30 10:56:29 -08:00
hdfs Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
include/rocksdb Add statistics for BlobDB GC (#6296) 2020-01-29 16:46:16 -08:00
java Reduce the need to re-download dependencies (#6318) 2020-01-29 08:01:56 -08:00
logging Increase max_log_size in FlushJob to 1024 bytes (#6258) 2020-01-06 10:16:52 -08:00
memory Charge block cache for cache internal usage (#5797) 2019-09-16 15:26:21 -07:00
memtable Misc hashing updates / upgrades (#5909) 2019-10-24 17:16:46 -07:00
monitoring Apply formatter to recent 200+ commits. (#5830) 2019-09-20 12:04:26 -07:00
options Add ReadOptions.auto_prefix_mode (#6314) 2020-01-28 14:44:05 -08:00
port Implement getfreespace for WinEnv (#6265) 2020-01-07 13:56:13 -08:00
table Add ReadOptions.auto_prefix_mode (#6314) 2020-01-28 14:44:05 -08:00
test_util unordered_write incompatible with max_successive_merges (#6284) 2020-01-10 16:53:19 -08:00
third-party Apply formatter to some recent commits (#6138) 2019-12-09 15:49:49 -08:00
tools Revert "crash_test to enable block-based table hash index (#6310)" (#6327) 2020-01-23 09:09:17 -08:00
trace_replay Misc hashing updates / upgrades (#5909) 2019-10-24 17:16:46 -07:00
util Warn on excessive keys for legacy Bloom filter with 32-bit hash (#6317) 2020-01-20 21:31:47 -08:00
utilities Add statistics for BlobDB GC (#6296) 2020-01-29 16:46:16 -08:00
.clang-format A script that automatically reformat affected lines 2014-01-14 12:21:24 -08:00
.gitignore Make buckifier python3 compatible (#5922) 2019-10-23 13:52:27 -07:00
.lgtm.yml Create lgtm.yml for LGTM.com C/C++ analysis (#4058) 2018-06-26 12:43:04 -07:00
.travis.yml Small tidy and speed up of the travis build (#6181) 2019-12-17 13:56:45 -08:00
.watchmanconfig Added .watchmanconfig file to rocksdb repo (#5593) 2019-07-19 15:00:33 -07:00
AUTHORS Update RocksDB Authors File 2017-10-18 14:42:10 -07:00
CMakeLists.txt Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
CODE_OF_CONDUCT.md Adopt Contributor Covenant 2019-08-29 23:21:01 -07:00
CONTRIBUTING.md Add Code of Conduct 2017-12-05 18:42:35 -08:00
COPYING Add GPLv2 as an alternative license. 2017-04-27 18:06:12 -07:00
DEFAULT_OPTIONS_HISTORY.md options.delayed_write_rate use the rate of rate_limiter by default. 2017-05-24 09:58:24 -07:00
DUMP_FORMAT.md First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
HISTORY.md Force a new manifest file if append to current one fails (#6331) 2020-01-30 10:56:29 -08:00
INSTALL.md Update the version of the dependencies used by the RocksJava static build (#4761) 2018-12-18 20:25:43 -08:00
LANGUAGE-BINDINGS.md LANGUAGE-BINDINGS.md: mention python-rocksdb 2019-03-20 11:10:48 -07:00
LICENSE.Apache Change RocksDB License 2017-07-15 16:11:23 -07:00
LICENSE.leveldb Add back the LevelDB license file 2017-07-16 18:42:18 -07:00
Makefile Reduce the need to re-download dependencies (#6318) 2020-01-29 08:01:56 -08:00
README.md Replaced some words (#5877) 2019-10-07 12:28:09 -07:00
ROCKSDB_LITE.md Fix some typos in comments and docs. 2018-03-08 10:27:25 -08:00
TARGETS Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
USERS.md add user nebula (#6271) 2020-01-08 13:46:43 -08:00
Vagrantfile Adding CentOS 7 Vagrantfile & build script 2018-02-26 15:27:17 -08:00
WINDOWS_PORT.md #5145 , rename port/dirent.h to port/port_dirent.h to avoid compile err when use port dir as header dir output (#5152) 2019-04-04 11:38:19 -07:00
appveyor.yml Add Visual Studio 2015 to AppVeyor (#5446) 2019-12-10 20:02:31 -08:00
defs.bzl Add clarifying/instructive header to TARGETS and defs.bzl 2019-11-05 20:20:33 -08:00
issue_template.md Add Google Group to Issue Template 2020-01-28 14:40:37 -08:00
src.mk Introduce a new storage specific Env API (#5761) 2019-12-13 14:48:41 -08:00
thirdparty.inc Fix build jemalloc api (#5470) 2019-06-24 17:40:32 -07:00

README.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

Linux/Mac Build Status Windows Build status PPC64le Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.