Find a file
Levi Tamasi b1c27a52d2 Add a consistency check that prevents the overflow of garbage in blob files (#9100)
Summary:
The number or total size of garbage blobs in any given blob file can
never exceed the number or total size of all blobs in the file. (This
would be a similar error to e.g. attempting to remove from the LSM tree
an SST file that has already been removed.) The patch builds on
https://github.com/facebook/rocksdb/issues/9085 and adds a
consistency check to `VersionBuilder` that prevents the above from
happening.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/9100

Test Plan: `make check`

Reviewed By: riversand963

Differential Revision: D32048982

Pulled By: ltamasi

fbshipit-source-id: 6f7e0793bf534ad04c3359cc0f696b8e4e5ef81c
2021-11-01 12:32:14 -07:00
.circleci Fix EnvLibrados and add to CI (#9088) 2021-10-29 08:19:03 -07:00
.github/workflows Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
buckifier Update buckify scripts (#9104) 2021-11-01 10:11:18 -07:00
build_tools Make format-diff.sh locale-independent (#9079) 2021-10-27 12:26:36 -07:00
cache Add lowest_used_cache_tier to ImmutableDBOptions to enable or disable Secondary Cache (#9050) 2021-10-19 15:54:23 -07:00
cmake
coverage Remove asan_symbolize.py for internal asan build (#8737) 2021-09-07 15:39:11 -07:00
db Add a consistency check that prevents the overflow of garbage in blob files (#9100) 2021-11-01 12:32:14 -07:00
db_stress_tool Avoid div-by-zero error in db_stress (#9086) 2021-10-31 22:16:03 -07:00
docs Misc doc fixes (#8983) 2021-10-07 11:22:17 -07:00
env Experimental support for SST unique IDs (#8990) 2021-10-18 23:32:01 -07:00
examples Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
file Add (Live)FileStorageInfo API (#8968) 2021-10-16 10:04:32 -07:00
fuzz
hdfs
include/rocksdb Fix EnvLibrados and add to CI (#9088) 2021-10-29 08:19:03 -07:00
java Java wrapper for blob_gc_force_threshold as blobGarbageCollectionForceThreshold (#9109) 2021-11-01 11:59:10 -07:00
logging Make SystemClock into a Customizable Class (#8636) 2021-09-21 09:23:48 -07:00
memory Fix and detect headers with missing dependencies (#8893) 2021-09-10 10:00:26 -07:00
memtable Fix race in WriteBufferManager (#9009) 2021-10-12 00:16:21 -07:00
microbench
monitoring remove bad extra RecordTick(stats_, WRITE_WITH_WAL) (#9064) 2021-11-01 11:43:14 -07:00
options Implement XXH3 block checksum type (#9069) 2021-10-28 22:15:17 -07:00
plugin
port Fix incorrect order of comments in win_thread.cc (#9033) 2021-10-27 13:25:01 -07:00
table Implement XXH3 block checksum type (#9069) 2021-10-28 22:15:17 -07:00
test_util SyncPoint::Process thrashes heap ... fix it (#9023) 2021-10-15 13:30:29 -07:00
third-party Add support for building on s390x platform (#8962) 2021-10-22 10:13:15 -07:00
tools Implement XXH3 block checksum type (#9069) 2021-10-28 22:15:17 -07:00
trace_replay Cleanup includes in dbformat.h (#8930) 2021-09-29 04:04:40 -07:00
util Add support for building on s390x platform (#8962) 2021-10-22 10:13:15 -07:00
utilities Fix a compaction bug for write-prepared txn (#9061) 2021-10-29 15:23:17 -07:00
.clang-format
.gitignore
.lgtm.yml
.travis.yml Temporarily disable s390x+cmake* Travis jobs (#9095) 2021-10-29 11:06:50 -07:00
.watchmanconfig
appveyor.yml
AUTHORS
CMakeLists.txt Add support for building on s390x platform (#8962) 2021-10-22 10:13:15 -07:00
CODE_OF_CONDUCT.md
CONTRIBUTING.md
COPYING
DEFAULT_OPTIONS_HISTORY.md
defs.bzl
DUMP_FORMAT.md
HISTORY.md remove bad extra RecordTick(stats_, WRITE_WITH_WAL) (#9064) 2021-11-01 11:43:14 -07:00
INSTALL.md
issue_template.md
LANGUAGE-BINDINGS.md Update branch name to "main" in README/LANGUAGE_BINDINGS (#8727) 2021-09-01 15:26:34 -07:00
LICENSE.Apache
LICENSE.leveldb
Makefile Fix EnvLibrados and add to CI (#9088) 2021-10-29 08:19:03 -07:00
PLUGINS.md
README.md Update branch name to "main" in README/LANGUAGE_BINDINGS (#8727) 2021-09-01 15:26:34 -07:00
ROCKSDB_LITE.md
src.mk Experimental support for SST unique IDs (#8990) 2021-10-18 23:32:01 -07:00
TARGETS internal_repo_rocksdb/repo 2021-10-29 19:34:39 -07:00
thirdparty.inc
USERS.md Update USERS.md (#8923) 2021-10-01 16:10:35 -07:00
Vagrantfile
WINDOWS_PORT.md Update branch name in WINDOWS_PORT.md (#8745) 2021-09-01 19:26:39 -07:00

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

CircleCI Status TravisCI Status Appveyor Build status PPC64le Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/main/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/ and https://rocksdb.slack.com/

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.