Find a file
Akanksha Mahajan 5ba1b6e549 Cache warming data blocks during flush (#8242)
Summary:
This PR prepopulates warm/hot data blocks which are already in memory
into block cache at the time of flush. On a flush, the data block that is
in memory (in memtables) get flushed to the device. If using Direct IO,
additional IO is incurred to read this data back into memory again, which
is avoided by enabling newly added option.

 Right now, this is enabled only for flush for data blocks. We plan to
expand this option to cover compactions in the future and for other types
 of blocks.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/8242

Test Plan: Add new unit test

Reviewed By: anand1976

Differential Revision: D28521703

Pulled By: akankshamahajan15

fbshipit-source-id: 7219d6958821cedce689a219c3963a6f1a9d5f05
2021-06-17 21:56:47 -07:00
.circleci Fix Windows build failure (#8400) 2021-06-14 09:35:32 -07:00
.github/workflows Update clang-format-diff.py path (#7944) 2021-02-09 12:49:38 -08:00
buckifier Modify script which generates TARGETS (#8366) 2021-06-04 16:28:59 -07:00
build_tools Remove TSAN crash test opt-in to platform007 (#8406) 2021-06-15 08:17:01 -07:00
cache Clarify some DB::Open,OpenForReadOnly semantics (#8379) 2021-06-14 16:09:21 -07:00
cmake
coverage Find the correct gcov (#6904) 2020-06-01 16:33:05 -07:00
db Cache warming data blocks during flush (#8242) 2021-06-17 21:56:47 -07:00
db_stress_tool Mark Ribbon filter and optimize_filters_for_memory as production (#8408) 2021-06-17 12:29:16 -07:00
docs Preset dictionary compression blog post (#8342) 2021-05-31 21:31:13 -07:00
env Add CreateFrom methods to Env/FileSystem (#8174) 2021-06-15 03:43:48 -07:00
examples make:Fix c header prototypes (#7994) 2021-03-09 20:44:23 -08:00
file Refactor kill point (#8241) 2021-05-05 15:50:29 -07:00
fuzz Remove Legacy and Custom FileWrapper classes from header files (#7851) 2021-01-28 22:10:32 -08:00
hdfs fix build with 'USE_HDFS' on windows (#6950) 2020-06-12 16:21:50 -07:00
include/rocksdb Cache warming data blocks during flush (#8242) 2021-06-17 21:56:47 -07:00
java Fix a typo in Javadoc (#8394) 2021-06-17 12:02:57 -07:00
logging Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
memory Use thread-safe strerror_r() to get error message (#8087) 2021-03-24 23:07:27 -07:00
memtable Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
monitoring Disable IOStatsContext/PerfContext if no thread local (#8117) 2021-04-13 07:56:59 -07:00
options Cache warming data blocks during flush (#8242) 2021-06-17 21:56:47 -07:00
plugin Makefile support to statically link external plugin code (#7918) 2021-02-10 08:35:34 -08:00
port Use deleters to label cache entries and collect stats (#8297) 2021-05-19 16:51:13 -07:00
table Cache warming data blocks during flush (#8242) 2021-06-17 21:56:47 -07:00
test_util Add CreateFrom methods to Env/FileSystem (#8174) 2021-06-15 03:43:48 -07:00
third-party Fix a compilation error in CircleCI vs2019 CXX20 (#8090) 2021-03-23 10:28:04 -07:00
tools Cache warming data blocks during flush (#8242) 2021-06-17 21:56:47 -07:00
trace_replay Add request_id in IODebugContext. (#8045) 2021-04-01 13:14:51 -07:00
util Implementation of Crc32c combine function (#8305) 2021-06-16 18:30:34 -07:00
utilities Add DeteleRange support for DBWithTTL (#8384) 2021-06-17 16:00:50 -07:00
.clang-format
.gitignore gitignore cmake-build-* for CLion integration (#7933) 2021-02-19 13:43:15 -08:00
.lgtm.yml
.travis.yml Move arm build from travis to circleci (#8203) 2021-04-19 20:07:02 -07:00
.watchmanconfig
appveyor.yml Remove 2019 from appveyor (#7038) 2020-06-29 14:31:41 -07:00
AUTHORS
CMakeLists.txt Increase cmake test DISCOVERY_TIMEOUT from 5 to 120 seconds (#8403) 2021-06-14 16:31:46 -07:00
CODE_OF_CONDUCT.md
CONTRIBUTING.md
COPYING
DEFAULT_OPTIONS_HISTORY.md
defs.bzl
DUMP_FORMAT.md
HISTORY.md Cache warming data blocks during flush (#8242) 2021-06-17 21:56:47 -07:00
INSTALL.md Update installation instructions (#8158) 2021-04-06 16:02:04 -07:00
issue_template.md
LANGUAGE-BINDINGS.md Add RestoreDBFromLatestBackup to C API, add new C# package (#7092) 2020-07-08 11:56:41 -07:00
LICENSE.Apache
LICENSE.leveldb
Makefile Add a clipping internal iterator (#8327) 2021-06-09 15:41:16 -07:00
PLUGINS.md Add ZenFS to plugin list (#8218) 2021-04-22 11:12:40 -07:00
README.md Fix the CI badge for ppc64le Jenkins (#7561) 2020-10-16 09:00:56 -07:00
ROCKSDB_LITE.md
src.mk Support for Merge in Integrated BlobDB with base values (#8292) 2021-06-10 12:58:37 -07:00
TARGETS Support for Merge in Integrated BlobDB with base values (#8292) 2021-06-10 12:58:37 -07:00
thirdparty.inc
USERS.md Add Apache Doris to USERS (#7865) 2021-01-19 15:31:56 -08:00
Vagrantfile
WINDOWS_PORT.md

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage

CircleCI Status TravisCI Status Appveyor Build status PPC64le Build Status

RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)

This code is a library that forms the core building block for a fast key-value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it especially suitable for storing multiple terabytes of data in a single database.

Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples

See the github wiki for more explanation.

The public interface is in include/. Callers should not include or rely on the details of any other header files in this package. Those internal APIs may be changed without warning.

Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/ and https://rocksdb.slack.com/

License

RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.