dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-12-02 10:15:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db/blob
History
Gang Liao deff48bcef Add blob source to retrieve blobs in RocksDB (#10198) 
Summary:
There is currently no caching mechanism for blobs, which is not ideal especially when the database resides on remote storage (where we cannot rely on the OS page cache). As part of this task, we would like to make it possible for the application to configure a blob cache.
In this task, we formally introduced the blob source to RocksDB.  BlobSource is a new abstraction layer that provides universal access to blobs, regardless of whether they are in the blob cache, secondary cache, or (remote) storage. Depending on user settings, it always fetch blobs from multi-tier cache and storage with minimal cost.

Note: The new `MultiGetBlob()` implementation is not included in the current PR. To go faster, we aim to create a separate PR for it in parallel!

This PR is a part of https://github.com/facebook/rocksdb/issues/10156

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10198

Reviewed By: ltamasi

Differential Revision: D37294735

Pulled By: gangliao

fbshipit-source-id: 9cb50422d9dd1bc03798501c2778b6c7520c7a1e
2022-06-20 20:58:11 -07:00
..
blob_constants.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_counting_iterator.h Log the amount of blob garbage generated by compactions in the MANIFEST (#8450) 2021-06-24 16:11:56 -07:00
blob_counting_iterator_test.cc Cleanup multiple implementations of VectorIterator (#8901) 2021-10-06 07:48:31 -07:00
blob_fetcher.cc Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
blob_fetcher.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
blob_file_addition.cc Print blob file checksums as hex (#8437) 2021-06-22 09:49:44 -07:00
blob_file_addition.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_addition_test.cc Print blob file checksums as hex (#8437) 2021-06-22 09:49:44 -07:00
blob_file_builder.cc Expose blob file information through the EventListener interface (#8675) 2021-09-16 17:23:36 -07:00
blob_file_builder.h Expose blob file information through the EventListener interface (#8675) 2021-09-16 17:23:36 -07:00
blob_file_builder_test.cc Make SystemClock into a Customizable Class (#8636) 2021-09-21 09:23:48 -07:00
blob_file_cache.cc Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_cache.h Rename ImmutableOptions variables (#8409) 2021-06-16 16:51:38 -07:00
blob_file_cache_test.cc Make SystemClock into a Customizable Class (#8636) 2021-09-21 09:23:48 -07:00
blob_file_completion_callback.h Fix LITE mode builds on MacOs (#8981) 2021-10-04 05:30:26 -07:00
blob_file_garbage.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage_test.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_meta.cc Add BlobMetaData retrieval methods (#8273) 2021-06-28 08:13:29 -07:00
blob_file_meta.h Add BlobMetaData retrieval methods (#8273) 2021-06-28 08:13:29 -07:00
blob_file_reader.cc Add rate limiter priority to ReadOptions (#9424) 2022-02-16 23:18:14 -08:00
blob_file_reader.h Add rate limiter priority to ReadOptions (#9424) 2022-02-16 23:18:14 -08:00
blob_file_reader_test.cc Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
blob_garbage_meter.cc Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_garbage_meter.h Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_garbage_meter_test.cc Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_index.h Remove own ToString() (#9955) 2022-05-06 13:03:58 -07:00
blob_log_format.cc Introduce BlobFileCache and add support for blob files to Get() (#7540) 2020-10-15 13:04:47 -07:00
blob_log_format.h Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_log_sequential_reader.cc Add rate limiter priority to ReadOptions (#9424) 2022-02-16 23:18:14 -08:00
blob_log_sequential_reader.h Fix a issue with initializing blob header buffer (#8537) 2021-08-02 17:15:06 -07:00
blob_log_writer.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
blob_log_writer.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
blob_source.cc Add blob source to retrieve blobs in RocksDB (#10198) 2022-06-20 20:58:11 -07:00
blob_source.h Add blob source to retrieve blobs in RocksDB (#10198) 2022-06-20 20:58:11 -07:00
blob_source_test.cc Read blob from blob cache if exists when GetBlob() (#10178) 2022-06-17 15:22:59 -07:00
db_blob_basic_test.cc Add blob source to retrieve blobs in RocksDB (#10198) 2022-06-20 20:58:11 -07:00
db_blob_compaction_test.cc Fix LITE build (#10106) 2022-06-02 23:42:41 -07:00
db_blob_corruption_test.cc Enable a few unit tests to use custom Env objects (#9087) 2021-11-08 11:05:59 -08:00
db_blob_index_test.cc Remove own ToString() (#9955) 2022-05-06 13:03:58 -07:00
prefetch_buffer_collection.cc Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
prefetch_buffer_collection.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00