rocksdb/db/compaction
Gang Liao deff48bcef Add blob source to retrieve blobs in RocksDB (#10198)
Summary:
There is currently no caching mechanism for blobs, which is not ideal especially when the database resides on remote storage (where we cannot rely on the OS page cache). As part of this task, we would like to make it possible for the application to configure a blob cache.
In this task, we formally introduced the blob source to RocksDB.  BlobSource is a new abstraction layer that provides universal access to blobs, regardless of whether they are in the blob cache, secondary cache, or (remote) storage. Depending on user settings, it always fetch blobs from multi-tier cache and storage with minimal cost.

Note: The new `MultiGetBlob()` implementation is not included in the current PR. To go faster, we aim to create a separate PR for it in parallel!

This PR is a part of https://github.com/facebook/rocksdb/issues/10156

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10198

Reviewed By: ltamasi

Differential Revision: D37294735

Pulled By: gangliao

fbshipit-source-id: 9cb50422d9dd1bc03798501c2778b6c7520c7a1e
2022-06-20 20:58:11 -07:00
..
clipping_iterator.h Add a clipping internal iterator (#8327) 2021-06-09 15:41:16 -07:00
clipping_iterator_test.cc Cleanup multiple implementations of VectorIterator (#8901) 2021-10-06 07:48:31 -07:00
compaction.cc Support specifying blob garbage collection parameters when CompactRange() (#10073) 2022-06-01 19:40:26 -07:00
compaction.h Support specifying blob garbage collection parameters when CompactRange() (#10073) 2022-06-01 19:40:26 -07:00
compaction_iteration_stats.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
compaction_iterator.cc Abort in dbg mode after logging (#10183) 2022-06-15 22:00:24 -07:00
compaction_iterator.h Consolidate manual_compaction_paused_ check (#10070) 2022-06-06 18:32:26 -07:00
compaction_iterator_test.cc Consolidate manual_compaction_paused_ check (#10070) 2022-06-06 18:32:26 -07:00
compaction_job.cc Consolidate manual_compaction_paused_ check (#10070) 2022-06-06 18:32:26 -07:00
compaction_job.h Consolidate manual_compaction_paused_ check (#10070) 2022-06-06 18:32:26 -07:00
compaction_job_stats_test.cc Remove own ToString() (#9955) 2022-05-06 13:03:58 -07:00
compaction_job_test.cc Add blob source to retrieve blobs in RocksDB (#10198) 2022-06-20 20:58:11 -07:00
compaction_picker.cc Support specifying blob garbage collection parameters when CompactRange() (#10073) 2022-06-01 19:40:26 -07:00
compaction_picker.h Add OpenAndTrimHistory API to support trimming data with specified timestamp (#9410) 2022-03-11 16:13:23 -08:00
compaction_picker_fifo.cc Add OpenAndTrimHistory API to support trimming data with specified timestamp (#9410) 2022-03-11 16:13:23 -08:00
compaction_picker_fifo.h Add OpenAndTrimHistory API to support trimming data with specified timestamp (#9410) 2022-03-11 16:13:23 -08:00
compaction_picker_level.cc Use std::numeric_limits<> (#9954) 2022-05-05 13:08:21 -07:00
compaction_picker_level.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
compaction_picker_test.cc Track SST unique id in MANIFEST and verify (#9990) 2022-05-19 11:04:21 -07:00
compaction_picker_universal.cc Use std::numeric_limits<> (#9954) 2022-05-05 13:08:21 -07:00
compaction_picker_universal.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
compaction_service_test.cc Fix failed VerifySstUniqueIds unittests (#10043) 2022-05-24 09:00:06 -07:00
file_pri.h Try to start TTL earlier with kMinOverlappingRatio is used (#8749) 2021-11-01 14:36:31 -07:00
sst_partitioner.cc Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362) 2022-01-11 06:33:48 -08:00