rocksdb/db/blob
akankshamahajan 5cb2d09d47 Refactor FilePrefetchBuffer code (#12097)
Summary:
Summary - Refactor FilePrefetchBuffer code
- Implementation:
FilePrefetchBuffer maintains a deque of free buffers (free_bufs_) of size num_buffers_ and buffers (bufs_) which contains the prefetched data. Whenever a buffer is consumed or is outdated (w.r.t. to requested offset), that buffer is cleared and returned to free_bufs_.

 If a buffer is available in free_bufs_, it's moved to bufs_ and is sent for prefetching. num_buffers_ defines how many buffers are maintained that contains prefetched data.
If num_buffers_ == 1, it's a sequential read flow. Read API will be called on that one buffer whenever the data is requested and is not in the buffer.
If num_buffers_ > 1, then the data is prefetched asynchronosuly in the buffers whenever the data is consumed from the buffers and that buffer is freed.
If num_buffers > 1, then requested data can be overlapping between 2 buffers. To return the continuous buffer overlap_bufs_ is used. The requested data is copied from 2 buffers to the overlap_bufs_ and overlap_bufs_ is returned to
the caller.

- Merged Sync and Async code flow into one in FilePrefetchBuffer.

Test Plan -
- Crash test passed
- Unit tests
- Pending - Benchmarks

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12097

Reviewed By: ajkr

Differential Revision: D51759552

Pulled By: akankshamahajan15

fbshipit-source-id: 69a352945affac2ed22be96048d55863e0168ad5
2024-01-05 09:29:01 -08:00
..
blob_constants.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_contents.cc Major Cache refactoring, CPU efficiency improvement (#10975) 2023-01-11 14:20:40 -08:00
blob_contents.h Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
blob_counting_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
blob_counting_iterator_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_fetcher.cc Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
blob_fetcher.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
blob_file_addition.cc Print blob file checksums as hex (#8437) 2021-06-22 09:49:44 -07:00
blob_file_addition.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_addition_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_file_builder.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
blob_file_builder.h Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
blob_file_builder_test.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
blob_file_cache.cc Improve memory efficiency of many OptimisticTransactionDBs (#11439) 2023-05-24 11:57:15 -07:00
blob_file_cache.h Improve memory efficiency of many OptimisticTransactionDBs (#11439) 2023-05-24 11:57:15 -07:00
blob_file_cache_test.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
blob_file_completion_callback.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
blob_file_garbage.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_file_meta.cc Add BlobMetaData retrieval methods (#8273) 2021-06-28 08:13:29 -07:00
blob_file_meta.h Add BlobMetaData retrieval methods (#8273) 2021-06-28 08:13:29 -07:00
blob_file_reader.cc Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
blob_file_reader.h Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444) 2023-08-08 17:26:50 -07:00
blob_file_reader_test.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
blob_garbage_meter.cc Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_garbage_meter.h Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_garbage_meter_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_index.h Remove local static string (#8103) 2022-08-05 23:03:51 -07:00
blob_log_format.cc Remove local static string (#8103) 2022-08-05 23:03:51 -07:00
blob_log_format.h Run clang-format on db/blob/ (#10856) 2022-10-24 16:00:32 -07:00
blob_log_sequential_reader.cc Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444) 2023-08-08 17:26:50 -07:00
blob_log_sequential_reader.h Fix a issue with initializing blob header buffer (#8537) 2021-08-02 17:15:06 -07:00
blob_log_writer.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
blob_log_writer.h Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
blob_read_request.h Enable blob caching for MultiGetBlob in RocksDB (#10272) 2022-06-30 13:24:35 -07:00
blob_source.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
blob_source.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
blob_source_test.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
db_blob_basic_test.cc Enforce status checking after Valid() returns false for IteratorWrapper (#11975) 2023-10-18 09:38:38 -07:00
db_blob_compaction_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_blob_corruption_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_blob_index_test.cc Track full_history_ts_low per SuperVersion (#11784) 2023-09-13 16:34:18 -07:00
prefetch_buffer_collection.cc Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
prefetch_buffer_collection.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00