rocksdb/table
Andrew Ryan Chang 7c98a2d130 Update MultiGet to respect the strict_capacity_limit block cache option (#13104)
Summary:
There is a `strict_capacity_limit` option which imposes a hard memory limit on the block cache. When the block cache is enabled, every read request is serviced from the block cache. If the required block is missing, it is first inserted into the cache. If `strict_capacity_limit` is `true` and the limit has been reached, the `Get` and `MultiGet` requests should fail. However, currently this is not happening for `MultiGet`.

I updated `MultiGet` to explicitly check the returned status of `MaybeReadBlockAndLoadToCache`, so the status does not get overwritten later.

Thank you anand1976 for the problem explanation.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/13104

Test Plan:
Added unit test for both `Get` and `MultiGet` with a `strict_capacity_limit` set.

Before the change, half of my unit test cases failed https://github.com/facebook/rocksdb/actions/runs/11604597524/job/32313608085?pr=13104. After I added the check for the status returned by `MaybeReadBlockAndLoadToCache`, they all pass.

I also ran these tests manually (I had to run `make clean` before):

```
make -j64 block_based_table_reader_test COMPILE_WITH_ASAN=1 ASSERT_STATUS_CHECKED=1

 ./block_based_table_reader_test --gtest_filter="*StrictCapacityLimitReaderTest.Get*"
 ./block_based_table_reader_test --gtest_filter="*StrictCapacityLimitReaderTest.MultiGet*"

```

Reviewed By: anand1976

Differential Revision: D65302470

Pulled By: archang19

fbshipit-source-id: 28dcc381e67e05a89fa9fc9607b4709976d6d90e
2024-11-01 13:22:27 -07:00
..
adaptive Fix race to make BlockBasedTableOptions effectively mutable (#13082) 2024-10-25 10:24:54 -07:00
block_based Update MultiGet to respect the strict_capacity_limit block cache option (#13104) 2024-11-01 13:22:27 -07:00
cuckoo Fix race to make BlockBasedTableOptions effectively mutable (#13082) 2024-10-25 10:24:54 -07:00
plain Fix race to make BlockBasedTableOptions effectively mutable (#13082) 2024-10-25 10:24:54 -07:00
block_fetcher.cc Add ticker stats for read corruption retries (#12923) 2024-08-12 15:32:07 -07:00
block_fetcher.h Fix stale memory access with FSBuffer and tiered sec cache (#12712) 2024-05-30 12:33:58 -07:00
block_fetcher_test.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
cleanable_test.cc Eliminate unnecessary (slow) block cache Ref()ing in MultiGet (#9899) 2022-04-26 21:59:24 -07:00
compaction_merging_iterator.cc Fix possible double-free on TruncatedRangeDelIterator (#12805) 2024-06-24 11:51:16 -07:00
compaction_merging_iterator.h Fix possible double-free on TruncatedRangeDelIterator (#12805) 2024-06-24 11:51:16 -07:00
format.cc Fix a couple of missing cases of retry on corruption (#13007) 2024-09-13 13:56:49 -07:00
format.h Fix a couple of missing cases of retry on corruption (#13007) 2024-09-13 13:56:49 -07:00
get_context.cc Fix kBlockCacheTier read when merge-chain base value is in a blob file (#12462) 2024-03-21 12:38:53 -07:00
get_context.h Fix kBlockCacheTier read when merge-chain base value is in a blob file (#12462) 2024-03-21 12:38:53 -07:00
internal_iterator.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
iter_heap.h Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
iterator.cc Fix a bug for surfacing write unix time (#13057) 2024-10-08 11:31:51 -07:00
iterator_wrapper.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
merger_test.cc internal_repo_rocksdb (435146444452818992) (#12115) 2023-12-01 11:15:17 -08:00
merging_iterator.cc Fix possible double-free on TruncatedRangeDelIterator (#12805) 2024-06-24 11:51:16 -07:00
merging_iterator.h Fix possible double-free on TruncatedRangeDelIterator (#12805) 2024-06-24 11:51:16 -07:00
meta_blocks.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
meta_blocks.h Add some checks at property block creation side (#12898) 2024-07-31 13:28:17 -07:00
mock_table.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
mock_table.h Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
multiget_context.h Add a new MultiGetEntity API (#11222) 2023-02-15 09:34:17 -08:00
persistent_cache_helper.cc Remove extra semi colon from internal_repo_rocksdb/repo/db/table_cache_sync_and_async.h 2024-03-04 06:34:44 -08:00
persistent_cache_helper.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
persistent_cache_options.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
sst_file_dumper.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
sst_file_dumper.h Augment sst_dump tool to verify num_entries in table property (#12322) 2024-02-01 14:35:03 -08:00
sst_file_reader.cc Remove extra semi colon from internal_repo_rocksdb/repo/table/sst_file_reader.cc 2024-05-22 07:14:52 -07:00
sst_file_reader_test.cc Add support in SstFileReader to get a raw table iterator (#12385) 2024-04-02 21:23:06 -07:00
sst_file_writer.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
sst_file_writer_collectors.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
table_builder.h Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
table_factory.cc Remove FactoryFunc from LoadXXXObject (#11203) 2023-02-17 12:54:07 -08:00
table_iterator.h Add support in SstFileReader to get a raw table iterator (#12385) 2024-04-02 21:23:06 -07:00
table_properties.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
table_properties_internal.h Improve / clean up meta block code & integrity (#9163) 2021-11-18 11:43:44 -08:00
table_reader.h Deshim coro in fbcode/internal_repo_rocksdb 2024-09-14 09:48:21 -07:00
table_reader_bench.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
table_test.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
two_level_iterator.cc internal_repo_rocksdb (435146444452818992) (#12115) 2023-12-01 11:15:17 -08:00
two_level_iterator.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
unique_id.cc internal_repo_rocksdb (435146444452818992) (#12115) 2023-12-01 11:15:17 -08:00
unique_id_impl.h Derive cache keys from SST unique IDs (#10394) 2022-08-12 13:49:49 -07:00