dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/table/block_based
History
Andrew Ryan Chang 7c98a2d130 Update MultiGet to respect the strict_capacity_limit block cache option (#13104) 
Summary:
There is a `strict_capacity_limit` option which imposes a hard memory limit on the block cache. When the block cache is enabled, every read request is serviced from the block cache. If the required block is missing, it is first inserted into the cache. If `strict_capacity_limit` is `true` and the limit has been reached, the `Get` and `MultiGet` requests should fail. However, currently this is not happening for `MultiGet`.

I updated `MultiGet` to explicitly check the returned status of `MaybeReadBlockAndLoadToCache`, so the status does not get overwritten later.

Thank you anand1976 for the problem explanation.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/13104

Test Plan:
Added unit test for both `Get` and `MultiGet` with a `strict_capacity_limit` set.

Before the change, half of my unit test cases failed https://github.com/facebook/rocksdb/actions/runs/11604597524/job/32313608085?pr=13104. After I added the check for the status returned by `MaybeReadBlockAndLoadToCache`, they all pass.

I also ran these tests manually (I had to run `make clean` before):

```
make -j64 block_based_table_reader_test COMPILE_WITH_ASAN=1 ASSERT_STATUS_CHECKED=1

 ./block_based_table_reader_test --gtest_filter="*StrictCapacityLimitReaderTest.Get*"
 ./block_based_table_reader_test --gtest_filter="*StrictCapacityLimitReaderTest.MultiGet*"

```

Reviewed By: anand1976

Differential Revision: D65302470

Pulled By: archang19

fbshipit-source-id: 28dcc381e67e05a89fa9fc9607b4709976d6d90e
 		2024-11-01 13:22:27 -07:00
..
binary_search_index_reader.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
binary_search_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
block.h Add some per key optimization for UDT in memtable only feature (#13031) 2024-10-03 17:57:50 -07:00
block_based_table_builder.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
block_based_table_builder.h Fix/cleanup SeqnoToTimeMapping (#12253) 2024-01-19 21:50:38 -08:00
block_based_table_factory.cc Fix race to make BlockBasedTableOptions effectively mutable (#13082) 2024-10-25 10:24:54 -07:00
block_based_table_factory.h Fix race to make BlockBasedTableOptions effectively mutable (#13082) 2024-10-25 10:24:54 -07:00
block_based_table_iterator.cc Trim readahead size based on prefix during prefix scan (#13040) 2024-10-17 15:52:55 -07:00
block_based_table_iterator.h Trim readahead size based on prefix during prefix scan (#13040) 2024-10-17 15:52:55 -07:00
block_based_table_reader.cc Add an ingestion option to not fill block cache (#13067) 2024-10-16 14:11:22 -07:00
block_based_table_reader.h Add an ingestion option to not fill block cache (#13067) 2024-10-16 14:11:22 -07:00
block_based_table_reader_impl.h Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
block_based_table_reader_sync_and_async.h Update MultiGet to respect the strict_capacity_limit block cache option (#13104) 2024-11-01 13:22:27 -07:00
block_based_table_reader_test.cc Update MultiGet to respect the strict_capacity_limit block cache option (#13104) 2024-11-01 13:22:27 -07:00
block_builder.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
block_builder.h Logically strip timestamp during flush (#11557) 2023-06-29 15:50:50 -07:00
block_cache.cc Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
block_cache.h Support pro-actively erasing obsolete block cache entries (#12694) 2024-06-07 08:57:11 -07:00
block_prefetcher.cc Respect ReadOptions::read_tier in prefetching (#12782) 2024-06-19 09:53:59 -07:00
block_prefetcher.h Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
block_prefix_index.cc Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_prefix_index.h Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_test.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
block_type.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
cachable_entry.h Support pro-actively erasing obsolete block cache entries (#12694) 2024-06-07 08:57:11 -07:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.cc Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
data_block_hash_index.h Fix build with gcc 13 by including <cstdint> (#11118) 2023-01-25 14:30:32 -08:00
data_block_hash_index_test.cc Record newest_key_time as a table property (#13083) 2024-11-01 10:08:35 -07:00
filter_block.h Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
filter_block_reader_common.cc Remove redundant no_io parameters to filter functions (#12762) 2024-06-12 18:47:11 -07:00
filter_block_reader_common.h Remove redundant no_io parameters to filter functions (#12762) 2024-06-12 18:47:11 -07:00
filter_policy.cc More valgrind fixes (#12990) 2024-09-06 10:11:34 -07:00
filter_policy_internal.h Optimize, simplify filter block building (fix regression) (#12931) 2024-08-14 15:13:16 -07:00
flush_block_policy.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
flush_block_policy_impl.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
full_filter_block.cc Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
full_filter_block.h Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
full_filter_block_test.cc Optimize, simplify filter block building (fix regression) (#12931) 2024-08-14 15:13:16 -07:00
hash_index_reader.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
hash_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
index_builder.cc Refactor IndexBuilder::AddIndexEntry (#12867) 2024-07-22 14:27:31 -07:00
index_builder.h Refactor IndexBuilder::AddIndexEntry (#12867) 2024-07-22 14:27:31 -07:00
index_reader_common.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
index_reader_common.h Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
mock_block_based_table.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
parsed_full_filter_block.cc Hide FilterBits{Builder,Reader} from public API (#9592) 2022-02-17 16:34:46 -08:00
parsed_full_filter_block.h Major Cache refactoring, CPU efficiency improvement (#10975) 2023-01-11 14:20:40 -08:00
partitioned_filter_block.cc Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
partitioned_filter_block.h Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
partitioned_filter_block_test.cc Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
partitioned_index_iterator.cc Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
partitioned_index_iterator.h Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
partitioned_index_reader.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
partitioned_index_reader.h Support pro-actively erasing obsolete block cache entries (#12694) 2024-06-07 08:57:11 -07:00
reader_common.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
reader_common.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
uncompression_dict_reader.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
uncompression_dict_reader.h Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00