dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-12-04 20:02:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/table/block_based
History
Peter Dillinger 9d5c8c89a1 Fix filter partition size logic (#12904) 
Summary:
Was checking == a desired number of entries added to a filter, when the combination of whole key and prefix filtering could add more than one entry per table internal key. This could lead to unnecessarily large filter partitions, which could affect performance and block cache fairness.

Also (only somewhat related because of other work in progress):
* Some variable renaming and a new assertion in BlockBasedTableBuilder, to add some clarity.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12904

Test Plan:
If you add assertion logic to the base revision checking that the partition cut is requested whenever `keys_added_to_partition_ >= keys_per_partition_`, it fails on a number of db_bloom_filter_test tests. However, such an assertion in the revised code would be essentially redundant with the new logic.

If I added a regression test for this, it would be tricky and fragile, so I don't think it's important enough to chase and maintain.  (Open to suggestions / input.)

Reviewed By: jowlyzhang

Differential Revision: D60557827

Pulled By: pdillinger

fbshipit-source-id: 77a56097d540da6e7851941a26d26ced2d944373
2024-08-02 14:49:02 -07:00
..
binary_search_index_reader.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
binary_search_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
block.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
block_based_table_builder.cc Fix filter partition size logic (#12904) 2024-08-02 14:49:02 -07:00
block_based_table_builder.h Fix/cleanup SeqnoToTimeMapping (#12253) 2024-01-19 21:50:38 -08:00
block_based_table_factory.cc Prevent data block compression with BlockBasedTableOptions::block_align (#12592) 2024-04-26 20:05:30 -07:00
block_based_table_factory.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
block_based_table_iterator.cc Ignore more non-critical IO error in BlockCacheLookupForReadAheadSize() in crash test (#12822) 2024-06-28 12:01:13 -07:00
block_based_table_iterator.h Lazily construct BlockBasedTableIterator::block_handles_ (#12616) 2024-05-03 17:18:13 -07:00
block_based_table_reader.cc Support ingesting SST files generated by a live DB (#12750) 2024-07-19 16:14:54 -07:00
block_based_table_reader.h Remove redundant no_io parameters to filter functions (#12762) 2024-06-12 18:47:11 -07:00
block_based_table_reader_impl.h Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
block_based_table_reader_sync_and_async.h Remove redundant no_io parameters to filter functions (#12762) 2024-06-12 18:47:11 -07:00
block_based_table_reader_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
block_builder.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
block_builder.h Logically strip timestamp during flush (#11557) 2023-06-29 15:50:50 -07:00
block_cache.cc Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
block_cache.h Support pro-actively erasing obsolete block cache entries (#12694) 2024-06-07 08:57:11 -07:00
block_prefetcher.cc Respect ReadOptions::read_tier in prefetching (#12782) 2024-06-19 09:53:59 -07:00
block_prefetcher.h Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
block_prefix_index.cc Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_prefix_index.h Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_test.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
block_type.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
cachable_entry.h Support pro-actively erasing obsolete block cache entries (#12694) 2024-06-07 08:57:11 -07:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.cc Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
data_block_hash_index.h Fix build with gcc 13 by including <cstdint> (#11118) 2023-01-25 14:30:32 -08:00
data_block_hash_index_test.cc Rename IntTblPropCollector -> InternalTblPropColl (#12320) 2024-02-02 14:14:43 -08:00
filter_block.h Clean up/refactor (Partitioned)FilterBlockBuilder (#12903) 2024-08-02 13:35:45 -07:00
filter_block_reader_common.cc Remove redundant no_io parameters to filter functions (#12762) 2024-06-12 18:47:11 -07:00
filter_block_reader_common.h Remove redundant no_io parameters to filter functions (#12762) 2024-06-12 18:47:11 -07:00
filter_policy.cc Set optimize_filters_for_memory by default (#12377) 2024-04-30 08:33:31 -07:00
filter_policy_internal.h Make RibbonFilterPolicy::bloom_before_level mutable (SetOptions()) (#11838) 2023-09-15 15:46:10 -07:00
flush_block_policy.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
flush_block_policy_impl.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
full_filter_block.cc Clean up/refactor (Partitioned)FilterBlockBuilder (#12903) 2024-08-02 13:35:45 -07:00
full_filter_block.h Clean up/refactor (Partitioned)FilterBlockBuilder (#12903) 2024-08-02 13:35:45 -07:00
full_filter_block_test.cc Clean up/refactor (Partitioned)FilterBlockBuilder (#12903) 2024-08-02 13:35:45 -07:00
hash_index_reader.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
hash_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
index_builder.cc Refactor IndexBuilder::AddIndexEntry (#12867) 2024-07-22 14:27:31 -07:00
index_builder.h Refactor IndexBuilder::AddIndexEntry (#12867) 2024-07-22 14:27:31 -07:00
index_reader_common.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
index_reader_common.h Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
mock_block_based_table.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
parsed_full_filter_block.cc Hide FilterBits{Builder,Reader} from public API (#9592) 2022-02-17 16:34:46 -08:00
parsed_full_filter_block.h Major Cache refactoring, CPU efficiency improvement (#10975) 2023-01-11 14:20:40 -08:00
partitioned_filter_block.cc Fix filter partition size logic (#12904) 2024-08-02 14:49:02 -07:00
partitioned_filter_block.h Clean up/refactor (Partitioned)FilterBlockBuilder (#12903) 2024-08-02 13:35:45 -07:00
partitioned_filter_block_test.cc Clean up/refactor (Partitioned)FilterBlockBuilder (#12903) 2024-08-02 13:35:45 -07:00
partitioned_index_iterator.cc Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
partitioned_index_iterator.h Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
partitioned_index_reader.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
partitioned_index_reader.h Support pro-actively erasing obsolete block cache entries (#12694) 2024-06-07 08:57:11 -07:00
reader_common.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
reader_common.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
uncompression_dict_reader.cc Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00
uncompression_dict_reader.h Eliminate some parameters redundant with ReadOptions (#12761) 2024-06-12 15:44:37 -07:00