dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-28 05:43:50 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/table/block_based
History
Yu Zhang 84a54e1e28 Fix some bugs in index builder and reader for the UDT in memtable only feature (#12062) 
Summary:
These bugs surfaced while I was trying to add the stress test for the feature:

Bug 1) On the index building path: the optimization to use user key instead of internal key as separator needed a bit tweak for when user defined timestamps can be removed. Because even though the user key look different now and eligible to be used as separator, when their user-defined timestamps are removed, they could be equal and that invariant no longer stands.

Bug 2) On the index reading path: one path that builds the second level index iterator for `PartitionedIndexReader` are not passing the corresponding `user_defined_timestamps_persisted` flag. As a result, the default `true` value be used leading to no minimum timestamps padded when they should be.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12062

Test Plan:
For bug 1): added separate unit test `BlockBasedTableReaderTest::Get` to exercise the `Get` API. It's a different code path from `MultiGet` so worth having its own test. Also in order to cover the bug, the test is modified to generate key values with the same user provided key, different timestamps and different sequence numbers. The test reads back different versions of the same user provided key.  `MultiGet` takes one `ReadOptions` with one read timestamp so we cannot test retrieving different versions of the same key easily.

For bug 2): simply added options `BlockBasedTableOptions.metadata_cache_options.partition_pinning = PinningTier::kAll` to exercise all the index iterator creating paths.

Reviewed By: ltamasi

Differential Revision: D51508280

Pulled By: jowlyzhang

fbshipit-source-id: 8b174d3d70373c0599266ac1f467f2bd4d7ea6e5
2023-11-21 14:05:02 -08:00
..
binary_search_index_reader.cc Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
binary_search_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block.cc PutEntity Support in SST File Writer (#11688) 2023-08-10 18:16:10 -07:00
block.h Add support to strip / pad timestamp when writing / reading a block (#11472) 2023-05-25 15:41:32 -07:00
block_based_table_builder.cc Ensure and clarify how RocksDB calls TablePropertiesCollector's functions (#12053) 2023-11-08 14:00:36 -08:00
block_based_table_builder.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
block_based_table_factory.cc java API - load block based table config (#10826) 2023-10-12 09:39:01 -07:00
block_based_table_factory.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
block_based_table_iterator.cc Fix corruption error in stress test for auto_readahead_size enabled (#11961) 2023-10-17 12:21:08 -07:00
block_based_table_iterator.h Fix corruption error in stress test for auto_readahead_size enabled (#11961) 2023-10-17 12:21:08 -07:00
block_based_table_reader.cc Fix some bugs in index builder and reader for the UDT in memtable only feature (#12062) 2023-11-21 14:05:02 -08:00
block_based_table_reader.h Add some stats to observe the usefulness of scan prefetching (#11981) 2023-10-23 14:42:44 -07:00
block_based_table_reader_impl.h Fix various failures in auto_readahead_size (#11884) 2023-10-02 17:47:24 -07:00
block_based_table_reader_sync_and_async.h Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
block_based_table_reader_test.cc Fix some bugs in index builder and reader for the UDT in memtable only feature (#12062) 2023-11-21 14:05:02 -08:00
block_builder.cc Logically strip timestamp during flush (#11557) 2023-06-29 15:50:50 -07:00
block_builder.h Logically strip timestamp during flush (#11557) 2023-06-29 15:50:50 -07:00
block_cache.cc Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
block_cache.h Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
block_prefetcher.cc Add some stats to observe the usefulness of scan prefetching (#11981) 2023-10-23 14:42:44 -07:00
block_prefetcher.h Lookup ahead in block cache ahead to tune Readaheadsize (#11860) 2023-09-22 18:12:08 -07:00
block_prefix_index.cc Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_prefix_index.h Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_test.cc Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
block_type.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
cachable_entry.h HyperClockCache support for SecondaryCache, with refactoring (#11301) 2023-03-17 20:23:49 -07:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.cc Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
data_block_hash_index.h Fix build with gcc 13 by including <cstdint> (#11118) 2023-01-25 14:30:32 -08:00
data_block_hash_index_test.cc Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
filter_block.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
filter_block_reader_common.cc Avoid double block cache lookup during Seek with async_io option (#11616) 2023-09-18 11:32:30 -07:00
filter_block_reader_common.h Use user-provided ReadOptions for metadata block reads more often (#11208) 2023-04-04 16:53:14 -07:00
filter_policy.cc Make RibbonFilterPolicy::bloom_before_level mutable (SetOptions()) (#11838) 2023-09-15 15:46:10 -07:00
filter_policy_internal.h Make RibbonFilterPolicy::bloom_before_level mutable (SetOptions()) (#11838) 2023-09-15 15:46:10 -07:00
flush_block_policy.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
flush_block_policy_impl.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
full_filter_block.cc Use user-provided ReadOptions for metadata block reads more often (#11208) 2023-04-04 16:53:14 -07:00
full_filter_block.h Use user-provided ReadOptions for metadata block reads more often (#11208) 2023-04-04 16:53:14 -07:00
full_filter_block_test.cc Use user-provided ReadOptions for metadata block reads more often (#11208) 2023-04-04 16:53:14 -07:00
hash_index_reader.cc Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
hash_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
index_builder.cc Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
index_builder.h Fix some bugs in index builder and reader for the UDT in memtable only feature (#12062) 2023-11-21 14:05:02 -08:00
index_reader_common.cc Avoid double block cache lookup during Seek with async_io option (#11616) 2023-09-18 11:32:30 -07:00
index_reader_common.h Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
mock_block_based_table.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
parsed_full_filter_block.cc Hide FilterBits{Builder,Reader} from public API (#9592) 2022-02-17 16:34:46 -08:00
parsed_full_filter_block.h Major Cache refactoring, CPU efficiency improvement (#10975) 2023-01-11 14:20:40 -08:00
partitioned_filter_block.cc Add some stats to observe the usefulness of scan prefetching (#11981) 2023-10-23 14:42:44 -07:00
partitioned_filter_block.h Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
partitioned_filter_block_test.cc Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
partitioned_index_iterator.cc Lookup ahead in block cache ahead to tune Readaheadsize (#11860) 2023-09-22 18:12:08 -07:00
partitioned_index_iterator.h Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
partitioned_index_reader.cc Add some stats to observe the usefulness of scan prefetching (#11981) 2023-10-23 14:42:44 -07:00
partitioned_index_reader.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
reader_common.cc format_version=6 and context-aware block checksums (#9058) 2023-07-30 16:40:01 -07:00
reader_common.h format_version=6 and context-aware block checksums (#9058) 2023-07-30 16:40:01 -07:00
uncompression_dict_reader.cc Avoid double block cache lookup during Seek with async_io option (#11616) 2023-09-18 11:32:30 -07:00
uncompression_dict_reader.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00