mirror of https://github.com/facebook/rocksdb.git
a1a102ffce
Summary: With some new use cases onboarding to prefix extractors/seek/filters, one of the risks is existing iterator code, e.g. for maintenance tasks, being unintentionally subject to prefix seek semantics. This is a longstanding known design flaw with prefix seek, and `prefix_same_as_start` and `auto_prefix_mode` were steps in the direction of making that obsolete. However, we can't just immediately set `total_order_seek` to true by default, because that would impact so much code instantly. Here we add a new DB option, `prefix_seek_opt_in_only` that basically allows users to transition to the future behavior when they are ready. When set to true, all iterators will be treated as if `total_order_seek=true` and then the only ways to get prefix seek semantics are with `prefix_same_as_start` or `auto_prefix_mode`. Related fixes / changes: * Make sure that `prefix_same_as_start` and `auto_prefix_mode` are compatible with (or override) `total_order_seek` (depending on your interpretation). * Fix a bug in which a new iterator after dynamically changing the prefix extractor might mix different prefix semantics between memtable and SSTs. Both should use the latest extractor semantics, which means iterators ignoring memtable prefix filters with an old extractor. And that means passing the latest prefix extractor to new memtable iterators that might use prefix seek. (Without the fix, the test added for this fails in many ways.) Suggested follow-up: * Investigate a FIXME where a MergeIteratorBuilder is created in db_impl.cc. No unit test detects a change in value that should impact correctness. * Make memtable prefix bloom compatible with `auto_prefix_mode`, which might require involving the memtablereps because we don't know at iterator creation time (only seek time) whether an auto_prefix_mode seek will be a prefix seek. * Add `prefix_same_as_start` testing to db_stress Pull Request resolved: https://github.com/facebook/rocksdb/pull/13026 Test Plan: tests updated, added. Add combination of `total_order_seek=true` and `auto_prefix_mode=true` to stress test. Ran `make blackbox_crash_test` for a long while. Manually ran tests with `prefix_seek_opt_in_only=true` as default, looking for unexpected issues. I inspected most of the results and migrated many tests to be ready for such a change (but not all). Reviewed By: ltamasi Differential Revision: D63147378 Pulled By: pdillinger fbshipit-source-id: 1f4477b730683d43b4be7e933338583702d3c25e |
||
|---|---|---|
| .. | ||
| adaptive | ||
| block_based | ||
| cuckoo | ||
| plain | ||
| block_fetcher.cc | ||
| block_fetcher.h | ||
| block_fetcher_test.cc | ||
| cleanable_test.cc | ||
| compaction_merging_iterator.cc | ||
| compaction_merging_iterator.h | ||
| format.cc | ||
| format.h | ||
| get_context.cc | ||
| get_context.h | ||
| internal_iterator.h | ||
| iter_heap.h | ||
| iterator.cc | ||
| iterator_wrapper.h | ||
| merger_test.cc | ||
| merging_iterator.cc | ||
| merging_iterator.h | ||
| meta_blocks.cc | ||
| meta_blocks.h | ||
| mock_table.cc | ||
| mock_table.h | ||
| multiget_context.h | ||
| persistent_cache_helper.cc | ||
| persistent_cache_helper.h | ||
| persistent_cache_options.h | ||
| sst_file_dumper.cc | ||
| sst_file_dumper.h | ||
| sst_file_reader.cc | ||
| sst_file_reader_test.cc | ||
| sst_file_writer.cc | ||
| sst_file_writer_collectors.h | ||
| table_builder.h | ||
| table_factory.cc | ||
| table_iterator.h | ||
| table_properties.cc | ||
| table_properties_internal.h | ||
| table_reader.h | ||
| table_reader_bench.cc | ||
| table_test.cc | ||
| two_level_iterator.cc | ||
| two_level_iterator.h | ||
| unique_id.cc | ||
| unique_id_impl.h | ||