mirror of https://github.com/facebook/rocksdb.git
56463d443d
Summary: The subcompaction logic currently picks file boundaries as subcompaction boundaries. This is not compatible with user-defined timestamps because of two issues. Issue1: ReadOptions.iterate_lower_bound and ReadOptions.iterate_upper_bound contains timestamps which results in assertion failure as BlockBasedTableIterator expects bounds to be without timestamps. As result, because of wrong comparison end key is returned as user_key resulting in assertion failure. Issue2: Since it might result in two keys that only differ by user timestamp getting processed by two different subcompactions (and thus two different CompactionIterator state machines), which in turn can cause data correction issues. This PR provide support to reenable subcompactions with user-defined timestamps. Pull Request resolved: https://github.com/facebook/rocksdb/pull/10344 Test Plan: Added new unit test - Without fix for Issue1 unit test MultipleSubCompactions fails with error: ``` db_with_timestamp_compaction_test: ./db/compaction/clipping_iterator.h:247: void rocksdb::ClippingIterat│ or::AssertBounds(): Assertion `!valid_ || !end_ || cmp_->Compare(key(), *end_) < 0' failed. Received signal 6 (Aborted) │ #0 /usr/local/fbcode/platform009/lib/libc.so.6(gsignal+0x100) [0x7f8fbbbfe530] db_with_timestamp_compaction_test: ./db/compaction/clipping_iterator.h:247: void rocksdb::ClippingIterator::AssertBounds(): Assertion `!valid_ || !end_ || cmp_->Compare(key(), *end_) < 0' failed. Aborted (core dumped) ``` Ran stress test `make crash_test_with_ts -j32` Reviewed By: riversand963 Differential Revision: D38220841 Pulled By: akankshamahajan15 fbshipit-source-id: 5d5cae2bd37fcaeba1e77fce0a69070ad4158ccb |
||
|---|---|---|
| .. | ||
| clipping_iterator.h | ||
| clipping_iterator_test.cc | ||
| compaction.cc | ||
| compaction.h | ||
| compaction_iteration_stats.h | ||
| compaction_iterator.cc | ||
| compaction_iterator.h | ||
| compaction_iterator_test.cc | ||
| compaction_job.cc | ||
| compaction_job.h | ||
| compaction_job_stats_test.cc | ||
| compaction_job_test.cc | ||
| compaction_outputs.cc | ||
| compaction_outputs.h | ||
| compaction_picker.cc | ||
| compaction_picker.h | ||
| compaction_picker_fifo.cc | ||
| compaction_picker_fifo.h | ||
| compaction_picker_level.cc | ||
| compaction_picker_level.h | ||
| compaction_picker_test.cc | ||
| compaction_picker_universal.cc | ||
| compaction_picker_universal.h | ||
| compaction_service_job.cc | ||
| compaction_service_test.cc | ||
| compaction_state.cc | ||
| compaction_state.h | ||
| file_pri.h | ||
| sst_partitioner.cc | ||
| subcompaction_state.cc | ||
| subcompaction_state.h | ||
| tiered_compaction_test.cc | ||