rocksdb/db/compaction
Changyu Bi e7896f03ad Enable unit test PrecludeLastLevelTest.RangeDelsCauseFileEndpointsToOverlap (#12064)
Summary:
Fixes https://github.com/facebook/rocksdb/issues/11909. The test passes after the change in https://github.com/facebook/rocksdb/issues/11917 to start mock clock from a non-zero time.

The reason for test failing is a bit complicated:
- The Put here e4ad4a0ef1/db/compaction/tiered_compaction_test.cc (L2045) happens before mock clock advances beyond 0.
- This causes oldest_key_time_ to be 0 for memtable.
- oldest_ancester_time of the first L0 file becomes 0
- L0 -> L5/6 compaction output files sets `oldest_ancestoer_time` to the current time due to these lines: 509947ce2c/db/compaction/compaction_job.cc (L1898C34-L1904).
- This causes some small sequence number to be mapped to current time: 509947ce2c/db/compaction/compaction_job.cc (L301)
- Keys in L6 is being moved up to L5 due to the unexpected seqno_to_time mapping
- When compacting keys from last level to the penultimate level, we only check keys to be within user key range of penultimate level input files. If we compact the following file 3 with file 1 and output keys to L5, we can get the reported inconsistency bug.
```
L5: file 1 [K5@20, K10@kMaxSeqno], file 2 [K10@30, K14@34)
L6: file 3 [K6@5, K10@20]
```

https://github.com/facebook/rocksdb/issues/12063 will add fixes to check internal key range when compacting keys from last level up to the penultimate level.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12064

Test Plan: the unit test passes

Reviewed By: ajkr

Differential Revision: D51281149

Pulled By: cbi42

fbshipit-source-id: 00b7f026c453454d9f3af5b2de441383a96f0c62
2023-11-13 15:26:52 -08:00
..
clipping_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
clipping_iterator_test.cc
compaction.cc Expose more info about input files in CompactionFilter::Context (#11857) 2023-09-20 13:34:39 -07:00
compaction.h Expose more info about input files in CompactionFilter::Context (#11857) 2023-09-20 13:34:39 -07:00
compaction_iteration_stats.h
compaction_iterator.cc Add a helper method WideColumnsHelper::SortColumns (#11823) 2023-09-12 12:36:07 -07:00
compaction_iterator.h Compare the number of input keys and processed keys for compactions (#11571) 2023-07-28 09:47:31 -07:00
compaction_iterator_test.cc Compare the number of input keys and processed keys for compactions (#11571) 2023-07-28 09:47:31 -07:00
compaction_job.cc Fix comments about creation_time/oldest_ancester_time/oldest_key_time (#11921) 2023-10-04 14:42:35 -07:00
compaction_job.h Refactor, clean up, fixes, and more testing for SeqnoToTimeMapping (#11905) 2023-09-29 11:21:59 -07:00
compaction_job_stats_test.cc Make option level_compaction_dynamic_level_bytes true by default (#11525) 2023-06-15 21:12:39 -07:00
compaction_job_test.cc Quarantine files in a limbo state after a manifest error (#12030) 2023-11-11 08:11:11 -08:00
compaction_outputs.cc Refactor, clean up, fixes, and more testing for SeqnoToTimeMapping (#11905) 2023-09-29 11:21:59 -07:00
compaction_outputs.h Refactor, clean up, fixes, and more testing for SeqnoToTimeMapping (#11905) 2023-09-29 11:21:59 -07:00
compaction_picker.cc Do not include last level in compaction when allow_ingest_behind=true (#11489) 2023-06-14 11:28:56 -07:00
compaction_picker.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_picker_fifo.cc New tickers on deletion compactions grouped by reasons (#11957) 2023-10-18 18:00:07 -07:00
compaction_picker_fifo.h Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
compaction_picker_level.cc Try to pick more files in LevelCompactionBuilder::TryExtendNonL0TrivialMove() (#11347) 2023-04-14 11:50:20 -07:00
compaction_picker_level.h
compaction_picker_test.cc Mark more files for periodic compaction during offpeak (#12031) 2023-11-06 11:43:59 -08:00
compaction_picker_universal.cc Conditionally exclude some L0 input files in size amp compaction (#11749) 2023-09-12 15:53:15 -07:00
compaction_picker_universal.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_service_job.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_service_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
compaction_state.cc
compaction_state.h
file_pri.h Avoid shifting component too large error in FileTtlBooster (#11673) 2023-08-04 14:29:50 -07:00
sst_partitioner.cc Remove FactoryFunc from LoadXXXObject (#11203) 2023-02-17 12:54:07 -08:00
subcompaction_state.cc
subcompaction_state.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
tiered_compaction_test.cc Enable unit test PrecludeLastLevelTest.RangeDelsCauseFileEndpointsToOverlap (#12064) 2023-11-13 15:26:52 -08:00