rocksdb/db/compaction
Changyu Bi 2e8cc98ab2 Fix subcompaction bug to allow running two subcompactions (#11501)
Summary:
as reported in https://github.com/facebook/rocksdb/issues/11476, RocksDB currently does not execute compactions in two subcompactions even when they qualify. This PR fixes this issue.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11501

Test Plan:
* Add a new unit test.
* Run crash test with max_subcompactions=2: `python3 tools/db_crashtest.py blackbox --simple  --subcompactions=2 --target_file_size_base=1048576 --compaction_style=0`
  * saw logs showing compactions being executed as 2 subcompactions
```
2023/06/01-17:28:44.028470 3147486 (Original Log Time 2023/06/01-17:28:44.025972) EVENT_LOG_v1 {"time_micros": 1685665724025939, "job": 6, "event": "compaction_finished", "compaction_time_micros": 34539, "compaction_time_cpu_micros": 26404, "output_level": 6, "num_output_files": 2, "total_output_size": 1109796, "num_input_records": 13188, "num_output_records": 13021, "num_subcompactions": 2, "output_compression": "NoCompression", "num_single_delete_mismatches": 0, "num_single_delete_fallthrough": 0, "lsm_state": [0, 0, 0, 0, 0, 0, 13]}
```

Reviewed By: ajkr

Differential Revision: D46411497

Pulled By: cbi42

fbshipit-source-id: 3ebfc02e19f78f782e114a9546dc3d481d496258
2023-06-06 13:36:02 -07:00
..
clipping_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
clipping_iterator_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
compaction.cc Drop range tombstone during non-bottommost compaction (#11459) 2023-06-05 10:26:40 -07:00
compaction.h Drop range tombstone during non-bottommost compaction (#11459) 2023-06-05 10:26:40 -07:00
compaction_iteration_stats.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
compaction_iterator.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
compaction_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
compaction_iterator_test.cc Basic Support for Merge with user-defined timestamp (#10819) 2022-10-31 22:28:58 -07:00
compaction_job.cc Fix subcompaction bug to allow running two subcompactions (#11501) 2023-06-06 13:36:02 -07:00
compaction_job.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
compaction_job_stats_test.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
compaction_job_test.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
compaction_outputs.cc Drop range tombstone during non-bottommost compaction (#11459) 2023-06-05 10:26:40 -07:00
compaction_outputs.h Drop range tombstone during non-bottommost compaction (#11459) 2023-06-05 10:26:40 -07:00
compaction_picker.cc CompactRange() always compacts to bottommost level for leveled compaction (#11468) 2023-06-01 15:27:29 -07:00
compaction_picker.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_picker_fifo.cc Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
compaction_picker_fifo.h Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
compaction_picker_level.cc Try to pick more files in LevelCompactionBuilder::TryExtendNonL0TrivialMove() (#11347) 2023-04-14 11:50:20 -07:00
compaction_picker_level.h Sort L0 files by newly introduced epoch_num (#10922) 2022-12-13 13:29:37 -08:00
compaction_picker_test.cc Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
compaction_picker_universal.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
compaction_picker_universal.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_service_job.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_service_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
compaction_state.cc Tiered Compaction: per key placement support (#9964) 2022-07-13 20:54:49 -07:00
compaction_state.h Tiered Compaction: per key placement support (#9964) 2022-07-13 20:54:49 -07:00
file_pri.h Try to start TTL earlier with kMinOverlappingRatio is used (#8749) 2021-11-01 14:36:31 -07:00
sst_partitioner.cc Remove FactoryFunc from LoadXXXObject (#11203) 2023-02-17 12:54:07 -08:00
subcompaction_state.cc Refactor Compaction file cut ShouldStopBefore() (#10629) 2022-09-14 22:09:12 -07:00
subcompaction_state.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
tiered_compaction_test.cc Add WaitForCompact with WaitForCompactOptions to public API (#11436) 2023-05-25 17:25:51 -07:00