rocksdb/db/compaction
Jay Huh 586d78b31e Remove wait_unscheduled from waitForCompact internal API (#11443)
Summary:
Context:

In pull request https://github.com/facebook/rocksdb/issues/11436, we are introducing a new public API `waitForCompact(const WaitForCompactOptions& wait_for_compact_options)`. This API invokes the internal implementation `waitForCompact(bool wait_unscheduled=false)`. The unscheduled parameter indicates the compactions that are not yet scheduled but are required to process items in the queue.

In certain cases, we are unable to wait for compactions, such as during a shutdown or when background jobs are paused. It is important to return the appropriate status in these scenarios. For all other cases, we should wait for all compaction and flush jobs, including the unscheduled ones. The primary purpose of this new API is to wait until the system has resolved its compaction debt. Currently, the usage of `wait_unscheduled` is limited to test code.

This pull request eliminates the usage of wait_unscheduled. The internal `waitForCompact()` API now waits for unscheduled compactions unless the db is undergoing a shutdown. In the event of a shutdown, the API returns `Status::ShutdownInProgress()`.

Additionally, a new parameter, `abort_on_pause`, has been introduced with a default value of `false`. This parameter addresses the possibility of waiting indefinitely for unscheduled jobs if `PauseBackgroundWork()` was called before `waitForCompact()` is invoked. By setting `abort_on_pause` to `true`, the API will immediately return `Status::Aborted`.

Furthermore, all tests that previously called `waitForCompact(true)` have been fixed.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11443

Test Plan:
Existing tests that involve a shutdown in progress:

- DBCompactionTest::CompactRangeShutdownWhileDelayed
- DBTestWithParam::PreShutdownMultipleCompaction
- DBTestWithParam::PreShutdownCompactionMiddle

Reviewed By: pdillinger

Differential Revision: D45923426

Pulled By: jaykorean

fbshipit-source-id: 7dc93fe6a6841a7d9d2d72866fa647090dba8eae
2023-05-17 18:13:50 -07:00
..
clipping_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
clipping_iterator_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
compaction.cc Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
compaction.h Drain unnecessary levels when level_compaction_dynamic_level_bytes=true (#11340) 2023-04-06 11:20:43 -07:00
compaction_iteration_stats.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
compaction_iterator.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
compaction_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
compaction_iterator_test.cc Basic Support for Merge with user-defined timestamp (#10819) 2022-10-31 22:28:58 -07:00
compaction_job.cc Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
compaction_job.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
compaction_job_stats_test.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
compaction_job_test.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
compaction_outputs.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
compaction_outputs.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
compaction_picker.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
compaction_picker.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_picker_fifo.cc Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
compaction_picker_fifo.h Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
compaction_picker_level.cc Try to pick more files in LevelCompactionBuilder::TryExtendNonL0TrivialMove() (#11347) 2023-04-14 11:50:20 -07:00
compaction_picker_level.h Sort L0 files by newly introduced epoch_num (#10922) 2022-12-13 13:29:37 -08:00
compaction_picker_test.cc Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
compaction_picker_universal.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
compaction_picker_universal.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_service_job.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compaction_service_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
compaction_state.cc Tiered Compaction: per key placement support (#9964) 2022-07-13 20:54:49 -07:00
compaction_state.h Tiered Compaction: per key placement support (#9964) 2022-07-13 20:54:49 -07:00
file_pri.h Try to start TTL earlier with kMinOverlappingRatio is used (#8749) 2021-11-01 14:36:31 -07:00
sst_partitioner.cc Remove FactoryFunc from LoadXXXObject (#11203) 2023-02-17 12:54:07 -08:00
subcompaction_state.cc Refactor Compaction file cut ShouldStopBefore() (#10629) 2022-09-14 22:09:12 -07:00
subcompaction_state.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
tiered_compaction_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00