rocksdb/db/compaction
Yueh-Hsuan Chiang e267909ecf Enable a multi-level db to smoothly migrate to FIFO via DB::Open (#10348)
Summary:
FIFO compaction can theoretically open a DB with any compaction style.
However, the current code only allows FIFO compaction to open a DB with
a single level.

This PR relaxes the limitation of FIFO compaction and allows it to open a
DB with multiple levels.  Below is the read / write / compaction behavior:

* The read behavior is untouched, and it works like a regular rocksdb instance.
* The write behavior is untouched as well.  When a FIFO compacted DB
is opened with multiple levels, all new files will still be in level 0, and no files
will be moved to a different level.
* Compaction logic is extended.  It will first identify the bottom-most non-empty level.
Then, it will delete the oldest file in that level.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10348

Test Plan:
Added a new test to verify the migration from level to FIFO where the db has multiple levels.
Extended existing test cases in db_test and db_basic_test to also verify
all entries of a key after reopening the DB with FIFO compaction.

Reviewed By: jay-zhuang

Differential Revision: D40233744

fbshipit-source-id: 6cc011d6c3467e6bfb9b6a4054b87619e69815e1
2022-10-18 14:38:13 -07:00
..
clipping_iterator.h Make InternalKeyComparator not configurable (#10342) 2022-07-14 10:09:31 -07:00
clipping_iterator_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
compaction.cc Allow the last level data moving up to penultimate level (#10782) 2022-10-10 22:50:34 -07:00
compaction.h Exclude timestamp when checking compaction boundaries (#10787) 2022-10-07 14:11:23 -07:00
compaction_iteration_stats.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
compaction_iterator.cc Add option preserve_internal_time_seconds to preserve the time info (#10747) 2022-10-07 18:49:40 -07:00
compaction_iterator.h Add option preserve_internal_time_seconds to preserve the time info (#10747) 2022-10-07 18:49:40 -07:00
compaction_iterator_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
compaction_job.cc Add option preserve_internal_time_seconds to preserve the time info (#10747) 2022-10-07 18:49:40 -07:00
compaction_job.h Add option preserve_internal_time_seconds to preserve the time info (#10747) 2022-10-07 18:49:40 -07:00
compaction_job_stats_test.cc Remove own ToString() (#9955) 2022-05-06 13:03:58 -07:00
compaction_job_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
compaction_outputs.cc Use sstableKeyCompare() for compaction output boundary check (#10763) 2022-10-06 15:54:58 -07:00
compaction_outputs.h Use sstableKeyCompare() for compaction output boundary check (#10763) 2022-10-06 15:54:58 -07:00
compaction_picker.cc Allow the last level data moving up to penultimate level (#10782) 2022-10-10 22:50:34 -07:00
compaction_picker.h Support subcmpct using reserved resources for round-robin priority (#10341) 2022-07-24 11:12:44 -07:00
compaction_picker_fifo.cc Enable a multi-level db to smoothly migrate to FIFO via DB::Open (#10348) 2022-10-18 14:38:13 -07:00
compaction_picker_fifo.h Add OpenAndTrimHistory API to support trimming data with specified timestamp (#9410) 2022-03-11 16:13:23 -08:00
compaction_picker_level.cc RoundRobin TTL compaction (#10725) 2022-10-04 14:53:32 -07:00
compaction_picker_level.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
compaction_picker_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
compaction_picker_universal.cc Fix overlapping check by excluding timestamp (#10615) 2022-09-08 13:03:07 -07:00
compaction_picker_universal.h Make ImmutableOptions struct that inherits from ImmutableCFOptions and ImmutableDBOptions (#8262) 2021-05-05 14:00:17 -07:00
compaction_service_job.cc Improve SubCompaction Partitioning (#10393) 2022-07-23 17:38:49 -07:00
compaction_service_test.cc Deflake CompactionServiceTest.BasicCompactions (#10697) 2022-09-19 14:10:05 -07:00
compaction_state.cc Tiered Compaction: per key placement support (#9964) 2022-07-13 20:54:49 -07:00
compaction_state.h Tiered Compaction: per key placement support (#9964) 2022-07-13 20:54:49 -07:00
file_pri.h Try to start TTL earlier with kMinOverlappingRatio is used (#8749) 2021-11-01 14:36:31 -07:00
sst_partitioner.cc Restore Regex support for ObjectLibrary::Register, rename new APIs to allow old one to be deprecated in the future (#9362) 2022-01-11 06:33:48 -08:00
subcompaction_state.cc Refactor Compaction file cut ShouldStopBefore() (#10629) 2022-09-14 22:09:12 -07:00
subcompaction_state.h Refactor Compaction file cut ShouldStopBefore() (#10629) 2022-09-14 22:09:12 -07:00
tiered_compaction_test.cc Allow the last level data moving up to penultimate level (#10782) 2022-10-10 22:50:34 -07:00