rocksdb/db/db_impl
Changyu Bi b3c43a5b99 Drain unnecessary levels when level_compaction_dynamic_level_bytes=true (#11340)
Summary:
When a user migrates to level compaction + `level_compaction_dynamic_level_bytes=true`, or when a DB shrinks, there can be unnecessary levels in the DB. Before this PR, this is no way to remove these levels except a manual compaction. These extra unnecessary levels make it harder to guarantee max_bytes_for_level_multiplier and can cause extra space amp. This PR boosts compaction score for these levels to allow RocksDB to automatically drain these levels. Together with https://github.com/facebook/rocksdb/issues/11321, this makes migration to `level_compaction_dynamic_level_bytes=true` automatic without needing user to do a one time full manual compaction. Credit: this PR is modified from https://github.com/facebook/rocksdb/issues/3921.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11340

Test Plan:
- New unit tests
- `python3 tools/db_crashtest.py whitebox --simple` which randomly sets level_compaction_dynamic_level_bytes in each run.

Reviewed By: ajkr

Differential Revision: D44563884

Pulled By: cbi42

fbshipit-source-id: e20d3620bd73dff22be18c5a91a07f340740bcc8
2023-04-06 11:20:43 -07:00
..
compacted_db_impl.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compacted_db_impl.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_impl.cc Fix bug of prematurely excluded CF in atomic flush contains unflushed data that should've been included in the atomic flush (#11148) 2023-03-14 16:53:20 -07:00
db_impl.h Fix bug of prematurely excluded CF in atomic flush contains unflushed data that should've been included in the atomic flush (#11148) 2023-03-14 16:53:20 -07:00
db_impl_compaction_flush.cc Fix bug of prematurely excluded CF in atomic flush contains unflushed data that should've been included in the atomic flush (#11148) 2023-03-14 16:53:20 -07:00
db_impl_debug.cc Fix bug of prematurely excluded CF in atomic flush contains unflushed data that should've been included in the atomic flush (#11148) 2023-03-14 16:53:20 -07:00
db_impl_experimental.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_impl_files.cc Simplify TEST_F(DBWALTest, FixSyncWalOnObseletedWalWithNewManifestCausingMissingWAL) (#11186) 2023-02-06 16:10:03 -08:00
db_impl_open.cc Drain unnecessary levels when level_compaction_dynamic_level_bytes=true (#11340) 2023-04-06 11:20:43 -07:00
db_impl_readonly.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_impl_readonly.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_impl_secondary.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_impl_secondary.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_impl_write.cc New stat rocksdb.{cf|db}-write-stall-stats exposed in a structural way (#11300) 2023-03-18 09:51:58 -07:00