rocksdb/db/db_impl
zczhu 30141461f9 Add basic kRoundRobin compaction policy (#10107)
Summary:
Add `kRoundRobin` as a compaction priority. The implementation is as follows.

- Define a cursor as the smallest Internal key in the successor of the selected file. Add `vector<InternalKey> compact_cursor_` into `VersionStorageInfo` where each element (`InternalKey`) in `compact_cursor_` represents a cursor. In round-robin compaction policy, we just need to select the first file (assuming files are sorted) and also has the smallest InternalKey larger than/equal to the cursor. After a file is chosen, we create a new `Fsize` vector which puts the selected file is placed at the first position in `temp`, the next cursor is then updated as the smallest InternalKey in successor of the selected file (the above logic is implemented in `SortFileByRoundRobin`).
- After a compaction succeeds, typically `InstallCompactionResults()`, we choose the next cursor for the input level and save it to `edit`. When calling `LogAndApply`, we save the next cursor with its level into some local variable and finally apply the change to `vstorage` in `SaveTo` function.
- Cursors are persist pair by pair (<level, InternalKey>) in `EncodeTo` so that they can be reconstructed when reopening. An empty cursor will not be encoded to MANIFEST

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10107

Test Plan: add unit test (`CompactionPriRoundRobin`) in `compaction_picker_test`, add `kRoundRobin` priority in `CompactionPriTest` from `db_compaction_test`, and add `PersistRoundRobinCompactCursor` in `db_compaction_test`

Reviewed By: ajkr

Differential Revision: D37316037

Pulled By: littlepig2013

fbshipit-source-id: 9f481748190ace416079139044e00df2968fb1ee
2022-06-21 11:56:53 -07:00
..
compacted_db_impl.cc Return "invalid argument" when read timestamp is too old (#10109) 2022-06-06 14:36:22 -07:00
compacted_db_impl.h Add timestamp support to CompactedDBImpl (#10030) 2022-05-24 12:14:10 -07:00
db_impl.cc Add blob source to retrieve blobs in RocksDB (#10198) 2022-06-20 20:58:11 -07:00
db_impl.h Fix race condition with WAL tracking and FlushWAL(true /* sync */) (#10185) 2022-06-17 16:45:28 -07:00
db_impl_compaction_flush.cc Add basic kRoundRobin compaction policy (#10107) 2022-06-21 11:56:53 -07:00
db_impl_debug.cc Use std::numeric_limits<> (#9954) 2022-05-05 13:08:21 -07:00
db_impl_experimental.cc Track SST unique id in MANIFEST and verify (#9990) 2022-05-19 11:04:21 -07:00
db_impl_files.cc Fix race condition with WAL tracking and FlushWAL(true /* sync */) (#10185) 2022-06-17 16:45:28 -07:00
db_impl_open.cc Add WriteOptions::protection_bytes_per_key (#10037) 2022-06-16 23:10:07 -07:00
db_impl_readonly.cc Return "invalid argument" when read timestamp is too old (#10109) 2022-06-06 14:36:22 -07:00
db_impl_readonly.h Add timestamp support to DBImplReadOnly (#10004) 2022-05-19 18:39:41 -07:00
db_impl_secondary.cc Consolidate manual_compaction_paused_ check (#10070) 2022-06-06 18:32:26 -07:00
db_impl_secondary.h Persist the new MANIFEST after successfully syncing the new WAL during recovery (#9922) 2022-06-01 10:52:26 -07:00
db_impl_write.cc Fix race condition with WAL tracking and FlushWAL(true /* sync */) (#10185) 2022-06-17 16:45:28 -07:00