rocksdb/db
Changyu Bi 71e38dbe25 Compact one file at a time for FIFO temperature change compactions (#13018)
Summary:
Per customer request, we should not merge multiple SST files together during temperature change compaction, since this can cause FIFO TTL compactions to be delayed. This PR changes the compaction picking logic to pick one file at a time.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/13018

Test Plan: * updated some existing unit tests to test this new behavior.

Reviewed By: jowlyzhang

Differential Revision: D62883292

Pulled By: cbi42

fbshipit-source-id: 6a9fc8c296b5d9b17168ef6645f25153241c8b93
2024-09-19 15:50:41 -07:00
..
blob Fixed MultiGet() error handling to not skip blob dereference (#12597) 2024-04-29 14:18:42 -07:00
compaction Compact one file at a time for FIFO temperature change compactions (#13018) 2024-09-19 15:50:41 -07:00
db_impl Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
wide Fix the output of ldb dump_wal for PutEntity records (#12677) 2024-05-20 17:04:14 -07:00
arena_wrapped_db_iter.cc Fix possible double-free on TruncatedRangeDelIterator (#12805) 2024-06-24 11:51:16 -07:00
arena_wrapped_db_iter.h Fix possible double-free on TruncatedRangeDelIterator (#12805) 2024-06-24 11:51:16 -07:00
attribute_group_iterator_impl.cc MultiCfIterator - AttributeGroupIter Impl & CoalescingIter Optimization (#12534) 2024-04-16 08:45:38 -07:00
attribute_group_iterator_impl.h MultiCfIterator - AttributeGroupIter Impl & CoalescingIter Optimization (#12534) 2024-04-16 08:45:38 -07:00
builder.cc Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
builder.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
c.cc Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
c_test.c Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
coalescing_iterator.cc MultiCfIterator - AttributeGroupIter Impl & CoalescingIter Optimization (#12534) 2024-04-16 08:45:38 -07:00
coalescing_iterator.h MultiCfIterator - AttributeGroupIter Impl & CoalescingIter Optimization (#12534) 2024-04-16 08:45:38 -07:00
column_family.cc Make compaction always use the input version with extra ref protection (#12992) 2024-09-06 14:07:33 -07:00
column_family.h Make compaction always use the input version with extra ref protection (#12992) 2024-09-06 14:07:33 -07:00
column_family_test.cc fix compaction speedup for marked files ut (#12912) 2024-08-09 15:05:02 -07:00
compact_files_test.cc Prevent data block compression with BlockBasedTableOptions::block_align (#12592) 2024-04-26 20:05:30 -07:00
comparator_db_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
convenience.cc Inject more errors to more files in stress test (#12713) 2024-06-19 08:42:00 -07:00
convenience_impl.h
corruption_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
cuckoo_table_db_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_basic_test.cc Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
db_block_cache_test.cc Fix rare failure in DBBlockCacheTypeTest.Uncache (#12775) 2024-06-14 20:50:36 -07:00
db_bloom_filter_test.cc Fix and generalize framework for filtering range queries, etc. (#13005) 2024-09-18 15:26:37 -07:00
db_clip_test.cc
db_compaction_filter_test.cc Replace ScopedArenaIterator with ScopedArenaPtr<InternalIterator> (#12470) 2024-03-22 13:40:42 -07:00
db_compaction_test.cc Compact one file at a time for FIFO temperature change compactions (#13018) 2024-09-19 15:50:41 -07:00
db_dynamic_level_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_encryption_test.cc
db_filesnapshot.cc Avoid unnecessary work in internal calls to GetSortedWalFiles (#12831) 2024-07-01 23:29:02 -07:00
db_flush_test.cc Refactor SyncWAL and SyncClosedLogs for code sharing (#12707) 2024-05-30 14:53:13 -07:00
db_follower_test.cc Move file tracking in VersionEditHandlerPointInTime to VersionBuilder (#12928) 2024-08-12 21:09:37 -07:00
db_info_dumper.cc Don't log an error when an auxiliary dir is missing (#12326) 2024-02-05 10:26:41 -08:00
db_info_dumper.h
db_inplace_update_test.cc
db_io_failure_test.cc Fix a couple of missing cases of retry on corruption (#13007) 2024-09-13 13:56:49 -07:00
db_iter.cc Add Iterator property "rocksdb.iterator.is-value-pinned" (#12659) 2024-05-15 19:11:52 -07:00
db_iter.h Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
db_iter_stress_test.cc Automated modernization (#12210) 2024-01-05 11:53:57 -08:00
db_iter_test.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
db_iterator_test.cc Add Iterator property "rocksdb.iterator.is-value-pinned" (#12659) 2024-05-15 19:11:52 -07:00
db_kv_checksum_test.cc Fix MultiGet dropping memtable kv checksum corruption (#12842) 2024-08-08 13:34:11 -07:00
db_log_iter_test.cc Disable flaky part of TransactionLogIteratorCheckWhenArchive (#12423) 2024-03-12 12:54:53 -07:00
db_logical_block_size_cache_test.cc
db_memtable_test.cc Add an option to verify memtable key order during reads (#12889) 2024-08-19 13:53:25 -07:00
db_merge_operand_test.cc Add ContinueCallback to GetMergeOperands() (#12438) 2024-03-15 12:25:49 -07:00
db_merge_operator_test.cc
db_options_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_properties_test.cc GetAggregatedIntProperty accumulates property once per block cache (#12755) 2024-06-18 10:46:55 -07:00
db_range_del_test.cc Fail DeleteRange() early when row_cache is configured (#12710) 2024-05-29 15:03:15 -07:00
db_rate_limiter_test.cc Fix db_rate_limiter_test for win (#12816) 2024-07-01 16:14:19 -07:00
db_readonly_with_timestamp_test.cc
db_secondary_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_sst_test.cc Fix orphaned files in SstFileManager (#13015) 2024-09-18 13:27:44 -07:00
db_statistics_test.cc Fix double counting of BYTES_WRITTEN ticker (#12111) 2023-12-08 17:12:11 -08:00
db_table_properties_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_tailing_iter_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_test.cc Disable "uncache" behavior in DB shutdown (#12751) 2024-06-11 15:57:40 -07:00
db_test2.cc Fix flaky test DBTest2.VariousFileTemperatures (#12974) 2024-08-26 14:08:21 -07:00
db_test_util.cc Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
db_test_util.h Options for file temperature for more files (#12957) 2024-08-23 19:49:25 -07:00
db_universal_compaction_test.cc Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
db_wal_test.cc Fix a bug with auto recovery on WAL write error (#12995) 2024-09-17 14:10:33 -07:00
db_with_timestamp_basic_test.cc Fix MultiGet with timestamps (#12943) 2024-08-23 13:58:04 -07:00
db_with_timestamp_compaction_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_with_timestamp_test_util.cc
db_with_timestamp_test_util.h
db_write_buffer_manager_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_write_test.cc Disallow memtable flush and sst ingest while WAL is locked (#12652) 2024-05-21 10:17:34 -07:00
dbformat.cc Add EntryType for TimedPut (#12669) 2024-05-16 15:18:12 -07:00
dbformat.h Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
dbformat_test.cc Refactor IndexBuilder::AddIndexEntry (#12867) 2024-07-22 14:27:31 -07:00
deletefile_test.cc Add an option to wait for purge in WaitForCompact (#12520) 2024-04-17 17:33:27 -07:00
error_handler.cc Fix a bug with auto recovery on WAL write error (#12995) 2024-09-17 14:10:33 -07:00
error_handler.h Fix a bug with auto recovery on WAL write error (#12995) 2024-09-17 14:10:33 -07:00
error_handler_fs_test.cc Remove the return value of SetBGError() (#12792) 2024-06-26 18:17:05 -07:00
event_helpers.cc Record largest seqno in table properties and verify in file ingestion (#12951) 2024-08-21 16:24:18 -07:00
event_helpers.h
experimental.cc Fix and generalize framework for filtering range queries, etc. (#13005) 2024-09-18 15:26:37 -07:00
external_sst_file_basic_test.cc Disallow memtable flush and sst ingest while WAL is locked (#12652) 2024-05-21 10:17:34 -07:00
external_sst_file_ingestion_job.cc Add a new file ingestion option link_files (#12980) 2024-09-03 13:06:25 -07:00
external_sst_file_ingestion_job.h Use extended file boundary for key range overlap check during file ingestion (#12735) 2024-06-04 13:39:51 -07:00
external_sst_file_test.cc Add a new file ingestion option link_files (#12980) 2024-09-03 13:06:25 -07:00
fault_injection_test.cc FaultInjectionTestFS follow-up and clean-up (#12861) 2024-07-15 10:28:34 -07:00
file_indexer.cc
file_indexer.h
file_indexer_test.cc
filename_test.cc
flush_job.cc Make compaction always use the input version with extra ref protection (#12992) 2024-09-06 14:07:33 -07:00
flush_job.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
flush_job_test.cc Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
flush_scheduler.cc
flush_scheduler.h
forward_iterator.cc Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
forward_iterator.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
forward_iterator_bench.cc
history_trimming_iterator.h
import_column_family_job.cc Fix a corruption bug in CreateColumnFamilyWithImport() (#12602) 2024-05-06 11:01:38 -07:00
import_column_family_job.h
import_column_family_test.cc Fix a corruption bug in CreateColumnFamilyWithImport() (#12602) 2024-05-06 11:01:38 -07:00
internal_stats.cc GetAggregatedIntProperty accumulates property once per block cache (#12755) 2024-06-18 10:46:55 -07:00
internal_stats.h GetAggregatedIntProperty accumulates property once per block cache (#12755) 2024-06-18 10:46:55 -07:00
job_context.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
kv_checksum.h
listener_test.cc Add some checks at property block creation side (#12898) 2024-07-31 13:28:17 -07:00
log_format.h
log_reader.cc Fix recycled WAL detection when wal_compression is enabled (#12643) 2024-05-22 15:34:37 -07:00
log_reader.h Enable recycle_log_file_num option for point in time recovery (#12403) 2024-03-21 12:29:35 -07:00
log_test.cc Fix recycled WAL detection when wal_compression is enabled (#12643) 2024-05-22 15:34:37 -07:00
log_writer.cc Handle injected write error after successful WAL write in crash test + misc (#12838) 2024-07-29 13:51:49 -07:00
log_writer.h Ensure Close() before LinkFile() for WALs in Checkpoint (#12734) 2024-06-12 11:48:45 -07:00
logs_with_prep_tracker.cc
logs_with_prep_tracker.h
lookup_key.h
malloc_stats.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
malloc_stats.h
manual_compaction_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
memtable.cc Add an option to verify memtable key order during reads (#12889) 2024-08-19 13:53:25 -07:00
memtable.h Add an option to verify memtable key order during reads (#12889) 2024-08-19 13:53:25 -07:00
memtable_list.cc Add an internal API MemTableList::GetEditForDroppingCurrentVersion (#13001) 2024-09-10 13:23:13 -07:00
memtable_list.h Add an internal API MemTableList::GetEditForDroppingCurrentVersion (#13001) 2024-09-10 13:23:13 -07:00
memtable_list_test.cc Fix MultiGet dropping memtable kv checksum corruption (#12842) 2024-08-08 13:34:11 -07:00
merge_context.h Add ContinueCallback to GetMergeOperands() (#12438) 2024-03-15 12:25:49 -07:00
merge_helper.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
merge_helper.h Eliminate some code duplication in MergeHelper (#12121) 2023-12-05 14:07:42 -08:00
merge_helper_test.cc
merge_operator.cc
merge_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
multi_cf_iterator_impl.h Fix heap-use-after-free in MultiCfIteratorImpl (#12784) 2024-06-21 11:56:10 -07:00
multi_cf_iterator_test.cc Fix IteratorsConsistentView tests (#12582) 2024-04-25 14:06:46 -07:00
obsolete_files_test.cc Remove the force mode for EnableFileDeletions API (#12337) 2024-02-13 18:36:25 -08:00
options_file_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
output_validator.cc Removed check_flush_compaction_key_order (#12311) 2024-01-31 16:30:26 -08:00
output_validator.h Removed check_flush_compaction_key_order (#12311) 2024-01-31 16:30:26 -08:00
perf_context_test.cc Add write_memtable_time to perf level kEnableWait (#12394) 2024-02-29 15:08:26 -08:00
periodic_task_scheduler.cc Remove extra semi colon from instagram/ranking/mezql/shots/parser/fast/Token.cpp 2024-03-04 06:32:50 -08:00
periodic_task_scheduler.h
periodic_task_scheduler_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
pinned_iterators_manager.h Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
plain_table_db_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
post_memtable_callback.h
pre_release_callback.h
prefix_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
range_del_aggregator.cc Replace ScopedArenaIterator with ScopedArenaPtr<InternalIterator> (#12470) 2024-03-22 13:40:42 -07:00
range_del_aggregator.h Replace ScopedArenaIterator with ScopedArenaPtr<InternalIterator> (#12470) 2024-03-22 13:40:42 -07:00
range_del_aggregator_bench.cc
range_del_aggregator_test.cc
range_tombstone_fragmenter.cc Add support for range deletion when user timestamps are not persisted (#12254) 2024-01-29 11:37:34 -08:00
range_tombstone_fragmenter.h Fix compile errors in C++23 (#12106) 2024-05-28 15:33:57 -07:00
range_tombstone_fragmenter_test.cc
read_callback.h
repair.cc Make compaction always use the input version with extra ref protection (#12992) 2024-09-06 14:07:33 -07:00
repair_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
seqno_time_test.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
seqno_to_time_mapping.cc Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
seqno_to_time_mapping.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
snapshot_checker.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
snapshot_impl.cc
snapshot_impl.h
table_cache.cc Fix possible double-free on TruncatedRangeDelIterator (#12805) 2024-06-24 11:51:16 -07:00
table_cache.h Fix possible double-free on TruncatedRangeDelIterator (#12805) 2024-06-24 11:51:16 -07:00
table_cache_sync_and_async.h Fix kBlockCacheTier read when merge-chain base value is in a blob file (#12462) 2024-03-21 12:38:53 -07:00
table_properties_collector.cc
table_properties_collector.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
table_properties_collector_test.cc Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
transaction_log_impl.cc Rename, deprecate LogFile and VectorLogPtr (#12695) 2024-05-28 09:24:49 -07:00
transaction_log_impl.h Rename, deprecate LogFile and VectorLogPtr (#12695) 2024-05-28 09:24:49 -07:00
trim_history_scheduler.cc
trim_history_scheduler.h
version_builder.cc Add some documentation for version edit handlers (#12948) 2024-08-21 10:09:10 -07:00
version_builder.h Best efforts recovery recover seqno prefix (#12938) 2024-08-16 17:18:54 -07:00
version_builder_test.cc Fix blob files not reclaimed after deleting all SSTs (#12235) 2024-01-16 11:15:23 -08:00
version_edit.cc Support ingesting SST files generated by a live DB (#12750) 2024-07-19 16:14:54 -07:00
version_edit.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
version_edit_handler.cc Best efforts recovery recover seqno prefix (#12938) 2024-08-16 17:18:54 -07:00
version_edit_handler.h Add some documentation for version edit handlers (#12948) 2024-08-21 10:09:10 -07:00
version_edit_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
version_set.cc Change the semantics of blob_garbage_collection_force_threshold to provide better control over space amp (#13022) 2024-09-19 15:47:13 -07:00
version_set.h Deshim coro in fbcode/internal_repo_rocksdb 2024-09-14 09:48:21 -07:00
version_set_sync_and_async.h
version_set_test.cc Change the semantics of blob_garbage_collection_force_threshold to provide better control over space amp (#13022) 2024-09-19 15:47:13 -07:00
version_util.h Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
wal_edit.cc
wal_edit.h
wal_edit_test.cc
wal_manager.cc Avoid unnecessary work in internal calls to GetSortedWalFiles (#12831) 2024-07-01 23:29:02 -07:00
wal_manager.h Avoid unnecessary work in internal calls to GetSortedWalFiles (#12831) 2024-07-01 23:29:02 -07:00
wal_manager_test.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
write_batch.cc Add a TransactionOptions to enable tracking timestamp size info inside WriteBatch (#12864) 2024-08-05 13:06:45 -07:00
write_batch_base.cc
write_batch_internal.h Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
write_batch_test.cc Replace ScopedArenaIterator with ScopedArenaPtr<InternalIterator> (#12470) 2024-03-22 13:40:42 -07:00
write_callback.h
write_callback_test.cc Add public API WriteWithCallback to support custom callbacks (#12603) 2024-05-31 19:30:19 -07:00
write_controller.cc
write_controller.h
write_controller_test.cc
write_stall_stats.cc
write_stall_stats.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
write_thread.cc Add public API WriteWithCallback to support custom callbacks (#12603) 2024-05-31 19:30:19 -07:00
write_thread.h Add public API WriteWithCallback to support custom callbacks (#12603) 2024-05-31 19:30:19 -07:00