rocksdb/db_stress_tool
Levi Tamasi 54ace7f340 Change the semantics of blob_garbage_collection_force_threshold to provide better control over space amp (#13022)
Summary:
Pull Request resolved: https://github.com/facebook/rocksdb/pull/13022

Currently, `blob_garbage_collection_force_threshold` applies to the oldest batch of blob files, which is typically only a small subset of the blob files currently eligible for garbage collection. This can result in a form of head-of-line blocking: no GC-triggered compactions will be scheduled if the oldest batch does not currently exceed the threshold, even if a lot of higher-numbered blob files do. This can in turn lead to high space amplification that exceeds the soft bound implicit in the force threshold (e.g. 50% would suggest a space amp of <2 and 75% would imply a space amp of <4). The patch changes the semantics of this configuration threshold to apply to the entire set of blob files that are eligible for garbage collection based on `blob_garbage_collection_age_cutoff`. This provides more intuitive semantics for the option and can provide a better write amp/space amp trade-off. (Note that GC-triggered compactions still pick the same SST files as before, so triggered GC still targets the oldest the blob files.)

Reviewed By: jowlyzhang

Differential Revision: D62977860

fbshipit-source-id: a999f31fe9cdda313de513f0e7a6fc707424d4a3
2024-09-19 15:47:13 -07:00
..
batched_ops_stress.cc Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
cf_consistency_stress.cc Remove unnecessary injected error logging in crash test (#12807) 2024-06-24 20:51:39 -07:00
CMakeLists.txt Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
db_stress.cc Disable tiered storage + BlobDB stress test (#10699) 2022-09-19 15:39:31 -07:00
db_stress_common.cc Fix nullptr access and race to fault_fs_guard (#12799) 2024-06-24 16:10:36 -07:00
db_stress_common.h Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
db_stress_compaction_filter.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_driver.cc Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_driver.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
db_stress_env_wrapper.h Deflake ThreadStatus related unit tests (#12858) 2024-07-15 09:56:09 -07:00
db_stress_filters.cc Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
db_stress_filters.h Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
db_stress_gflags.cc Change the semantics of blob_garbage_collection_force_threshold to provide better control over space amp (#13022) 2024-09-19 15:47:13 -07:00
db_stress_listener.cc Fix/improve temperature handling for file ingestion (#12402) 2024-03-05 16:56:08 -08:00
db_stress_listener.h Handle injected write error after successful WAL write in crash test + misc (#12838) 2024-07-29 13:51:49 -07:00
db_stress_shared_state.cc Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
db_stress_shared_state.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_stat.cc Fix Statistics in db_stress (#9260) 2021-12-07 16:24:22 -08:00
db_stress_stat.h Fix Statistics in db_stress (#9260) 2021-12-07 16:24:22 -08:00
db_stress_table_properties_collector.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
db_stress_test_base.cc Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
db_stress_test_base.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_tool.cc Fix race in multiops txn stress test (#12847) 2024-07-09 16:51:38 -07:00
db_stress_wide_merge_operator.cc Add the wide-column aware merge API to the stress tests (#11906) 2023-09-29 08:54:50 -07:00
db_stress_wide_merge_operator.h Add the wide-column aware merge API to the stress tests (#11906) 2023-09-29 08:54:50 -07:00
expected_state.cc Retain previous trace file in db_stress for debugging purposes (#12978) 2024-08-27 12:43:47 -07:00
expected_state.h Fix false-positive TestBackupRestore corruption (#12917) 2024-08-09 14:51:36 -07:00
expected_value.cc Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
expected_value.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
multi_ops_txns_stress.cc Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
multi_ops_txns_stress.h Disable AttributeGroup in multiops txn test (#12781) 2024-06-18 16:05:18 -07:00
no_batched_ops_stress.cc Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00