rocksdb/tools
Andrew Kryczka 9402a77a37 No filesystem reads during Merge() writes (#12365)
Summary:
This occasional filesystem read in the write path has caused user pain. It doesn't seem very useful considering it only limits one component's merge chain length, and only helps merge uncached (i.e., infrequently read) values. This PR proposes allowing `max_successive_merges` to be exceeded when the value cannot be read from in-memory components. I included a rollback flag (`strict_max_successive_merges`) just in case.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12365

Test Plan:
"rocksdb.block.cache.data.add" is number of data blocks read from filesystem. Since the benchmark is write-only, compaction is disabled, and flush doesn't read data blocks, any nonzero value means the user write issued the read.

```
$ for s in false true; do echo -n "strict_max_successive_merges=$s: " && ./db_bench -value_size=64 -write_buffer_size=131072 -writes=128 -num=1 -benchmarks=mergerandom,flush,mergerandom -merge_operator=stringappend -disable_auto_compactions=true -compression_type=none -strict_max_successive_merges=$s -max_successive_merges=100 -statistics=true |& grep 'block.cache.data.add COUNT' ; done
strict_max_successive_merges=false: rocksdb.block.cache.data.add COUNT : 0
strict_max_successive_merges=true: rocksdb.block.cache.data.add COUNT : 1
```

Reviewed By: hx235

Differential Revision: D53982520

Pulled By: ajkr

fbshipit-source-id: e40f761a60bd601f232417ac0058e4a33ee9c0f4
2024-02-21 13:41:44 -08:00
..
advisor Fix lint issues after enable BLACK (#10717) 2022-09-21 13:37:51 -07:00
block_cache_analyzer Block cache analyzer: Calculate miss ratio for each caller (#10823) 2024-01-10 14:02:14 -08:00
dump internal_repo_rocksdb (435146444452818992) (#12115) 2023-12-01 11:15:17 -08:00
analyze_txn_stress_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
auto_sanity_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
backup_db.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
benchmark.sh add exe and script path check (#11621) 2023-07-19 12:05:24 -07:00
benchmark_ci.py Remove NUMA setting for benchmark-linux (#11180) 2023-02-02 15:15:09 -08:00
benchmark_compare.sh Fix file modes (#10815) 2022-10-13 09:00:37 -07:00
benchmark_leveldb.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
blob_dump.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
check_all_python.py Enable BLACK for internal_repo_rocksdb (#10710) 2022-09-20 17:47:52 -07:00
check_format_compatible.sh Update HISTORY/version/format compatibility script for the 8.10 release (#12154) 2023-12-15 14:44:23 -08:00
CMakeLists.txt Mark dependencies as PRIVATE and fix missing dependencies in tools. (#6790) 2020-05-12 21:07:55 -07:00
db_bench.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
db_bench_tool.cc No filesystem reads during Merge() writes (#12365) 2024-02-21 13:41:44 -08:00
db_bench_tool_test.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
db_crashtest.py Clean up after long-running whitebox crashtest (#12248) 2024-01-19 16:25:39 -08:00
db_repl_stress.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_sanity_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
dbench_monitor Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
Dockerfile adding docker build script and dockerfile 2015-05-22 16:03:39 -07:00
generate_random_db.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
ingest_external_sst.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
io_tracer_parser.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
io_tracer_parser_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
io_tracer_parser_tool.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
io_tracer_parser_tool.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
ldb.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
ldb_cmd.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
ldb_cmd_impl.h GetEntity and PutEntity Support in ldb (#11796) 2023-09-12 16:32:40 -07:00
ldb_cmd_test.cc Lightweight verification of MANIFEST file after close on shutdown (#12174) 2023-12-28 18:25:29 -08:00
ldb_test.py GetEntity and PutEntity Support in ldb (#11796) 2023-09-12 16:32:40 -07:00
ldb_tool.cc internal_repo_rocksdb (435146444452818992) (#12115) 2023-12-01 11:15:17 -08:00
pflag Fix /bin/bash shebangs 2017-08-03 15:56:46 -07:00
reduce_levels_test.cc Make option level_compaction_dynamic_level_bytes true by default (#11525) 2023-06-15 21:12:39 -07:00
regression_test.sh Fix regression script for async_io benchmarks (#11462) 2023-05-22 15:32:12 -07:00
restore_db.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
rocksdb_dump_test.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
run_blob_bench.sh add exe and script path check (#11621) 2023-07-19 12:05:24 -07:00
run_flash_bench.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
run_leveldb.sh Add copyright headers per FB open-source checkup tool. (#5199) 2019-04-18 10:55:01 -07:00
sample-dump.dmp First version of rocksdb_dump and rocksdb_undump. 2015-06-19 16:24:36 -07:00
simulated_hybrid_file_system.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
simulated_hybrid_file_system.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
sst_dump.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
sst_dump_test.cc Fix and defend against FilePrefetchBuffer combined with mmap reads (#12206) 2024-01-04 18:39:05 -08:00
sst_dump_tool.cc internal_repo_rocksdb (435146444452818992) (#12115) 2023-12-01 11:15:17 -08:00
trace_analyzer.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
trace_analyzer_test.cc internal_repo_rocksdb (435146444452818992) (#12115) 2023-12-01 11:15:17 -08:00
trace_analyzer_tool.cc Trace analyzer: replace number with enumeration type (#10827) 2023-12-27 10:38:53 -08:00
trace_analyzer_tool.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
verify_random_db.sh Fix some bugs in verify_random_db.sh (#10112) 2022-06-03 16:35:13 -07:00
write_external_sst.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
write_stress.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
write_stress_runner.py Enable BLACK for internal_repo_rocksdb (#10710) 2022-09-20 17:47:52 -07:00