rocksdb/tools
Changyu Bi bc04ec85db Make option level_compaction_dynamic_level_bytes true by default (#11525)
Summary:
after https://github.com/facebook/rocksdb/issues/11321 and https://github.com/facebook/rocksdb/issues/11340 (both included in RocksDB v8.2), migration from `level_compaction_dynamic_level_bytes=false` to `level_compaction_dynamic_level_bytes=true` is automatic by RocksDB and requires no manual compaction from user. Making the option true by default as it has several advantages: 1. better space amplification guarantee (a more stable LSM shape). 2. compaction is more adaptive to write traffic. 3. automatic draining of unneeded levels. Wiki is updated with more detail: https://github.com/facebook/rocksdb/wiki/Leveled-Compaction#option-level_compaction_dynamic_level_bytes-and-levels-target-size.

The PR mostly contains fixes for unit tests as they assumed `level_compaction_dynamic_level_bytes=false`. Most notable change is commit f742be330c and b1928e42b3 which override the default option in DBTestBase to still set `level_compaction_dynamic_level_bytes=false` by default. This helps to reduce the change needed for unit tests. I think this default option override in unit tests is okay since the behavior of `level_compaction_dynamic_level_bytes=true` is tested by explicitly setting this option. Also, `level_compaction_dynamic_level_bytes=false` may be more desired in unit tests as it makes it easier to create a desired LSM shape.

Comment for option `level_compaction_dynamic_level_bytes` is updated to reflect this change and change made in https://github.com/facebook/rocksdb/issues/10057.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11525

Test Plan: `make -j32 J=32 check` several times to try to catch flaky tests due to this option change.

Reviewed By: ajkr

Differential Revision: D46654256

Pulled By: cbi42

fbshipit-source-id: 6b5827dae124f6f1fdc8cca2ac6f6fcd878830e1
2023-06-15 21:12:39 -07:00
..
advisor Fix lint issues after enable BLACK (#10717) 2022-09-21 13:37:51 -07:00
block_cache_analyzer Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
dump Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
analyze_txn_stress_test.sh
auto_sanity_test.sh
backup_db.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
benchmark.sh Fix file modes (#10815) 2022-10-13 09:00:37 -07:00
benchmark_ci.py Remove NUMA setting for benchmark-linux (#11180) 2023-02-02 15:15:09 -08:00
benchmark_compare.sh Fix file modes (#10815) 2022-10-13 09:00:37 -07:00
benchmark_leveldb.sh
blob_dump.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
check_all_python.py Enable BLACK for internal_repo_rocksdb (#10710) 2022-09-20 17:47:52 -07:00
check_format_compatible.sh Update HISTORY.md/version.h/format compatiblity test for 8.3 release (#11464) 2023-05-19 18:42:04 -07:00
CMakeLists.txt
db_bench.cc Add (& fix) some simple source code checks (#8821) 2021-09-07 21:19:27 -07:00
db_bench_tool.cc Integrate CacheReservationManager with compressed secondary cache (#11449) 2023-05-30 14:05:48 -07:00
db_bench_tool_test.cc Remove deprecated util functions in options_util.h (#11126) 2023-01-27 11:10:53 -08:00
db_crashtest.py Update db_crashtest.py for support for dir creation on remote storage (#11448) 2023-05-23 09:42:35 -07:00
db_repl_stress.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_sanity_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
dbench_monitor
Dockerfile
generate_random_db.sh
ingest_external_sst.sh
io_tracer_parser.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
io_tracer_parser_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
io_tracer_parser_tool.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
io_tracer_parser_tool.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
ldb.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
ldb_cmd.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
ldb_cmd_impl.h Run clang format against files under tools/ and db_stress_tool/ (#10868) 2022-10-25 14:29:41 -07:00
ldb_cmd_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
ldb_test.py Enable BLACK for internal_repo_rocksdb (#10710) 2022-09-20 17:47:52 -07:00
ldb_tool.cc Support single delete help message in ldb (#11493) 2023-05-31 14:24:54 -07:00
pflag
reduce_levels_test.cc Make option level_compaction_dynamic_level_bytes true by default (#11525) 2023-06-15 21:12:39 -07:00
regression_test.sh Fix regression script for async_io benchmarks (#11462) 2023-05-22 15:32:12 -07:00
restore_db.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
rocksdb_dump_test.sh
run_blob_bench.sh Support prepopulating/warming the blob cache (#10298) 2022-07-17 07:13:59 -07:00
run_flash_bench.sh
run_leveldb.sh
sample-dump.dmp
simulated_hybrid_file_system.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
simulated_hybrid_file_system.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
sst_dump.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
sst_dump_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
sst_dump_tool.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
trace_analyzer.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
trace_analyzer_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
trace_analyzer_tool.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
trace_analyzer_tool.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
verify_random_db.sh Fix some bugs in verify_random_db.sh (#10112) 2022-06-03 16:35:13 -07:00
write_external_sst.sh Revamp check_format_compatible.sh (#8012) 2021-03-02 11:42:27 -08:00
write_stress.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
write_stress_runner.py Enable BLACK for internal_repo_rocksdb (#10710) 2022-09-20 17:47:52 -07:00