mirror of https://github.com/facebook/rocksdb.git
e6432dfd4c
Summary: Currently, if blob files are enabled (i.e. `enable_blob_files` is true), large values are extracted both during flush/recovery (when SST files are written into level 0 of the LSM tree) and during compaction into any LSM tree level. For certain use cases that have a mix of short-lived and long-lived values, it might make sense to support extracting large values only during compactions whose output level is greater than or equal to a specified LSM tree level (e.g. compactions into L1/L2/... or above). This could reduce the space amplification caused by large values that are turned into garbage shortly after being written at the price of some write amplification incurred by long-lived values whose extraction to blob files is delayed. In order to achieve this, we would like to do the following: - Add a new configuration option `blob_file_starting_level` (default: 0) to `AdvancedColumnFamilyOptions` (and `MutableCFOptions` and extend the related logic) - Instantiate `BlobFileBuilder` in `BuildTable` (used during flush and recovery, where the LSM tree level is L0) and `CompactionJob` iff `enable_blob_files` is set and the LSM tree level is `>= blob_file_starting_level` - Add unit tests for the new functionality, and add the new option to our stress tests (`db_stress` and `db_crashtest.py` ) - Add the new option to our benchmarking tool `db_bench` and the BlobDB benchmark script `run_blob_bench.sh` - Add the new option to the `ldb` tool (see https://github.com/facebook/rocksdb/wiki/Administration-and-Data-Access-Tool) - Ideally extend the C and Java bindings with the new option - Update the BlobDB wiki to document the new option. Pull Request resolved: https://github.com/facebook/rocksdb/pull/10077 Reviewed By: ltamasi Differential Revision: D36884156 Pulled By: gangliao fbshipit-source-id: 942bab025f04633edca8564ed64791cb5e31627d |
||
|---|---|---|
| .. | ||
| CMakeLists.txt | ||
| batched_ops_stress.cc | ||
| cf_consistency_stress.cc | ||
| db_stress.cc | ||
| db_stress_common.cc | ||
| db_stress_common.h | ||
| db_stress_compaction_filter.h | ||
| db_stress_driver.cc | ||
| db_stress_driver.h | ||
| db_stress_env_wrapper.h | ||
| db_stress_gflags.cc | ||
| db_stress_listener.cc | ||
| db_stress_listener.h | ||
| db_stress_shared_state.cc | ||
| db_stress_shared_state.h | ||
| db_stress_stat.cc | ||
| db_stress_stat.h | ||
| db_stress_table_properties_collector.h | ||
| db_stress_test_base.cc | ||
| db_stress_test_base.h | ||
| db_stress_tool.cc | ||
| expected_state.cc | ||
| expected_state.h | ||
| multi_ops_txns_stress.cc | ||
| multi_ops_txns_stress.h | ||
| no_batched_ops_stress.cc | ||