rocksdb/options
Shobhit Dayal b45b1cde3e Feature for sampling and reporting compressibility (#4842)
Summary:
This is a feature to sample data-block compressibility and and report them as stats. 1 in N (tunable) blocks is sampled for compressibility using two algorithms:
1. lz4 or snappy for fast compression
2. zstd or zlib for slow but higher compression.

The stats are reported to the caller as raw-bytes and compressed-bytes. The block continues to be compressed for storage using the specified CompressionType.

The db_bench_tool how has a command line option for specifying the sampling rate. It's default value is 0 (no sampling). To test the overhead for a certain value, users can compare the performance of db_bench_tool, varying the sampling rate. It is unlikely to have a noticeable impact for high values like 20.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/4842

Differential Revision: D13629011

Pulled By: shobhitdayal

fbshipit-source-id: 14ca668bcab6499b2a1734edf848eb62a4f4fafa
2019-03-18 12:15:34 -07:00
..
cf_options.cc add whole key bloom filter support in memtables (#4985) 2019-02-19 12:15:39 -08:00
cf_options.h Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
db_options.cc add GetStatsHistory to retrieve stats snapshots (#4748) 2019-02-20 15:52:54 -08:00
db_options.h add GetStatsHistory to retrieve stats snapshots (#4748) 2019-02-20 15:52:54 -08:00
options.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
options_helper.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
options_helper.h add OptionType kInt32T and kInt64T 2019-03-12 13:49:52 -07:00
options_parser.cc add OptionType kInt32T and kInt64T 2019-03-12 13:49:52 -07:00
options_parser.h WritePrepared Txn: Optimize for recoverable state 2017-11-01 17:26:46 -07:00
options_sanity_check.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
options_sanity_check.h Change RocksDB License 2017-07-15 16:11:23 -07:00
options_settable_test.cc Feature for sampling and reporting compressibility (#4842) 2019-03-18 12:15:34 -07:00
options_test.cc add OptionType kInt32T and kInt64T 2019-03-12 13:49:52 -07:00