rocksdb/options
Sagar Vemuri 04c11b867d Level Compaction with TTL
Summary:
Level Compaction with TTL.

As of today, a file could exist in the LSM tree without going through the compaction process for a really long time if there are no updates to the data in the file's key range. For example, in certain use cases, the keys are not actually "deleted"; instead they are just set to empty values. There might not be any more writes to this "deleted" key range, and if so, such data could remain in the LSM for a really long time resulting in wasted space.

Introducing a TTL could solve this problem. Files (and, in turn, data) older than TTL will be scheduled for compaction when there is no other background work. This will make the data go through the regular compaction process and get rid of old unwanted data.
This also has the (good) side-effect of all the data in the non-bottommost level being newer than ttl, and all data in the bottommost level older than ttl. It could lead to more writes while reducing space.

This functionality can be controlled by the newly introduced column family option -- ttl.

TODO for later:
- Make ttl mutable
- Extend TTL to Universal compaction as well? (TTL is already supported in FIFO)
- Maybe deprecate CompactionOptionsFIFO.ttl in favor of this new ttl option.
Closes https://github.com/facebook/rocksdb/pull/3591

Differential Revision: D7275442

Pulled By: sagar0

fbshipit-source-id: dcba484717341200d419b0953dafcdf9eb2f0267
2018-04-02 22:14:28 -07:00
..
cf_options.cc Level Compaction with TTL 2018-04-02 22:14:28 -07:00
cf_options.h Level Compaction with TTL 2018-04-02 22:14:28 -07:00
db_options.cc Make DBOption compaction_readahead_size dynamic 2017-11-16 17:57:25 -08:00
db_options.h Make DBOption compaction_readahead_size dynamic 2017-11-16 17:57:25 -08:00
options.cc Level Compaction with TTL 2018-04-02 22:14:28 -07:00
options_helper.cc Level Compaction with TTL 2018-04-02 22:14:28 -07:00
options_helper.h DB::DumpSupportInfo should log all supported compression types 2018-01-23 14:44:12 -08:00
options_parser.cc Comment out unused variables 2018-03-05 13:13:41 -08:00
options_parser.h WritePrepared Txn: Optimize for recoverable state 2017-11-01 17:26:46 -07:00
options_sanity_check.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
options_sanity_check.h Change RocksDB License 2017-07-15 16:11:23 -07:00
options_settable_test.cc Level Compaction with TTL 2018-04-02 22:14:28 -07:00
options_test.cc RocksDBOptionsParser::Parse()'s ignore_unknown_options argument only ingores options from higher version. 2018-02-22 13:28:12 -08:00