rocksdb/memtable
Maysam Yabandeh 638d239507 Charge block cache for cache internal usage (#5797)
Summary:
For our default block cache, each additional entry has extra memory overhead. It include LRUHandle (72 bytes currently) and the cache key (two varint64, file id and offset). The usage is not negligible. For example for block_size=4k, the overhead accounts for an extra 2% memory usage for the cache. The patch charging the cache for the extra usage, reducing untracked memory usage outside block cache. The feature is enabled by default and can be disabled by passing kDontChargeCacheMetadata to the cache constructor.
This PR builds up on https://github.com/facebook/rocksdb/issues/4258
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5797

Test Plan:
- Existing tests are updated to either disable the feature when the test has too much dependency on the old way of accounting the usage or increasing the cache capacity to account for the additional charge of metadata.
- The Usage tests in cache_test.cc are augmented to test the cache usage under kFullChargeCacheMetadata.

Differential Revision: D17396833

Pulled By: maysamyabandeh

fbshipit-source-id: 7684ccb9f8a40ca595e4f5efcdb03623afea0c6f
2019-09-16 15:26:21 -07:00
..
alloc_tracker.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
hash_linklist_rep.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
hash_linklist_rep.h Change RocksDB License 2017-07-15 16:11:23 -07:00
hash_skiplist_rep.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
hash_skiplist_rep.h Change RocksDB License 2017-07-15 16:11:23 -07:00
inlineskiplist.h Add insert hints for each writebatch (#5728) 2019-09-12 17:15:18 -07:00
inlineskiplist_test.cc Add insert hints for each writebatch (#5728) 2019-09-12 17:15:18 -07:00
memtablerep_bench.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
skiplist.h Use delete to disable automatic generated methods. (#5009) 2019-09-11 18:09:00 -07:00
skiplist_test.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
skiplistrep.cc Add insert hints for each writebatch (#5728) 2019-09-12 17:15:18 -07:00
stl_wrappers.h Consolidate hash function used for non-persistent data in a new function (#5155) 2019-04-08 13:32:06 -07:00
vectorrep.cc Move some memory related files from util/ to memory/ (#5382) 2019-05-30 17:44:09 -07:00
write_buffer_manager.cc WriteBufferManager's dummy entry size to block cache 1MB -> 256KB (#5175) 2019-04-16 12:03:07 -07:00
write_buffer_manager_test.cc Charge block cache for cache internal usage (#5797) 2019-09-16 15:26:21 -07:00