dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-25 22:44:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/cache
History
Peter Dillinger 1aac814578 Use optimized folly DistributedMutex in LRUCache when available (#10179) 
Summary:
folly DistributedMutex is faster than standard mutexes though
imposes some static obligations on usage. See
https://github.com/facebook/folly/blob/main/folly/synchronization/DistributedMutex.h
for details. Here we use this alternative for our Cache implementations
(especially LRUCache) for better locking performance, when RocksDB is
compiled with folly.

Also added information about which distributed mutex implementation is
being used to cache_bench output and to DB LOG.

Intended follow-up:
* Use DMutex in more places, perhaps improving API to support non-scoped
locking
* Fix linking with fbcode compiler (needs ROCKSDB_NO_FBCODE=1 currently)

Credit: Thanks Siying for reminding me about this line of work that was previously
left unfinished.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10179

Test Plan:
for correctness, existing tests. CircleCI config updated.
Also Meta-internal buck build updated.

For performance, ran simultaneous before & after cache_bench. Out of three
comparison runs, the middle improvement to ops/sec was +21%:

Baseline: USE_CLANG=1 DEBUG_LEVEL=0 make -j24 cache_bench (fbcode
compiler)

```
Complete in 20.201 s; Rough parallel ops/sec = 1584062
Thread ops/sec = 107176

Operation latency (ns):
Count: 32000000 Average: 9257.9421  StdDev: 122412.04
Min: 134  Median: 3623.0493  Max: 56918500
Percentiles: P50: 3623.05 P75: 10288.02 P99: 30219.35 P99.9: 683522.04 P99.99: 7302791.63
```

New: (add USE_FOLLY=1)

```
Complete in 16.674 s; Rough parallel ops/sec = 1919135  (+21%)
Thread ops/sec = 135487

Operation latency (ns):
Count: 32000000 Average: 7304.9294  StdDev: 108530.28
Min: 132  Median: 3777.6012  Max: 91030902
Percentiles: P50: 3777.60 P75: 10169.89 P99: 24504.51 P99.9: 59721.59 P99.99: 1861151.83
```

Reviewed By: anand1976

Differential Revision: D37182983

Pulled By: pdillinger

fbshipit-source-id: a17eb05f25b832b6a2c1356f5c657e831a5af8d1
2022-06-17 13:08:45 -07:00
..
cache.cc Enable SecondaryCache::CreateFromString to create sec cache based on the uri for CompressedSecondaryCache (#10132) 2022-06-10 12:23:10 -07:00
cache_bench.cc
cache_bench_tool.cc Use optimized folly DistributedMutex in LRUCache when available (#10179) 2022-06-17 13:08:45 -07:00
cache_entry_roles.cc Account memory of FileMetaData in global memory limit (#9924) 2022-06-14 13:06:40 -07:00
cache_entry_roles.h Expose CacheEntryRole and map keys for block cache stat collections (#9838) 2022-04-14 09:38:55 -07:00
cache_entry_stats.h
cache_helpers.h
cache_key.cc
cache_key.h
cache_reservation_manager.cc Account memory of FileMetaData in global memory limit (#9924) 2022-06-14 13:06:40 -07:00
cache_reservation_manager.h Account memory of FileMetaData in global memory limit (#9924) 2022-06-14 13:06:40 -07:00
cache_reservation_manager_test.cc Meta-internal folly integration with F14FastMap (#9546) 2022-04-13 07:34:01 -07:00
cache_test.cc Make the per-shard hash table fixed-size. (#10154) 2022-06-13 20:29:00 -07:00
clock_cache.cc Use optimized folly DistributedMutex in LRUCache when available (#10179) 2022-06-17 13:08:45 -07:00
clock_cache.h
compressed_secondary_cache.cc
compressed_secondary_cache.h
compressed_secondary_cache_test.cc Enable SecondaryCache::CreateFromString to create sec cache based on the uri for CompressedSecondaryCache (#10132) 2022-06-10 12:23:10 -07:00
fast_lru_cache.cc Use optimized folly DistributedMutex in LRUCache when available (#10179) 2022-06-17 13:08:45 -07:00
fast_lru_cache.h Use optimized folly DistributedMutex in LRUCache when available (#10179) 2022-06-17 13:08:45 -07:00
lru_cache.cc Use optimized folly DistributedMutex in LRUCache when available (#10179) 2022-06-17 13:08:45 -07:00
lru_cache.h Use optimized folly DistributedMutex in LRUCache when available (#10179) 2022-06-17 13:08:45 -07:00
lru_cache_test.cc Make the per-shard hash table fixed-size. (#10154) 2022-06-13 20:29:00 -07:00
sharded_cache.cc
sharded_cache.h