New stable, fixed-length cache keys (#9126)
Summary:
This change standardizes on a new 16-byte cache key format for
block cache (incl compressed and secondary) and persistent cache (but
not table cache and row cache).
The goal is a really fast cache key with practically ideal stability and
uniqueness properties without external dependencies (e.g. from FileSystem).
A fixed key size of 16 bytes should enable future optimizations to the
concurrent hash table for block cache, which is a heavy CPU user /
bottleneck, but there appears to be measurable performance improvement
even with no changes to LRUCache.
This change replaces a lot of disjointed and ugly code handling cache
keys with calls to a simple, clean new internal API (cache_key.h).
(Preserving the old cache key logic under an option would be very ugly
and likely negate the performance gain of the new approach. Complete
replacement carries some inherent risk, but I think that's acceptable
with sufficient analysis and testing.)
The scheme for encoding new cache keys is complicated but explained
in cache_key.cc.
Also: EndianSwapValue is moved to math.h to be next to other bit
operations. (Explains some new include "math.h".) ReverseBits operation
added and unit tests added to hash_test for both.
Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126
Test Plan:
### Basic correctness
Several tests needed updates to work with the new functionality, mostly
because we are no longer relying on filesystem for stable cache keys
so table builders & readers need more context info to agree on cache
keys. This functionality is so core, a huge number of existing tests
exercise the cache key functionality.
### Performance
Create db with
`TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters`
And test performance with
`TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4`
using DEBUG_LEVEL=0 and simultaneous before & after runs.
Before ops/sec, avg over 100 runs: 121924
After ops/sec, avg over 100 runs: 125385 (+2.8%)
### Collision probability
I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity
over many months, by making some pessimistic simplifying assumptions:
* Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys)
* All of every file is cached for its entire lifetime
We use a simple table with skewed address assignment and replacement on address collision
to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output
with `./cache_bench -stress_cache_key -sck_keep_bits=40`:
```
Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day
Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached)
```
These come from default settings of 2.5M files per day of 32 MB each, and
`-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of
the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation
is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality.
More default assumptions, relatively pessimistic:
* 100 DBs in same process (doesn't matter much)
* Re-open DB in same process (new session ID related to old session ID) on average
every 100 files generated
* Restart process (all new session IDs unrelated to old) 24 times per day
After enough data, we get a result at the end:
```
(keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected)
```
If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data:
```
(keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected)
(keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected)
```
The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases:
```
197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected)
```
I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data.
Reviewed By: zhichao-cao
Differential Revision: D33171746
Pulled By: pdillinger
fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
2021-12-17 01:13:55 +00:00
|
|
|
// Copyright (c) Facebook, Inc. and its affiliates. All Rights Reserved.
|
|
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
#include <cstdint>
|
|
|
|
|
|
|
|
#include "rocksdb/rocksdb_namespace.h"
|
|
|
|
#include "rocksdb/slice.h"
|
|
|
|
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
|
|
|
|
class Cache;
|
|
|
|
|
|
|
|
// A standard holder for fixed-size block cache keys (and for related caches).
|
|
|
|
// They are created through one of these, each using its own range of values:
|
|
|
|
// * CacheKey::CreateUniqueForCacheLifetime
|
|
|
|
// * CacheKey::CreateUniqueForProcessLifetime
|
|
|
|
// * Default ctor ("empty" cache key)
|
|
|
|
// * OffsetableCacheKey->WithOffset
|
|
|
|
//
|
|
|
|
// The first two use atomic counters to guarantee uniqueness over the given
|
|
|
|
// lifetime and the last uses a form of universally unique identifier for
|
|
|
|
// uniqueness with very high probabilty (and guaranteed for files generated
|
|
|
|
// during a single process lifetime).
|
|
|
|
//
|
|
|
|
// CacheKeys are currently used by calling AsSlice() to pass as a key to
|
|
|
|
// Cache. For performance, the keys are endianness-dependent (though otherwise
|
|
|
|
// portable). (Persistable cache entries are not intended to cross platforms.)
|
|
|
|
class CacheKey {
|
|
|
|
public:
|
|
|
|
// For convenience, constructs an "empty" cache key that is never returned
|
|
|
|
// by other means.
|
|
|
|
inline CacheKey() : session_etc64_(), offset_etc64_() {}
|
|
|
|
|
|
|
|
inline bool IsEmpty() const {
|
|
|
|
return (session_etc64_ == 0) & (offset_etc64_ == 0);
|
|
|
|
}
|
|
|
|
|
|
|
|
// Use this cache key as a Slice (byte order is endianness-dependent)
|
|
|
|
inline Slice AsSlice() const {
|
|
|
|
static_assert(sizeof(*this) == 16, "Standardized on 16-byte cache key");
|
|
|
|
assert(!IsEmpty());
|
|
|
|
return Slice(reinterpret_cast<const char *>(this), sizeof(*this));
|
|
|
|
}
|
|
|
|
|
|
|
|
// Create a CacheKey that is unique among others associated with this Cache
|
|
|
|
// instance. Depends on Cache::NewId. This is useful for block cache
|
|
|
|
// "reservations".
|
|
|
|
static CacheKey CreateUniqueForCacheLifetime(Cache *cache);
|
|
|
|
|
|
|
|
// Create a CacheKey that is unique among others for the lifetime of this
|
|
|
|
// process. This is useful for saving in a static data member so that
|
|
|
|
// different DB instances can agree on a cache key for shared entities,
|
|
|
|
// such as for CacheEntryStatsCollector.
|
|
|
|
static CacheKey CreateUniqueForProcessLifetime();
|
|
|
|
|
|
|
|
protected:
|
|
|
|
friend class OffsetableCacheKey;
|
|
|
|
CacheKey(uint64_t session_etc64, uint64_t offset_etc64)
|
|
|
|
: session_etc64_(session_etc64), offset_etc64_(offset_etc64) {}
|
|
|
|
uint64_t session_etc64_;
|
|
|
|
uint64_t offset_etc64_;
|
|
|
|
};
|
|
|
|
|
2022-06-23 18:26:50 +00:00
|
|
|
constexpr uint8_t kCacheKeySize = static_cast<uint8_t>(sizeof(CacheKey));
|
|
|
|
|
New stable, fixed-length cache keys (#9126)
Summary:
This change standardizes on a new 16-byte cache key format for
block cache (incl compressed and secondary) and persistent cache (but
not table cache and row cache).
The goal is a really fast cache key with practically ideal stability and
uniqueness properties without external dependencies (e.g. from FileSystem).
A fixed key size of 16 bytes should enable future optimizations to the
concurrent hash table for block cache, which is a heavy CPU user /
bottleneck, but there appears to be measurable performance improvement
even with no changes to LRUCache.
This change replaces a lot of disjointed and ugly code handling cache
keys with calls to a simple, clean new internal API (cache_key.h).
(Preserving the old cache key logic under an option would be very ugly
and likely negate the performance gain of the new approach. Complete
replacement carries some inherent risk, but I think that's acceptable
with sufficient analysis and testing.)
The scheme for encoding new cache keys is complicated but explained
in cache_key.cc.
Also: EndianSwapValue is moved to math.h to be next to other bit
operations. (Explains some new include "math.h".) ReverseBits operation
added and unit tests added to hash_test for both.
Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126
Test Plan:
### Basic correctness
Several tests needed updates to work with the new functionality, mostly
because we are no longer relying on filesystem for stable cache keys
so table builders & readers need more context info to agree on cache
keys. This functionality is so core, a huge number of existing tests
exercise the cache key functionality.
### Performance
Create db with
`TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters`
And test performance with
`TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4`
using DEBUG_LEVEL=0 and simultaneous before & after runs.
Before ops/sec, avg over 100 runs: 121924
After ops/sec, avg over 100 runs: 125385 (+2.8%)
### Collision probability
I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity
over many months, by making some pessimistic simplifying assumptions:
* Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys)
* All of every file is cached for its entire lifetime
We use a simple table with skewed address assignment and replacement on address collision
to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output
with `./cache_bench -stress_cache_key -sck_keep_bits=40`:
```
Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day
Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached)
```
These come from default settings of 2.5M files per day of 32 MB each, and
`-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of
the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation
is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality.
More default assumptions, relatively pessimistic:
* 100 DBs in same process (doesn't matter much)
* Re-open DB in same process (new session ID related to old session ID) on average
every 100 files generated
* Restart process (all new session IDs unrelated to old) 24 times per day
After enough data, we get a result at the end:
```
(keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected)
```
If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data:
```
(keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected)
(keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected)
```
The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases:
```
197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected)
```
I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data.
Reviewed By: zhichao-cao
Differential Revision: D33171746
Pulled By: pdillinger
fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
2021-12-17 01:13:55 +00:00
|
|
|
// A file-specific generator of cache keys, sometimes referred to as the
|
|
|
|
// "base" cache key for a file because all the cache keys for various offsets
|
|
|
|
// within the file are computed using simple arithmetic. The basis for the
|
|
|
|
// general approach is dicussed here: https://github.com/pdillinger/unique_id
|
|
|
|
// Heavily related to GetUniqueIdFromTableProperties.
|
|
|
|
//
|
|
|
|
// If the db_id, db_session_id, and file_number come from the file's table
|
|
|
|
// properties, then the keys will be stable across DB::Open/Close, backup/
|
|
|
|
// restore, import/export, etc.
|
|
|
|
//
|
|
|
|
// This class "is a" CacheKey only privately so that it is not misused as
|
|
|
|
// a ready-to-use CacheKey.
|
|
|
|
class OffsetableCacheKey : private CacheKey {
|
|
|
|
public:
|
|
|
|
// For convenience, constructs an "empty" cache key that should not be used.
|
|
|
|
inline OffsetableCacheKey() : CacheKey() {}
|
|
|
|
|
|
|
|
// Constructs an OffsetableCacheKey with the given information about a file.
|
|
|
|
// max_offset is based on file size (see WithOffset) and is required here to
|
|
|
|
// choose an appropriate (sub-)encoding. This constructor never generates an
|
|
|
|
// "empty" base key.
|
|
|
|
OffsetableCacheKey(const std::string &db_id, const std::string &db_session_id,
|
|
|
|
uint64_t file_number, uint64_t max_offset);
|
|
|
|
|
|
|
|
inline bool IsEmpty() const {
|
|
|
|
bool result = session_etc64_ == 0;
|
|
|
|
assert(!(offset_etc64_ > 0 && result));
|
|
|
|
return result;
|
|
|
|
}
|
|
|
|
|
|
|
|
// Construct a CacheKey for an offset within a file, which must be
|
|
|
|
// <= max_offset provided in constructor. An offset is not necessarily a
|
|
|
|
// byte offset if a smaller unique identifier of keyable offsets is used.
|
|
|
|
//
|
|
|
|
// This class was designed to make this hot code extremely fast.
|
|
|
|
inline CacheKey WithOffset(uint64_t offset) const {
|
|
|
|
assert(!IsEmpty());
|
|
|
|
assert(offset <= max_offset_);
|
|
|
|
return CacheKey(session_etc64_, offset_etc64_ ^ offset);
|
|
|
|
}
|
|
|
|
|
|
|
|
// The "common prefix" is a shared prefix for all the returned CacheKeys,
|
|
|
|
// that also happens to usually be the same among many files in the same DB,
|
|
|
|
// so is efficient and highly accurate (not perfectly) for DB-specific cache
|
|
|
|
// dump selection (but not file-specific).
|
|
|
|
static constexpr size_t kCommonPrefixSize = 8;
|
|
|
|
inline Slice CommonPrefixSlice() const {
|
|
|
|
static_assert(sizeof(session_etc64_) == kCommonPrefixSize,
|
|
|
|
"8 byte common prefix expected");
|
|
|
|
assert(!IsEmpty());
|
|
|
|
assert(&this->session_etc64_ == static_cast<const void *>(this));
|
|
|
|
|
|
|
|
return Slice(reinterpret_cast<const char *>(this), kCommonPrefixSize);
|
|
|
|
}
|
|
|
|
|
|
|
|
// For any max_offset <= this value, the same encoding scheme is guaranteed.
|
|
|
|
static constexpr uint64_t kMaxOffsetStandardEncoding = 0xffffffffffU;
|
|
|
|
|
|
|
|
private:
|
|
|
|
#ifndef NDEBUG
|
|
|
|
uint64_t max_offset_ = 0;
|
|
|
|
#endif
|
|
|
|
};
|
|
|
|
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|