mirror of
https://github.com/facebook/rocksdb.git
synced 2024-12-02 10:15:54 +00:00
d490bfcdb6
Summary:
**Summary:**
When a block is firstly `Lookup` from the secondary cache, we just insert a dummy block in the primary cache (charging the actual size of the block) and don’t erase the block from the secondary cache. A standalone handle is returned from `Lookup`. Only if the block is hit again, we erase it from the secondary cache and add it into the primary cache.
When a block is firstly evicted from the primary cache to the secondary cache, we just insert a dummy block (size 0) in the secondary cache. When the block is evicted again, it is treated as a hot block and is inserted into the secondary cache.
**Implementation Details**
Add a new state of LRUHandle: The handle is never inserted into the LRUCache (both hash table and LRU list) and it doesn't experience the above three states. The entry can be freed when refs becomes 0. (refs >= 1 && in_cache == false && IS_STANDALONE == true)
The behaviors of `LRUCacheShard::Lookup()` are updated if the secondary_cache is CompressedSecondaryCache:
1. If a handle is found in primary cache:
1.1. If the handle's value is not nullptr, it is returned immediately.
1.2. If the handle's value is nullptr, this means the handle is a dummy one. For a dummy handle, if it was retrieved from secondary cache, it may still exist in secondary cache.
- 1.2.1. If no valid handle can be `Lookup` from secondary cache, return nullptr.
- 1.2.2. If the handle from secondary cache is valid, erase it from the secondary cache and add it into the primary cache.
2. If a handle is not found in primary cache:
2.1. If no valid handle can be `Lookup` from secondary cache, return nullptr.
2.2. If the handle from secondary cache is valid, insert a dummy block in the primary cache (charging the actual size of the block) and return a standalone handle.
The behaviors of `LRUCacheShard::Promote()` are updated as follows:
1. If `e->sec_handle` has value, one of the following steps can happen:
1.1. Insert a dummy handle and return a standalone handle to caller when `secondary_cache_` is `CompressedSecondaryCache` and e is a standalone handle.
1.2. Insert the item into the primary cache and return the handle to caller.
1.3. Exception handling.
3. If `e->sec_handle` has no value, mark the item as not in cache and charge the cache as its only metadata that'll shortly be released.
The behavior of `CompressedSecondaryCache::Insert()` is updated:
1. If a block is evicted from the primary cache for the first time, a dummy item is inserted.
4. If a dummy item is found for a block, the block is inserted into the secondary cache.
The behavior of `CompressedSecondaryCache:::Lookup()` is updated:
1. If a handle is not found or it is a dummy item, a nullptr is returned.
2. If `erase_handle` is true, the handle is erased.
The behaviors of `LRUCacheShard::Release()` are adjusted for the standalone handles.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10527
Test Plan:
1. stress tests.
5. unit tests.
6. CPU profiling for db_bench.
Reviewed By: siying
Differential Revision: D38747613
Pulled By: gitbw95
fbshipit-source-id: 74a1eba7e1957c9affb2bd2ae3e0194584fa6eca
101 lines
3 KiB
C++
101 lines
3 KiB
C++
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
#include "rocksdb/secondary_cache.h"
|
|
#include "util/random.h"
|
|
#include "util/thread_local.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
// This class implements a custom SecondaryCache that randomly injects an
|
|
// error status into Inserts/Lookups based on a specified probability.
|
|
// Its used by db_stress to verify correctness in the presence of
|
|
// secondary cache errors.
|
|
//
|
|
class FaultInjectionSecondaryCache : public SecondaryCache {
|
|
public:
|
|
explicit FaultInjectionSecondaryCache(
|
|
const std::shared_ptr<SecondaryCache>& base, uint32_t seed, int prob)
|
|
: base_(base),
|
|
seed_(seed),
|
|
prob_(prob),
|
|
thread_local_error_(new ThreadLocalPtr(DeleteThreadLocalErrorContext)) {
|
|
if (std::strcmp(base_->Name(), "CompressedSecondaryCache") == 0) {
|
|
base_is_compressed_sec_cache_ = true;
|
|
}
|
|
}
|
|
|
|
virtual ~FaultInjectionSecondaryCache() override {}
|
|
|
|
const char* Name() const override { return "FaultInjectionSecondaryCache"; }
|
|
|
|
Status Insert(const Slice& key, void* value,
|
|
const Cache::CacheItemHelper* helper) override;
|
|
|
|
std::unique_ptr<SecondaryCacheResultHandle> Lookup(
|
|
const Slice& key, const Cache::CreateCallback& create_cb, bool wait,
|
|
bool advise_erase, bool& is_in_sec_cache) override;
|
|
|
|
bool SupportForceErase() const override { return base_->SupportForceErase(); }
|
|
|
|
void Erase(const Slice& key) override;
|
|
|
|
void WaitAll(std::vector<SecondaryCacheResultHandle*> handles) override;
|
|
|
|
std::string GetPrintableOptions() const override {
|
|
return base_->GetPrintableOptions();
|
|
}
|
|
|
|
private:
|
|
class ResultHandle : public SecondaryCacheResultHandle {
|
|
public:
|
|
ResultHandle(FaultInjectionSecondaryCache* cache,
|
|
std::unique_ptr<SecondaryCacheResultHandle>&& base)
|
|
: cache_(cache), base_(std::move(base)), value_(nullptr), size_(0) {}
|
|
|
|
~ResultHandle() override {}
|
|
|
|
bool IsReady() override;
|
|
|
|
void Wait() override;
|
|
|
|
void* Value() override;
|
|
|
|
size_t Size() override;
|
|
|
|
static void WaitAll(FaultInjectionSecondaryCache* cache,
|
|
std::vector<SecondaryCacheResultHandle*> handles);
|
|
|
|
private:
|
|
static void UpdateHandleValue(ResultHandle* handle);
|
|
|
|
FaultInjectionSecondaryCache* cache_;
|
|
std::unique_ptr<SecondaryCacheResultHandle> base_;
|
|
void* value_;
|
|
size_t size_;
|
|
};
|
|
|
|
static void DeleteThreadLocalErrorContext(void* p) {
|
|
ErrorContext* ctx = static_cast<ErrorContext*>(p);
|
|
delete ctx;
|
|
}
|
|
|
|
const std::shared_ptr<SecondaryCache> base_;
|
|
uint32_t seed_;
|
|
int prob_;
|
|
bool base_is_compressed_sec_cache_{false};
|
|
|
|
struct ErrorContext {
|
|
Random rand;
|
|
|
|
explicit ErrorContext(uint32_t seed) : rand(seed) {}
|
|
};
|
|
std::unique_ptr<ThreadLocalPtr> thread_local_error_;
|
|
|
|
ErrorContext* GetErrorContext();
|
|
};
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|