mirror of
https://github.com/facebook/rocksdb.git
synced 2024-11-27 11:43:49 +00:00
dc576af0fd
Summary: Saw this in stress test: ``` db_stress: cache/clock_cache.cc:3152:[...] Assertion `i < 0x2000' failed. ``` The problem is related to Lookups on a chain currently involved in a Grow operation. To avoid Lookup waiting on Grow, Lookup is able to walk a chain whose first part is already migrated and tail is not yet migrated, so is mixed with entries with a different destination home (according to `home_shift`) than what we're looking for. This is fine until we save one of these entries as a safe point in the chain to backtrack to (`read_ref_on_chain`) in case of concurrent modification and end up backtracking to it. In that case, we can get stuck on the wrong destination chain and keep trying to backtrack to an entry that is supposed to be on the correct chain but is not (anymore). For some reason I haven't quite worked out, I believe it's usually able to recover after some 1000+ looop iterations, so reproducibility depends on the threshold at which we consider a Lookup loop to be too many iterations for a plausibly valid Lookup. Detecting and working around this case is relatively simple. We can (and must) keep going on the chain but ensure we don't save it as a safe entry to backtrack to. Pull Request resolved: https://github.com/facebook/rocksdb/pull/11948 Test Plan: The problem could be reproduced in a few minutes with this (debug build): ``` $ while ./cache_bench -cache_type=auto_hyper_clock_cache -histograms=0 -cache_size=80000000 -threads=32 -populate_cache=0 -ops_per_thread=10000 -degenerate_hash_bits=6 -num_shard_bits=0; do :; done ``` At least with a lower threshold on suspiciously high number of iterations. I've lowered the thresholds quite a bit and no longer able to reproduce a failure. Reviewed By: jowlyzhang Differential Revision: D50236574 Pulled By: pdillinger fbshipit-source-id: 2cb54a4e02bb51d5933eea41fcd489ab9d34aa96 |
||
|---|---|---|
| .. | ||
| cache.cc | ||
| cache_bench.cc | ||
| cache_bench_tool.cc | ||
| cache_entry_roles.cc | ||
| cache_entry_roles.h | ||
| cache_entry_stats.h | ||
| cache_helpers.cc | ||
| cache_helpers.h | ||
| cache_key.cc | ||
| cache_key.h | ||
| cache_reservation_manager.cc | ||
| cache_reservation_manager.h | ||
| cache_reservation_manager_test.cc | ||
| cache_test.cc | ||
| charged_cache.cc | ||
| charged_cache.h | ||
| clock_cache.cc | ||
| clock_cache.h | ||
| compressed_secondary_cache.cc | ||
| compressed_secondary_cache.h | ||
| compressed_secondary_cache_test.cc | ||
| lru_cache.cc | ||
| lru_cache.h | ||
| lru_cache_test.cc | ||
| secondary_cache.cc | ||
| secondary_cache_adapter.cc | ||
| secondary_cache_adapter.h | ||
| sharded_cache.cc | ||
| sharded_cache.h | ||
| tiered_secondary_cache.cc | ||
| tiered_secondary_cache.h | ||
| tiered_secondary_cache_test.cc | ||
| typed_cache.h | ||