rocksdb/db_stress_tool
Hui Xiao 75a1230ce8 Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933)
Summary:
**Context:**
Adding assertion `!PendingPut()&&!PendingDelete()` in `ExpectedValute::Exists()` surfaced a couple improper usages of `ExpectedValute::Exists()` in the crash test
- Commit phase of `ExpectedValue::Delete()`/`SyncDelete()`:
When we issue delete to expected value during commit phase or `SyncDelete()` (used in crash recovery verification) as below, we don't really care about the result.
d458331ee9/db_stress_tool/expected_state.cc (L73)
d458331ee9/db_stress_tool/expected_value.cc (L52)
That means, we don't really need to check for `Exists()` d458331ee9/db_stress_tool/expected_value.cc (L24-L26).
This actually gives an alternative solution to b65e29a4a9 to solve false-positive assertion violation.
- TestMultiGetXX() path: `Exists()` is called without holding the lock as required

f63428bcc7/db_stress_tool/no_batched_ops_stress.cc (L2688)
```
void MaybeAddKeyToTxnForRYW(
      ThreadState* thread, int column_family, int64_t key, Transaction* txn,
      std::unordered_map<std::string, ExpectedValue>& ryw_expected_values) {
    assert(thread);
    assert(txn);

    SharedState* const shared = thread->shared;
    assert(shared);

    if (!shared->AllowsOverwrite(key) && shared->Exists(column_family, key)) {
      // Just do read your write checks for keys that allow overwrites.
      return;
    }

    // With a 1 in 10 probability, insert the just added key in the batch
    // into the transaction. This will create an overlap with the MultiGet
    // keys and exercise some corner cases in the code
    if (thread->rand.OneIn(10)) {
```

f63428bcc7/db_stress_tool/expected_state.h (L74-L76)

The assertion also failed if db stress compaction filter was invoked before crash recovery verification (`VerifyDB()`->`VerifyOrSyncValue()`) finishes.
f63428bcc7/db_stress_tool/db_stress_compaction_filter.h (L53)
It failed because it can encounter a key with pending state when checking for `Exists()` since that key's expected state has not been sync-ed with db state in `VerifyOrSyncValue()`.
f63428bcc7/db_stress_tool/no_batched_ops_stress.cc (L2579-L2591)

**Summary:**
This PR fixes above issues by
- not checking `Exists()` in commit phase/`SyncDelete()`
- using the concurrent version of key existence check like in other read
- conditionally temporarily disabling compaction till after crash recovery verification succeeds()

And add back the assertion `!PendingPut()&&!PendingDelete()`

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12933

Test Plan: Rehearsal CI

Reviewed By: cbi42

Differential Revision: D61214889

Pulled By: hx235

fbshipit-source-id: ef25ba896e64330ddf330182314981516880c3e4
2024-08-15 12:32:59 -07:00
..
batched_ops_stress.cc Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
cf_consistency_stress.cc Remove unnecessary injected error logging in crash test (#12807) 2024-06-24 20:51:39 -07:00
CMakeLists.txt Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
db_stress.cc Disable tiered storage + BlobDB stress test (#10699) 2022-09-19 15:39:31 -07:00
db_stress_common.cc Fix nullptr access and race to fault_fs_guard (#12799) 2024-06-24 16:10:36 -07:00
db_stress_common.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_compaction_filter.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_driver.cc Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_driver.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
db_stress_env_wrapper.h Deflake ThreadStatus related unit tests (#12858) 2024-07-15 09:56:09 -07:00
db_stress_filters.cc Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
db_stress_filters.h Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
db_stress_gflags.cc Decouple sync fault and write injection in FaultInjectionTestFS & fix tracing issue under WAL write error injection (#12797) 2024-06-26 14:56:35 -07:00
db_stress_listener.cc Fix/improve temperature handling for file ingestion (#12402) 2024-03-05 16:56:08 -08:00
db_stress_listener.h Handle injected write error after successful WAL write in crash test + misc (#12838) 2024-07-29 13:51:49 -07:00
db_stress_shared_state.cc Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
db_stress_shared_state.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_stat.cc Fix Statistics in db_stress (#9260) 2021-12-07 16:24:22 -08:00
db_stress_stat.h Fix Statistics in db_stress (#9260) 2021-12-07 16:24:22 -08:00
db_stress_table_properties_collector.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
db_stress_test_base.cc Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_test_base.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
db_stress_tool.cc Fix race in multiops txn stress test (#12847) 2024-07-09 16:51:38 -07:00
db_stress_wide_merge_operator.cc Add the wide-column aware merge API to the stress tests (#11906) 2023-09-29 08:54:50 -07:00
db_stress_wide_merge_operator.h Add the wide-column aware merge API to the stress tests (#11906) 2023-09-29 08:54:50 -07:00
expected_state.cc Fix false-positive TestBackupRestore corruption (#12917) 2024-08-09 14:51:36 -07:00
expected_state.h Fix false-positive TestBackupRestore corruption (#12917) 2024-08-09 14:51:36 -07:00
expected_value.cc Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
expected_value.h Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00
multi_ops_txns_stress.cc Add experimental range filters to stress/crash test (#12769) 2024-06-18 16:16:09 -07:00
multi_ops_txns_stress.h Disable AttributeGroup in multiops txn test (#12781) 2024-06-18 16:05:18 -07:00
no_batched_ops_stress.cc Fix improper ExpectedValute::Exists() usages and disable compaction during VerifyDB() in crash test (#12933) 2024-08-15 12:32:59 -07:00