rocksdb/db/db_impl
Yanqin Jin c93ba7db5d Revise LockWAL/UnlockWAL implementation (#11020)
Summary:
RocksDB has two public APIs: `DB::LockWAL()`/`DB::UnlockWAL()`. The current implementation acquires and
releases the internal `DBImpl::log_write_mutex_`.

According to the comment on `DBImpl::log_write_mutex_`: https://github.com/facebook/rocksdb/blob/7.8.fb/db/db_impl/db_impl.h#L2287:L2288
> Note: to avoid dealock, if needed to acquire both log_write_mutex_ and mutex_, the order should be first mutex_ and then log_write_mutex_.

This puts limitations on how applications can use the `LockWAL()` API. After `LockWAL()` returns ok, then application
should not perform any operation that acquires `mutex_`. Currently, the use case of `LockWAL()` is MyRocks implementing
the MySQL storage engine handlerton `lock_hton_log` interface. The operation that MyRocks performs after `LockWAL()`
is `GetSortedWalFiless()` which not only acquires mutex_, but also `log_write_mutex_`.

There are two issues:
1. Applications using these two APIs may hang if one thread calls `GetSortedWalFiles()` after
calling `LockWAL()` because log_write_mutex is not recursive.
2. Two threads may dead lock due to lock order inversion.

To fix these issues, we can modify the implementation of LockWAL so that it does not keep
`log_write_mutex_` held until UnlockWAL. To achieve the goal of locking the WAL, we can
instead manually inject a write stall so that all future writes will be stopped.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11020

Test Plan: make check

Reviewed By: ajkr

Differential Revision: D41785203

Pulled By: riversand963

fbshipit-source-id: 5ccb7a9c6eb9a2c3fa80fd2c399cc2568b8f89ce
2022-12-13 21:45:00 -08:00
..
compacted_db_impl.cc Run clang-format on some files in db/db_impl directory (#10869) 2022-10-25 13:49:09 -07:00
compacted_db_impl.h Run clang-format on some files in db/db_impl directory (#10869) 2022-10-25 13:49:09 -07:00
db_impl.cc Revise LockWAL/UnlockWAL implementation (#11020) 2022-12-13 21:45:00 -08:00
db_impl.h Revise LockWAL/UnlockWAL implementation (#11020) 2022-12-13 21:45:00 -08:00
db_impl_compaction_flush.cc Sort L0 files by newly introduced epoch_num (#10922) 2022-12-13 13:29:37 -08:00
db_impl_debug.cc Add an unittest for Periodic compaction conflict with ongoing compaction (#10908) 2022-12-12 10:37:55 -08:00
db_impl_experimental.cc Sort L0 files by newly introduced epoch_num (#10922) 2022-12-13 13:29:37 -08:00
db_impl_files.cc Fix missing WAL in new manifest by rolling over the WAL deletion record from prev manifest (#10892) 2022-11-29 14:14:43 -08:00
db_impl_open.cc Sort L0 files by newly introduced epoch_num (#10922) 2022-12-13 13:29:37 -08:00
db_impl_readonly.cc Skip swaths of range tombstone covered keys in merging iterator (2022 edition) (#10449) 2022-09-02 09:51:19 -07:00
db_impl_readonly.h Run clang-format on some files in db/db_impl directory (#10869) 2022-10-25 13:49:09 -07:00
db_impl_secondary.cc Run clang-format on some files in db/db_impl directory (#10869) 2022-10-25 13:49:09 -07:00
db_impl_secondary.h Run clang-format on some files in db/db_impl directory (#10869) 2022-10-25 13:49:09 -07:00
db_impl_write.cc Revise LockWAL/UnlockWAL implementation (#11020) 2022-12-13 21:45:00 -08:00