rocksdb/db/db_impl
anand76 e36b0a2da4 Fix corruption bug when recycle_log_file_num changed from 0 (#12591)
Summary:
When `recycle_log_file_num` is changed from 0 to non-zero and the DB is reopened, any log files from the previous session that are still alive get reused. However, the WAL records in those files are not in the recyclable format. If one of those files is reused and is empty, a subsequent re-open, in `RecoverLogFiles`, can replay those records and insert stale data into the memtable. Another manifestation of this is an assertion failure `first_seqno_ == 0 || s >= first_seqno_` in `rocksdb::MemTable::Add`.

We could fix this by either 1) Writing a special record when reusing a log file, or 2) Implement more rigorous checking in `RecoverLogFiles` to ensure we don't replay stale records, or 3) Not reuse files created by a previous DB session. We choose option 3 as its the simplest, and flipping `recycle_log_file_num` is expected to be a rare event.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12591

Test Plan: 1. Add a unit test to verify the bug and fix

Reviewed By: jowlyzhang

Differential Revision: D56655812

Pulled By: anand1976

fbshipit-source-id: aa3a26b4a5e892d39a54b5a0658233cbebebac87
2024-04-29 12:25:00 -07:00
..
compacted_db_impl.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
compacted_db_impl.h Deprecate some variants of Get and MultiGet (#12327) 2024-02-16 09:21:06 -08:00
db_impl.cc Fix IteratorsConsistentView tests (#12582) 2024-04-25 14:06:46 -07:00
db_impl.h Fix corruption bug when recycle_log_file_num changed from 0 (#12591) 2024-04-29 12:25:00 -07:00
db_impl_compaction_flush.cc Prevent data block compression with `BlockBasedTableOptions::block_align` (#12592) 2024-04-26 20:05:30 -07:00
db_impl_debug.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_impl_experimental.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_impl_files.cc Fix corruption bug when recycle_log_file_num changed from 0 (#12591) 2024-04-29 12:25:00 -07:00
db_impl_follower.cc Make sure DBImplFollower::stop_requested_ is initialized (#12572) 2024-04-22 12:02:28 -07:00
db_impl_follower.h Basic RocksDB follower implementation (#12540) 2024-04-19 19:13:31 -07:00
db_impl_open.cc Fix corruption bug when recycle_log_file_num changed from 0 (#12591) 2024-04-29 12:25:00 -07:00
db_impl_readonly.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_impl_readonly.h Remove the force mode for EnableFileDeletions API (#12337) 2024-02-13 18:36:25 -08:00
db_impl_secondary.cc Retry DB::Open upon a corruption detected while reading the MANIFEST (#12518) 2024-04-18 17:36:33 -07:00
db_impl_secondary.h Basic RocksDB follower implementation (#12540) 2024-04-19 19:13:31 -07:00
db_impl_write.cc Enable recycle_log_file_num option for point in time recovery (#12403) 2024-03-21 12:29:35 -07:00