dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Enhance corruption status message for record mismatch in compaction (#12297) 

Summary:
... to include the actual numbers of processed and expected records, and the file number for input files. The purpose is to be able to find the offending files even when the relevant LOG file is gone.

Another change is to check the record count even when `compaction_verify_record_count` is false, and log a warning message without setting corruption status if there is a mismatch. This is consistent with how we check the record count for flush.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12297

Test Plan:
print the status message in `DBCompactionTest.VerifyRecordCount`
```
before
Corruption: Compaction number of input keys does not match number of keys processed.
after
Compaction number of input keys does not match number of keys processed. Expected 20 but processed 10. Compaction summary: Base version 4 Base level 0, inputs: [11(2156B) 9(2156B)]
```

Reviewed By: ajkr

Differential Revision: D53110130

Pulled By: cbi42

fbshipit-source-id: 6325cbfb8f71f25ce37f23f8277ebe9264863c3b
This commit is contained in:
Changyu Bi 2024-01-26 09:12:07 -08:00 committed by Facebook GitHub Bot
parent a31fded253
commit 2233a2f4c0
1 changed files with 11 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
db/compaction/compaction_job.cc
View File

@ -831,24 +831,27 @@ Status CompactionJob::Run() {
// input keys. So the number of keys it processed is not suitable for
// verification here.
// TODO: support verification when trim_ts_ is non-empty.
if (!(ts_sz > 0 && !trim_ts_.empty()) &&
db_options_.compaction_verify_record_count) {
if (!(ts_sz > 0 && !trim_ts_.empty())) {
assert(compaction_stats_.stats.num_input_records > 0);
// TODO: verify the number of range deletion entries.
uint64_t expected =
compaction_stats_.stats.num_input_records - num_input_range_del;
uint64_t actual = compaction_job_stats_->num_input_records;
if (expected != actual) {
char scratch[2345];
compact_->compaction->Summary(scratch, sizeof(scratch));
std::string msg =
"Total number of input records: " + std::to_string(expected) +
", but processed " + std::to_string(actual) + " records.";
"Compaction number of input keys does not match "
"number of keys processed. Expected " +
std::to_string(expected) + " but processed " +
std::to_string(actual) + ". Compaction summary: " + scratch;
ROCKS_LOG_WARN(
db_options_.info_log, "[%s] [JOB %d] Compaction %s",
db_options_.info_log, "[%s] [JOB %d] Compaction with status: %s",
compact_->compaction->column_family_data()->GetName().c_str(),
job_context_->job_id, msg.c_str());
status = Status::Corruption(
"Compaction number of input keys does not match number of keys "
"processed.");
if (db_options_.compaction_verify_record_count) {
status = Status::Corruption(msg);
}
}
}
}