rocksdb/file
Hui Xiao 7d83b4e3e5 Fix file checksum mismatch due to padded bytes when block_align=true (#12542)
Summary:
**Context/Summary:**
When `BlockBasedTableOptions::block_align=true`, we pad bytes to align blocks d41e568b1c/table/block_based/block_based_table_builder.cc (L1415-L1421).
Those bytes are not included in generating the file checksum upon file creation. But `VerifyFileChecksums()` includes those bytes in generating the file check to compare against the checksum generating upon file creation. Therefore a file checksum mismatch is returned in `VerifyFileChecksums()`.

We decided to include those padded bytes in generating the checksum upon file creation.

Bonus: also fix surrounding code to use actual padded bytes for verification - see https://github.com/facebook/rocksdb/pull/12542#discussion_r1571429163

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12542

Test Plan:
- New UT
- Benchmark
```
TEST_TMPDIR=/dev/shm  ./db_bench --benchmarks=fillseq[-X300] --num=100000 --block_align=1 --compression_type=none
```
Pre-PR:
fillseq [AVG    300 runs] : 422857 (± 3942) ops/sec;   46.8 (± 0.4) MB/sec
Post-PR:
fillseq [AVG    300 runs] : 424707 (± 3799) ops/sec;   47.0 (± 0.4) MB/sec

Reviewed By: ajkr

Differential Revision: D56168447

Pulled By: hx235

fbshipit-source-id: 96209ef950d42943d336f11968ae3fcf9872fc2c
2024-04-22 14:07:34 -07:00
..
delete_scheduler.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
delete_scheduler.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
delete_scheduler_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
file_prefetch_buffer.cc Change ReadAsync callback API to remove const from FSReadRequest (#11649) 2024-02-16 09:14:55 -08:00
file_prefetch_buffer.h Change ReadAsync callback API to remove const from FSReadRequest (#11649) 2024-02-16 09:14:55 -08:00
file_util.cc Fix/improve temperature handling for file ingestion (#12402) 2024-03-05 16:56:08 -08:00
file_util.h Fix/improve temperature handling for file ingestion (#12402) 2024-03-05 16:56:08 -08:00
filename.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
filename.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
line_file_reader.cc Support read rate-limiting in SequentialFileReader (#9973) 2022-05-24 10:28:57 -07:00
line_file_reader.h Support read rate-limiting in SequentialFileReader (#9973) 2022-05-24 10:28:57 -07:00
prefetch_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
random_access_file_reader.cc default_write_temperature option (#12388) 2024-02-28 14:36:13 -08:00
random_access_file_reader.h Change ReadAsync callback API to remove const from FSReadRequest (#11649) 2024-02-16 09:14:55 -08:00
random_access_file_reader_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
read_write_util.cc Run Clang format on file folder (#10860) 2022-10-24 18:34:52 -07:00
read_write_util.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
readahead_file_info.h Reuse internal auto readhead_size at each Level (expect L0) for Iterations (#9056) 2021-11-10 16:20:04 -08:00
readahead_raf.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
readahead_raf.h Make StringEnv, StringSink, StringSource use FS classes (#7786) 2021-01-04 16:01:01 -08:00
sequence_file_reader.cc Retry DB::Open upon a corruption detected while reading the MANIFEST (#12518) 2024-04-18 17:36:33 -07:00
sequence_file_reader.h Retry DB::Open upon a corruption detected while reading the MANIFEST (#12518) 2024-04-18 17:36:33 -07:00
sst_file_manager_impl.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
sst_file_manager_impl.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
writable_file_writer.cc Fix file checksum mismatch due to padded bytes when block_align=true (#12542) 2024-04-22 14:07:34 -07:00
writable_file_writer.h Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00