rocksdb/db/db_impl
Zhichao Cao 4369f2c7bb Checksum for each SST file and stores in MANIFEST (#6216)
Summary:
In the current code base, RocksDB generate the checksum for each block and verify the checksum at usage. Current PR enable SST file checksum. After a SST file is generated by Flush or Compaction, RocksDB generate the SST file checksum and store the checksum value and checksum method name in the vs_info and MANIFEST as part for the FileMetadata.

Added the enable_sst_file_checksum to Options to enable or disable file checksum. Added sst_file_checksum to Options such that user can plugin their own SST file checksum calculate method via overriding the SstFileChecksum class. The checksum information inlcuding uint32_t checksum value and a checksum name (string).  A new tool is added to LDB such that user can dump out a list of file checksum information from MANIFEST. If user enables the file checksum but does not provide the sst_file_checksum instance, RocksDB will use the default crc32checksum implemented in table/sst_file_checksum_crc32c.h
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6216

Test Plan: Added the testing case in table_test and ldb_cmd_test to verify checksum is correct in different level. Pass make asan_check.

Differential Revision: D19171461

Pulled By: zhichao-cao

fbshipit-source-id: b2e53479eefc5bb0437189eaa1941670e5ba8b87
2020-02-10 15:52:52 -08:00
..
db_impl.cc Add an option to prevent DB::Open() from querying sizes of all sst files (#6353) 2020-02-04 01:27:26 -08:00
db_impl.h Avoid create directory for every column families (#6358) 2020-02-03 14:13:39 -08:00
db_impl_compaction_flush.cc Checksum for each SST file and stores in MANIFEST (#6216) 2020-02-10 15:52:52 -08:00
db_impl_debug.cc Fix IngestExternalFile's bug with two_write_queue (#5976) 2019-11-15 14:00:37 -08:00
db_impl_experimental.cc Checksum for each SST file and stores in MANIFEST (#6216) 2020-02-10 15:52:52 -08:00
db_impl_files.cc Clean up VersionEdit a bit (#6383) 2020-02-07 13:27:06 -08:00
db_impl_open.cc Checksum for each SST file and stores in MANIFEST (#6216) 2020-02-10 15:52:52 -08:00
db_impl_readonly.cc Apply formatter on recent 45 commits. (#5827) 2019-09-19 12:34:17 -07:00
db_impl_readonly.h Use delete to disable automatic generated methods. (#5009) 2019-09-11 18:09:00 -07:00
db_impl_secondary.cc Add an option to prevent DB::Open() from querying sizes of all sst files (#6353) 2020-02-04 01:27:26 -08:00
db_impl_secondary.h Double Crash in kPointInTimeRecovery with TransactionDB (#6313) 2020-01-29 11:40:55 -08:00
db_impl_write.cc Add error status for no_slowdown & low priority write (#6396) 2020-02-10 12:33:16 -08:00
db_secondary_test.cc Break db_stress_tool.cc to a list of source files (#6134) 2019-12-08 23:51:01 -08:00