rocksdb/utilities
Zitan Chen b923dc720b BackupEngine computes table checksums only once if db session ids are available (#7110)
Summary:
BackupEngine requires computing table checksums twice when backing up table files to the `shared_checksum` directory.

The repeated computation can be avoided by utilizing the db session id stored as a part of the table properties.

Filenames of table files in the `shared_checksum` directory depend on the following conditions:
1. the naming scheme is `kOptionalChecksumAndDbSessionId`,
2. `db_session_id` is not empty,
3. checksum is available in the DB manifest.

If 1,2,3 are satisfied, then the filenames will be of the form `<file_number>_<checksum>_<db_session_id>.sst`.
If 1,2 are satisfied, then the filenames will be of the form `<file_number>_<db_session_id>.sst`.
In all other cases, the filenames are of the form `<file_number>_<checksum>_<size>.sst`.

Additionally, if `kOptionalChecksumAndDbSessionId` is used (and not falling back to `kChecksumAndFileSize`), the `<checksum>` appeared in the filenames is hexadecimally encoded, instead of being plain `uint32_t` value.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7110

Test Plan: backupable_db_test and manual tests.

Reviewed By: ajkr

Differential Revision: D22508992

Pulled By: gg814

fbshipit-source-id: 5669f0ea9ad5a097f69f6d87aca4abba15032389
2020-07-21 10:35:40 -07:00
..
backupable BackupEngine computes table checksums only once if db session ids are available (#7110) 2020-07-21 10:35:40 -07:00
blob_db Remove some more dead code around syncing blob files (#7138) 2020-07-15 18:53:54 -07:00
cassandra Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
checkpoint More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
compaction_filters Compaction filter support for BlobDB (#6850) 2020-06-29 17:32:14 -07:00
convenience Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
leveldb_options Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
memory More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
merge_operators Make StringAppendOperatorTest a parameterized test (#6930) 2020-06-04 14:17:11 -07:00
option_change_migration More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
options Fixed Factory construct just for calling .Name() (#7080) 2020-07-08 11:54:00 -07:00
persistent_cache More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
simulator_cache Revert "Whole DBTest to skip fsync (#7049)" (#7070) 2020-07-02 10:22:43 -07:00
table_properties_collectors Trigger compaction in CompactOnDeletionCollector based on deletion ratio (#6806) 2020-05-18 08:42:05 -07:00
trace Pass a timeout to FileSystem for random reads (#6751) 2020-04-30 14:50:39 -07:00
transactions Add CLANG analyze to CircleCI (#7114) 2020-07-13 12:33:16 -07:00
ttl Compaction filter support for BlobDB (#6850) 2020-06-29 17:32:14 -07:00
write_batch_with_index Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
debug.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_librados.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_librados.md Add EnvLibrados - RocksDB Env of RADOS (#1222) 2016-07-21 11:16:34 -07:00
env_librados_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_mirror.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_mirror_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_timed.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
env_timed_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
fault_injection_env.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
fault_injection_env.h More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
fault_injection_fs.cc More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
fault_injection_fs.h More Makefile Cleanup (#7097) 2020-07-09 14:35:17 -07:00
merge_operators.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
object_registry.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
object_registry_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
util_merge_operators_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00