rocksdb/db/db_impl
Gang Liao e228515740 Pass the size of blob files to SstFileManager during DB open (#10062)
Summary:
RocksDB uses the (no longer aptly named) SST file manager (see https://github.com/facebook/rocksdb/wiki/Managing-Disk-Space-Utilization) to track and potentially limit the space used by SST and blob files (as well as to rate-limit the deletion of these data files). The SST file manager tracks the SST and blob file sizes in an in-memory hash map, which has to be rebuilt during DB open. File sizes can be generally obtained by querying the file system; however, there is a performance optimization possibility here since the sizes of SST and blob files are also tracked in the RocksDB MANIFEST, so we can simply pass the file sizes stored there instead of consulting the file system for each file. Currently, this optimization is only implemented for SST files; we would like to extend it to blob files as well.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10062

Test Plan:
Add unit tests for the change to the test suite
ltamasi riversand963  akankshamahajan15

Reviewed By: ltamasi

Differential Revision: D36726621

Pulled By: gangliao

fbshipit-source-id: 4010dc46ef7306142f1c2e0d1c3bf75b196ef82a
2022-05-27 05:58:43 -07:00
..
compacted_db_impl.cc Add timestamp support to CompactedDBImpl (#10030) 2022-05-24 12:14:10 -07:00
compacted_db_impl.h Add timestamp support to CompactedDBImpl (#10030) 2022-05-24 12:14:10 -07:00
db_impl.cc Update VersionSet last seqno after LogAndApply (#10051) 2022-05-25 10:05:17 -07:00
db_impl.h Add timestamp support to DBImplReadOnly (#10004) 2022-05-19 18:39:41 -07:00
db_impl_compaction_flush.cc Move three info logging within DB Mutex to use log buffer (#10029) 2022-05-23 10:09:37 -07:00
db_impl_debug.cc Use std::numeric_limits<> (#9954) 2022-05-05 13:08:21 -07:00
db_impl_experimental.cc Track SST unique id in MANIFEST and verify (#9990) 2022-05-19 11:04:21 -07:00
db_impl_files.cc FindObsoleteFiles() to directly check whether candidate files are live (#10040) 2022-05-25 12:43:48 -07:00
db_impl_open.cc Pass the size of blob files to SstFileManager during DB open (#10062) 2022-05-27 05:58:43 -07:00
db_impl_readonly.cc Add timestamp support to DBImplReadOnly (#10004) 2022-05-19 18:39:41 -07:00
db_impl_readonly.h Add timestamp support to DBImplReadOnly (#10004) 2022-05-19 18:39:41 -07:00
db_impl_secondary.cc Add timestamp support to secondary instance (#10061) 2022-05-26 19:45:31 -07:00
db_impl_secondary.h Add timestamp support to secondary instance (#10061) 2022-05-26 19:45:31 -07:00
db_impl_write.cc Do not release and re-acquire dbmutex on memtable-switch if no listener (#9758) 2022-03-30 20:48:23 -07:00