rocksdb/env
Gang Liao e228515740 Pass the size of blob files to SstFileManager during DB open (#10062)
Summary:
RocksDB uses the (no longer aptly named) SST file manager (see https://github.com/facebook/rocksdb/wiki/Managing-Disk-Space-Utilization) to track and potentially limit the space used by SST and blob files (as well as to rate-limit the deletion of these data files). The SST file manager tracks the SST and blob file sizes in an in-memory hash map, which has to be rebuilt during DB open. File sizes can be generally obtained by querying the file system; however, there is a performance optimization possibility here since the sizes of SST and blob files are also tracked in the RocksDB MANIFEST, so we can simply pass the file sizes stored there instead of consulting the file system for each file. Currently, this optimization is only implemented for SST files; we would like to extend it to blob files as well.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10062

Test Plan:
Add unit tests for the change to the test suite
ltamasi riversand963  akankshamahajan15

Reviewed By: ltamasi

Differential Revision: D36726621

Pulled By: gangliao

fbshipit-source-id: 4010dc46ef7306142f1c2e0d1c3bf75b196ef82a
2022-05-27 05:58:43 -07:00
..
composite_env.cc Option type info functions (#9411) 2022-05-13 04:57:08 -07:00
composite_env_wrapper.h Make the Env class Customizable (#9293) 2022-01-04 16:45:49 -08:00
emulated_clock.h Make SystemClock into a Customizable Class (#8636) 2021-09-21 09:23:48 -07:00
env.cc Option type info functions (#9411) 2022-05-13 04:57:08 -07:00
env_basic_test.cc Make SystemClock into a Customizable Class (#8636) 2021-09-21 09:23:48 -07:00
env_chroot.cc Make FileSystem a Customizable Class (#8649) 2021-11-02 09:07:11 -07:00
env_chroot.h Make FileSystem a Customizable Class (#8649) 2021-11-02 09:07:11 -07:00
env_encryption.cc Added GetFactoryCount/Names/Types to ObjectRegistry (#9358) 2022-05-16 09:44:43 -07:00
env_encryption_ctr.h Added GetFactoryCount/Names/Types to ObjectRegistry (#9358) 2022-05-16 09:44:43 -07:00
env_posix.cc Fix potential ambiguities in/around port/sys_time.h (#10045) 2022-05-24 18:20:17 -07:00
env_test.cc Fix potential ambiguities in/around port/sys_time.h (#10045) 2022-05-24 18:20:17 -07:00
file_system.cc Introduce a CountedFileSystem for counting file operations (#9283) 2022-02-03 15:01:23 -08:00
file_system_tracer.cc Provide support for IOTracing for ReadAsync API (#9833) 2022-05-25 19:47:03 -07:00
file_system_tracer.h Provide support for IOTracing for ReadAsync API (#9833) 2022-05-25 19:47:03 -07:00
fs_posix.cc Use STATIC_AVOID_DESTRUCTION for static objects with non-trivial destructors (#9958) 2022-05-17 09:39:22 -07:00
fs_readonly.h Make FileSystem a Customizable Class (#8649) 2021-11-02 09:07:11 -07:00
fs_remap.cc Test refactoring for Backups+Temperatures (#9655) 2022-03-04 12:32:30 -08:00
fs_remap.h Make FileSystem a Customizable Class (#8649) 2021-11-02 09:07:11 -07:00
io_posix.cc Remove own ToString() (#9955) 2022-05-06 13:03:58 -07:00
io_posix.h Support read rate-limiting in SequentialFileReader (#9973) 2022-05-24 10:28:57 -07:00
io_posix_test.cc
mock_env.cc Pass the size of blob files to SstFileManager during DB open (#10062) 2022-05-27 05:58:43 -07:00
mock_env.h Make the Env class Customizable (#9293) 2022-01-04 16:45:49 -08:00
mock_env_test.cc Make SystemClock into a Customizable Class (#8636) 2021-09-21 09:23:48 -07:00
unique_id_gen.cc New stable, fixed-length cache keys (#9126) 2021-12-16 17:15:13 -08:00
unique_id_gen.h New stable, fixed-length cache keys (#9126) 2021-12-16 17:15:13 -08:00