rocksdb/utilities/blob_db
Levi Tamasi 279c488395 Mark blob files not needed by any memtables/SSTs obsolete (#6032)
Summary:
The patch adds logic to mark no longer needed blob files obsolete upon database open
and whenever a flush or compaction completes. Unneeded blob files are detected by
iterating through live immutable non-TTL blob files starting from the lowest-numbered one,
and stopping when a blob file used by any SSTs or potentially used by memtables is found.
(The latter is determined by comparing the sequence number at which the blob file
became immutable with the largest sequence number received in flush notifications.)

In addition, the patch cleans up the logic around closing and obsoleting blob files and
enforces invariants around this area (blob files are now guaranteed to go through the
stages mutable-non-obsolete, immutable-non-obsolete, and immutable-obsolete in this
order).
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6032

Test Plan: Extended unit tests and tested using the BlobDB mode of `db_bench`.

Differential Revision: D18495610

Pulled By: ltamasi

fbshipit-source-id: 11825b84af74f3f4abfd9bcae04e80870ae58961
2019-11-18 16:30:06 -08:00
..
blob_compaction_filter.cc Apply modernize-use-override (2nd iteration) 2019-02-14 14:41:36 -08:00
blob_compaction_filter.h Move blob_index.h to db/ (#5919) 2019-10-14 12:54:05 -07:00
blob_db.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
blob_db.h BlobDB: Maintain mapping between blob files and SSTs (#6020) 2019-11-11 14:01:34 -08:00
blob_db_impl.cc Mark blob files not needed by any memtables/SSTs obsolete (#6032) 2019-11-18 16:30:06 -08:00
blob_db_impl.h Mark blob files not needed by any memtables/SSTs obsolete (#6032) 2019-11-18 16:30:06 -08:00
blob_db_impl_filesnapshot.cc Add file number/oldest referenced blob file number to {Sst,Live}FileMetaData (#6011) 2019-11-07 14:04:16 -08:00
blob_db_iterator.h Refactor ArenaWrappedDBIter into separate files (#5801) 2019-09-13 13:50:43 -07:00
blob_db_listener.h BlobDB: Maintain mapping between blob files and SSTs (#6020) 2019-11-11 14:01:34 -08:00
blob_db_test.cc Mark blob files not needed by any memtables/SSTs obsolete (#6032) 2019-11-18 16:30:06 -08:00
blob_dump_tool.cc Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
blob_dump_tool.h Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
blob_file.cc Mark blob files not needed by any memtables/SSTs obsolete (#6032) 2019-11-18 16:30:06 -08:00
blob_file.h Mark blob files not needed by any memtables/SSTs obsolete (#6032) 2019-11-18 16:30:06 -08:00
blob_log_format.cc utilities: Fix build failure with -Werror=maybe-uninitialized (#5074) 2019-03-18 11:35:06 -07:00
blob_log_format.h BlobDB: use char array instead of string as buffer (#4662) 2018-11-13 12:49:29 -08:00
blob_log_reader.cc Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
blob_log_reader.h Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
blob_log_writer.cc Divide file_reader_writer.h and .cc (#5803) 2019-09-16 10:33:51 -07:00
blob_log_writer.h Use delete to disable automatic generated methods. (#5009) 2019-09-11 18:09:00 -07:00