rocksdb/db/blob
Levi Tamasi b0e7834100 Integrate blob file writing with the flush logic (#7345)
Summary:
The patch adds support for writing blob files during flush by integrating
`BlobFileBuilder` with the flush logic, most importantly, `BuildTable` and
`CompactionIterator`. If `enable_blob_files` is set, large values are extracted
to blob files and replaced with references. The resulting blob files are then
logged to the MANIFEST as part of the flush job's `VersionEdit` and
added to the `Version`, similarly to table files. Errors related to writing
blob files fail the flush, and any blob files written by such jobs are immediately
deleted (again, similarly to how SST files are handled). In addition, the patch
extends the logging and statistics around flushes to account for the presence
of blob files (e.g. `InternalStats::CompactionStats::bytes_written`, which is
used for calculating write amplification, now considers the blob files as well).

Pull Request resolved: https://github.com/facebook/rocksdb/pull/7345

Test Plan: Tested using `make check` and `db_bench`.

Reviewed By: riversand963

Differential Revision: D23506369

Pulled By: ltamasi

fbshipit-source-id: 646885f22dfbe063f650d38a1fedc132f499a159
2020-09-14 21:11:43 -07:00
..
blob_constants.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_addition.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_addition.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_addition_test.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_builder.cc Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
blob_file_builder.h Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
blob_file_builder_test.cc Integrate blob file writing with the flush logic (#7345) 2020-09-14 21:11:43 -07:00
blob_file_garbage.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage_test.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_meta.cc Maintain the set of linked SSTs in BlobFileMetaData (#6945) 2020-06-12 09:54:39 -07:00
blob_file_meta.h Maintain the set of linked SSTs in BlobFileMetaData (#6945) 2020-06-12 09:54:39 -07:00
blob_index.h Build blob file reader/writer classes in LITE mode as well (#7272) 2020-08-17 15:19:05 -07:00
blob_log_format.cc Build blob file reader/writer classes in LITE mode as well (#7272) 2020-08-17 15:19:05 -07:00
blob_log_format.h Build blob file reader/writer classes in LITE mode as well (#7272) 2020-08-17 15:19:05 -07:00
blob_log_reader.cc Add a blob file builder class that can be used in background jobs (#7306) 2020-08-27 11:55:54 -07:00
blob_log_reader.h Add a blob file builder class that can be used in background jobs (#7306) 2020-08-27 11:55:54 -07:00
blob_log_writer.cc Add a blob file builder class that can be used in background jobs (#7306) 2020-08-27 11:55:54 -07:00
blob_log_writer.h Add a blob file builder class that can be used in background jobs (#7306) 2020-08-27 11:55:54 -07:00
db_blob_index_test.cc Whole DBTest to skip fsync (#7274) 2020-08-17 18:42:25 -07:00