Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
|
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
|
|
|
#include <cassert>
|
|
|
|
#include <iosfwd>
|
|
|
|
#include <memory>
|
|
|
|
#include <string>
|
2020-06-12 16:52:14 +00:00
|
|
|
#include <unordered_set>
|
|
|
|
|
|
|
|
#include "rocksdb/rocksdb_namespace.h"
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
|
|
|
|
// SharedBlobFileMetaData represents the immutable part of blob files' metadata,
|
|
|
|
// like the blob file number, total number and size of blobs, or checksum
|
|
|
|
// method and value. There is supposed to be one object of this class per blob
|
|
|
|
// file (shared across all versions that include the blob file in question);
|
|
|
|
// hence, the type is neither copyable nor movable. A blob file can be marked
|
|
|
|
// obsolete when the corresponding SharedBlobFileMetaData object is destroyed.
|
|
|
|
|
|
|
|
class SharedBlobFileMetaData {
|
|
|
|
public:
|
2020-04-23 20:41:32 +00:00
|
|
|
static std::shared_ptr<SharedBlobFileMetaData> Create(
|
|
|
|
uint64_t blob_file_number, uint64_t total_blob_count,
|
|
|
|
uint64_t total_blob_bytes, std::string checksum_method,
|
|
|
|
std::string checksum_value) {
|
|
|
|
return std::shared_ptr<SharedBlobFileMetaData>(new SharedBlobFileMetaData(
|
|
|
|
blob_file_number, total_blob_count, total_blob_bytes,
|
|
|
|
std::move(checksum_method), std::move(checksum_value)));
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
}
|
|
|
|
|
2020-04-23 20:41:32 +00:00
|
|
|
template <typename Deleter>
|
|
|
|
static std::shared_ptr<SharedBlobFileMetaData> Create(
|
|
|
|
uint64_t blob_file_number, uint64_t total_blob_count,
|
|
|
|
uint64_t total_blob_bytes, std::string checksum_method,
|
|
|
|
std::string checksum_value, Deleter deleter) {
|
|
|
|
return std::shared_ptr<SharedBlobFileMetaData>(
|
|
|
|
new SharedBlobFileMetaData(blob_file_number, total_blob_count,
|
|
|
|
total_blob_bytes, std::move(checksum_method),
|
|
|
|
std::move(checksum_value)),
|
|
|
|
deleter);
|
|
|
|
}
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
|
|
|
|
SharedBlobFileMetaData(const SharedBlobFileMetaData&) = delete;
|
|
|
|
SharedBlobFileMetaData& operator=(const SharedBlobFileMetaData&) = delete;
|
|
|
|
|
2020-04-23 20:41:32 +00:00
|
|
|
SharedBlobFileMetaData(SharedBlobFileMetaData&&) = delete;
|
|
|
|
SharedBlobFileMetaData& operator=(SharedBlobFileMetaData&&) = delete;
|
|
|
|
|
2021-06-28 15:12:32 +00:00
|
|
|
uint64_t GetBlobFileSize() const;
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
uint64_t GetBlobFileNumber() const { return blob_file_number_; }
|
|
|
|
uint64_t GetTotalBlobCount() const { return total_blob_count_; }
|
|
|
|
uint64_t GetTotalBlobBytes() const { return total_blob_bytes_; }
|
|
|
|
const std::string& GetChecksumMethod() const { return checksum_method_; }
|
|
|
|
const std::string& GetChecksumValue() const { return checksum_value_; }
|
|
|
|
|
|
|
|
std::string DebugString() const;
|
|
|
|
|
|
|
|
private:
|
2020-04-23 20:41:32 +00:00
|
|
|
SharedBlobFileMetaData(uint64_t blob_file_number, uint64_t total_blob_count,
|
|
|
|
uint64_t total_blob_bytes, std::string checksum_method,
|
|
|
|
std::string checksum_value)
|
|
|
|
: blob_file_number_(blob_file_number),
|
|
|
|
total_blob_count_(total_blob_count),
|
|
|
|
total_blob_bytes_(total_blob_bytes),
|
|
|
|
checksum_method_(std::move(checksum_method)),
|
|
|
|
checksum_value_(std::move(checksum_value)) {
|
|
|
|
assert(checksum_method_.empty() == checksum_value_.empty());
|
|
|
|
}
|
|
|
|
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
uint64_t blob_file_number_;
|
|
|
|
uint64_t total_blob_count_;
|
|
|
|
uint64_t total_blob_bytes_;
|
|
|
|
std::string checksum_method_;
|
|
|
|
std::string checksum_value_;
|
|
|
|
};
|
|
|
|
|
|
|
|
std::ostream& operator<<(std::ostream& os,
|
|
|
|
const SharedBlobFileMetaData& shared_meta);
|
|
|
|
|
|
|
|
// BlobFileMetaData contains the part of the metadata for blob files that can
|
|
|
|
// vary across versions, like the amount of garbage in the blob file. In
|
|
|
|
// addition, BlobFileMetaData objects point to and share the ownership of the
|
|
|
|
// SharedBlobFileMetaData object for the corresponding blob file. Similarly to
|
|
|
|
// SharedBlobFileMetaData, BlobFileMetaData are not copyable or movable. They
|
|
|
|
// are meant to be jointly owned by the versions in which the blob file has the
|
|
|
|
// same (immutable *and* mutable) state.
|
|
|
|
|
|
|
|
class BlobFileMetaData {
|
|
|
|
public:
|
2020-06-12 16:52:14 +00:00
|
|
|
using LinkedSsts = std::unordered_set<uint64_t>;
|
|
|
|
|
2020-04-23 20:41:32 +00:00
|
|
|
static std::shared_ptr<BlobFileMetaData> Create(
|
|
|
|
std::shared_ptr<SharedBlobFileMetaData> shared_meta,
|
2020-06-12 16:52:14 +00:00
|
|
|
LinkedSsts linked_ssts, uint64_t garbage_blob_count,
|
|
|
|
uint64_t garbage_blob_bytes) {
|
|
|
|
return std::shared_ptr<BlobFileMetaData>(
|
|
|
|
new BlobFileMetaData(std::move(shared_meta), std::move(linked_ssts),
|
|
|
|
garbage_blob_count, garbage_blob_bytes));
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
BlobFileMetaData(const BlobFileMetaData&) = delete;
|
|
|
|
BlobFileMetaData& operator=(const BlobFileMetaData&) = delete;
|
|
|
|
|
2020-04-23 20:41:32 +00:00
|
|
|
BlobFileMetaData(BlobFileMetaData&&) = delete;
|
|
|
|
BlobFileMetaData& operator=(BlobFileMetaData&&) = delete;
|
|
|
|
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
const std::shared_ptr<SharedBlobFileMetaData>& GetSharedMeta() const {
|
|
|
|
return shared_meta_;
|
|
|
|
}
|
|
|
|
|
2021-06-28 15:12:32 +00:00
|
|
|
uint64_t GetBlobFileSize() const {
|
|
|
|
assert(shared_meta_);
|
|
|
|
return shared_meta_->GetBlobFileSize();
|
|
|
|
}
|
|
|
|
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
uint64_t GetBlobFileNumber() const {
|
|
|
|
assert(shared_meta_);
|
|
|
|
return shared_meta_->GetBlobFileNumber();
|
|
|
|
}
|
|
|
|
uint64_t GetTotalBlobCount() const {
|
|
|
|
assert(shared_meta_);
|
|
|
|
return shared_meta_->GetTotalBlobCount();
|
|
|
|
}
|
|
|
|
uint64_t GetTotalBlobBytes() const {
|
|
|
|
assert(shared_meta_);
|
|
|
|
return shared_meta_->GetTotalBlobBytes();
|
|
|
|
}
|
|
|
|
const std::string& GetChecksumMethod() const {
|
|
|
|
assert(shared_meta_);
|
|
|
|
return shared_meta_->GetChecksumMethod();
|
|
|
|
}
|
|
|
|
const std::string& GetChecksumValue() const {
|
|
|
|
assert(shared_meta_);
|
|
|
|
return shared_meta_->GetChecksumValue();
|
|
|
|
}
|
|
|
|
|
2020-06-12 16:52:14 +00:00
|
|
|
const LinkedSsts& GetLinkedSsts() const { return linked_ssts_; }
|
|
|
|
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
uint64_t GetGarbageBlobCount() const { return garbage_blob_count_; }
|
|
|
|
uint64_t GetGarbageBlobBytes() const { return garbage_blob_bytes_; }
|
|
|
|
|
|
|
|
std::string DebugString() const;
|
|
|
|
|
|
|
|
private:
|
2020-04-23 20:41:32 +00:00
|
|
|
BlobFileMetaData(std::shared_ptr<SharedBlobFileMetaData> shared_meta,
|
2020-06-12 16:52:14 +00:00
|
|
|
LinkedSsts linked_ssts, uint64_t garbage_blob_count,
|
|
|
|
uint64_t garbage_blob_bytes)
|
2020-04-23 20:41:32 +00:00
|
|
|
: shared_meta_(std::move(shared_meta)),
|
2020-06-12 16:52:14 +00:00
|
|
|
linked_ssts_(std::move(linked_ssts)),
|
2020-04-23 20:41:32 +00:00
|
|
|
garbage_blob_count_(garbage_blob_count),
|
|
|
|
garbage_blob_bytes_(garbage_blob_bytes) {
|
|
|
|
assert(shared_meta_);
|
|
|
|
assert(garbage_blob_count_ <= shared_meta_->GetTotalBlobCount());
|
|
|
|
assert(garbage_blob_bytes_ <= shared_meta_->GetTotalBlobBytes());
|
|
|
|
}
|
|
|
|
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
std::shared_ptr<SharedBlobFileMetaData> shared_meta_;
|
2020-06-12 16:52:14 +00:00
|
|
|
LinkedSsts linked_ssts_;
|
Add blob files to VersionStorageInfo/VersionBuilder (#6597)
Summary:
The patch adds a couple of classes to represent metadata about
blob files: `SharedBlobFileMetaData` contains the information elements
that are immutable (once the blob file is closed), e.g. blob file number,
total number and size of blob files, checksum method/value, while
`BlobFileMetaData` contains attributes that can vary across versions like
the amount of garbage in the file. There is a single `SharedBlobFileMetaData`
for each blob file, which is jointly owned by the `BlobFileMetaData` objects
that point to it; `BlobFileMetaData` objects, in turn, are owned by `Version`s
and can also be shared if the (immutable _and_ mutable) state of the blob file
is the same in two versions.
In addition, the patch adds the blob file metadata to `VersionStorageInfo`, and extends
`VersionBuilder` so that it can apply blob file related `VersionEdit`s (i.e. those
containing `BlobFileAddition`s and/or `BlobFileGarbage`), and save blob file metadata
to a new `VersionStorageInfo`. Consistency checks are also extended to ensure
that table files point to blob files that are part of the `Version`, and that all blob files
that are part of any given `Version` have at least some _non_-garbage data in them.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6597
Test Plan: `make check`
Reviewed By: riversand963
Differential Revision: D20656803
Pulled By: ltamasi
fbshipit-source-id: f1f74d135045b3b42d0146f03ee576ef0a4bfd80
2020-03-27 01:48:55 +00:00
|
|
|
uint64_t garbage_blob_count_;
|
|
|
|
uint64_t garbage_blob_bytes_;
|
|
|
|
};
|
|
|
|
|
|
|
|
std::ostream& operator<<(std::ostream& os, const BlobFileMetaData& meta);
|
|
|
|
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|