Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
// Copyright (c) Meta Platforms, Inc. and affiliates.
|
|
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
|
|
|
|
|
|
#pragma once
|
|
|
|
|
2024-03-18 16:05:30 +00:00
|
|
|
#include <variant>
|
|
|
|
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
#include "rocksdb/comparator.h"
|
|
|
|
#include "rocksdb/iterator.h"
|
|
|
|
#include "rocksdb/options.h"
|
|
|
|
#include "util/heap.h"
|
2024-03-18 16:05:30 +00:00
|
|
|
#include "util/overload.h"
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
|
|
|
|
// UNDER CONSTRUCTION - DO NOT USE
|
|
|
|
// A cross-column-family iterator from a consistent database state.
|
|
|
|
// When the same key exists in more than one column families, the iterator
|
|
|
|
// selects the value from the first column family containing the key, in the
|
|
|
|
// order provided in the `column_families` parameter.
|
|
|
|
class MultiCfIterator : public Iterator {
|
|
|
|
public:
|
|
|
|
MultiCfIterator(const Comparator* comparator,
|
|
|
|
const std::vector<ColumnFamilyHandle*>& column_families,
|
|
|
|
const std::vector<Iterator*>& child_iterators)
|
|
|
|
: comparator_(comparator),
|
2024-03-18 16:05:30 +00:00
|
|
|
heap_(MultiCfMinHeap(
|
|
|
|
MultiCfHeapItemComparator<std::greater<int>>(comparator_))) {
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
assert(column_families.size() > 0 &&
|
|
|
|
column_families.size() == child_iterators.size());
|
|
|
|
cfh_iter_pairs_.reserve(column_families.size());
|
|
|
|
for (size_t i = 0; i < column_families.size(); ++i) {
|
|
|
|
cfh_iter_pairs_.emplace_back(
|
|
|
|
column_families[i], std::unique_ptr<Iterator>(child_iterators[i]));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
~MultiCfIterator() override { status_.PermitUncheckedError(); }
|
|
|
|
|
|
|
|
// No copy allowed
|
|
|
|
MultiCfIterator(const MultiCfIterator&) = delete;
|
|
|
|
MultiCfIterator& operator=(const MultiCfIterator&) = delete;
|
|
|
|
|
|
|
|
private:
|
|
|
|
std::vector<std::pair<ColumnFamilyHandle*, std::unique_ptr<Iterator>>>
|
|
|
|
cfh_iter_pairs_;
|
|
|
|
ReadOptions read_options_;
|
|
|
|
Status status_;
|
|
|
|
|
|
|
|
AttributeGroups attribute_groups_;
|
|
|
|
|
|
|
|
struct MultiCfIteratorInfo {
|
|
|
|
Iterator* iterator;
|
|
|
|
ColumnFamilyHandle* cfh;
|
|
|
|
int order;
|
|
|
|
};
|
|
|
|
|
2024-03-18 16:05:30 +00:00
|
|
|
template <typename CompareOp>
|
|
|
|
class MultiCfHeapItemComparator {
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
public:
|
2024-03-18 16:05:30 +00:00
|
|
|
explicit MultiCfHeapItemComparator(const Comparator* comparator)
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
: comparator_(comparator) {}
|
|
|
|
bool operator()(const MultiCfIteratorInfo& a,
|
|
|
|
const MultiCfIteratorInfo& b) const {
|
|
|
|
assert(a.iterator);
|
|
|
|
assert(b.iterator);
|
|
|
|
assert(a.iterator->Valid());
|
|
|
|
assert(b.iterator->Valid());
|
|
|
|
int c = comparator_->Compare(a.iterator->key(), b.iterator->key());
|
|
|
|
assert(c != 0 || a.order != b.order);
|
2024-03-18 16:05:30 +00:00
|
|
|
return c == 0 ? a.order - b.order > 0 : CompareOp()(c, 0);
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
}
|
|
|
|
|
|
|
|
private:
|
|
|
|
const Comparator* comparator_;
|
|
|
|
};
|
|
|
|
const Comparator* comparator_;
|
|
|
|
using MultiCfMinHeap =
|
2024-03-18 16:05:30 +00:00
|
|
|
BinaryHeap<MultiCfIteratorInfo,
|
|
|
|
MultiCfHeapItemComparator<std::greater<int>>>;
|
|
|
|
using MultiCfMaxHeap = BinaryHeap<MultiCfIteratorInfo,
|
|
|
|
MultiCfHeapItemComparator<std::less<int>>>;
|
|
|
|
|
|
|
|
using MultiCfIterHeap = std::variant<MultiCfMinHeap, MultiCfMaxHeap>;
|
|
|
|
|
|
|
|
MultiCfIterHeap heap_;
|
|
|
|
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
// TODO: Lower and Upper bounds
|
|
|
|
|
2024-03-18 16:05:30 +00:00
|
|
|
Iterator* current() const {
|
2024-03-22 21:51:16 +00:00
|
|
|
if (std::holds_alternative<MultiCfMaxHeap>(heap_)) {
|
2024-03-18 16:05:30 +00:00
|
|
|
auto& max_heap = std::get<MultiCfMaxHeap>(heap_);
|
|
|
|
return max_heap.top().iterator;
|
|
|
|
}
|
|
|
|
auto& min_heap = std::get<MultiCfMinHeap>(heap_);
|
|
|
|
return min_heap.top().iterator;
|
|
|
|
}
|
|
|
|
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
Slice key() const override {
|
|
|
|
assert(Valid());
|
2024-03-18 16:05:30 +00:00
|
|
|
return current()->key();
|
|
|
|
}
|
|
|
|
Slice value() const override {
|
|
|
|
assert(Valid());
|
|
|
|
return current()->value();
|
|
|
|
}
|
|
|
|
const WideColumns& columns() const override {
|
|
|
|
assert(Valid());
|
|
|
|
return current()->columns();
|
|
|
|
}
|
|
|
|
|
|
|
|
bool Valid() const override {
|
2024-03-22 21:51:16 +00:00
|
|
|
if (std::holds_alternative<MultiCfMaxHeap>(heap_)) {
|
2024-03-18 16:05:30 +00:00
|
|
|
auto& max_heap = std::get<MultiCfMaxHeap>(heap_);
|
|
|
|
return !max_heap.empty() && status_.ok();
|
|
|
|
}
|
|
|
|
auto& min_heap = std::get<MultiCfMinHeap>(heap_);
|
|
|
|
return !min_heap.empty() && status_.ok();
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
}
|
2024-03-18 16:05:30 +00:00
|
|
|
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
Status status() const override { return status_; }
|
|
|
|
void considerStatus(Status s) {
|
|
|
|
if (!s.ok() && status_.ok()) {
|
|
|
|
status_ = std::move(s);
|
|
|
|
}
|
|
|
|
}
|
2024-03-22 21:51:16 +00:00
|
|
|
|
|
|
|
template <typename HeapType, typename InitFunc>
|
|
|
|
HeapType& GetHeap(InitFunc initFunc) {
|
|
|
|
if (!std::holds_alternative<HeapType>(heap_)) {
|
|
|
|
initFunc();
|
|
|
|
}
|
|
|
|
return std::get<HeapType>(heap_);
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
}
|
|
|
|
|
2024-03-18 16:05:30 +00:00
|
|
|
void InitMinHeap() {
|
|
|
|
heap_.emplace<MultiCfMinHeap>(
|
|
|
|
MultiCfHeapItemComparator<std::greater<int>>(comparator_));
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
}
|
2024-03-18 16:05:30 +00:00
|
|
|
void InitMaxHeap() {
|
|
|
|
heap_.emplace<MultiCfMaxHeap>(
|
|
|
|
MultiCfHeapItemComparator<std::less<int>>(comparator_));
|
|
|
|
}
|
|
|
|
|
2024-03-22 21:51:16 +00:00
|
|
|
template <typename BinaryHeap, typename ChildSeekFuncType>
|
|
|
|
void SeekCommon(BinaryHeap& heap, ChildSeekFuncType child_seek_func);
|
|
|
|
template <typename BinaryHeap, typename AdvanceFuncType>
|
|
|
|
void AdvanceIterator(BinaryHeap& heap, AdvanceFuncType advance_func);
|
2024-03-18 16:05:30 +00:00
|
|
|
|
|
|
|
void SeekToFirst() override;
|
|
|
|
void SeekToLast() override;
|
|
|
|
void Seek(const Slice& /*target*/) override;
|
|
|
|
void SeekForPrev(const Slice& /*target*/) override;
|
|
|
|
void Next() override;
|
|
|
|
void Prev() override;
|
Introduce MultiCfIterator (#12153)
Summary:
This PR introduces a new implementation of `Iterator` via a new public API called `NewMultiCfIterator()`. The new API takes a vector of column family handles to build a cross-column-family iterator, which internally maintains multiple `DBIter`s as child iterators from a consistent database state. When a key exists in multiple column families, the iterator selects the value (and wide columns) from the first column family containing the key, following the order provided in the `column_families` parameter. Similar to the merging iterator, a min heap is used to iterate across the child iterators. Backward iteration and direction change functionalities will be implemented in future PRs.
The comparator used to compare keys across different column families will be derived from the iterator of the first column family specified in `column_families`. This comparator will be checked against the comparators from all other column families that the iterator will traverse. If there's a mismatch with any of the comparators, the initialization of the iterator will fail.
Please note that this PR is not enough for users to start using `MultiCfIterator`. The `MultiCfIterator` and related APIs are still marked as "**DO NOT USE - UNDER CONSTRUCTION**". This PR is just the first of many PRs that will follow soon.
This PR includes the following:
- Introduction and partial implementation of the `MultiCfIterator`, which implements the generic `Iterator` interface. The implementation includes the construction of the iterator, `SeekToFirst()`, `Next()`, `Valid()`, `key()`, `value()`, and `columns()`.
- Unit tests to verify iteration across multiple column families in two distinct scenarios: (1) keys are unique across all column families, and (2) the same keys exist in multiple column families.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/12153
Reviewed By: pdillinger
Differential Revision: D52308697
Pulled By: jaykorean
fbshipit-source-id: b03e69f13b40af5a8f0598d0f43a0bec01ef8294
2024-03-05 18:22:43 +00:00
|
|
|
};
|
|
|
|
|
|
|
|
} // namespace ROCKSDB_NAMESPACE
|