mirror of
https://github.com/facebook/rocksdb.git
synced 2024-11-30 04:41:49 +00:00
c00c16855d
Summary: In the current implementation of iterators, `DBImpl*` and `ColumnFamilyData*` are held in `DBIter` and `ArenaWrappedDBIter` for two purposes: tracing and Refresh() API. With the introduction of a new iterator called MultiCfIterator in PR https://github.com/facebook/rocksdb/issues/12153 , which is a cross-column-family iterator that maintains multiple DBIters as child iterators from a consistent database state, we need to make some changes to the existing implementation. The new iterator will still be exposed through the generic Iterator interface with an additional capability to return AttributeGroups (via `attribute_groups()`) which is a list of wide columns grouped by column family. For more information about AttributeGroup, please refer to previous PRs: https://github.com/facebook/rocksdb/issues/11925 #11943, and https://github.com/facebook/rocksdb/issues/11977. To be able to return AttributeGroup in the default single CF iterator created, access to `ColumnFamilyHandle*` within `DBIter` is necessary. However, this is not currently available in `DBIter`. Since `DBImpl*` and `ColumnFamilyData*` can be easily accessed via `ColumnFamilyHandleImpl*`, we have decided to replace the pointers to `ColumnFamilyData` and `DBImpl` in `DBIter` with a pointer to `ColumnFamilyHandleImpl`. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12395 Test Plan: # Summary In the current implementation of iterators, `DBImpl*` and `ColumnFamilyData*` are held in `DBIter` and `ArenaWrappedDBIter` for two purposes: tracing and Refresh() API. With the introduction of a new iterator called MultiCfIterator in PR #12153 , which is a cross-column-family iterator that maintains multiple DBIters as child iterators from a consistent database state, we need to make some changes to the existing implementation. The new iterator will still be exposed through the generic Iterator interface with an additional capability to return AttributeGroups (via `attribute_groups()`) which is a list of wide columns grouped by column family. For more information about AttributeGroup, please refer to previous PRs: #11925 #11943, and #11977. To be able to return AttributeGroup in the default single CF iterator created, access to `ColumnFamilyHandle*` within `DBIter` is necessary. However, this is not currently available in `DBIter`. Since `DBImpl*` and `ColumnFamilyData*` can be easily accessed via `ColumnFamilyHandleImpl*`, we have decided to replace the pointers to `ColumnFamilyData` and `DBImpl` in `DBIter` with a pointer to `ColumnFamilyHandleImpl`. # Test Plan There should be no behavior changes. Existing tests and CI for the correctness tests. **Test for Perf Regression** Build ``` $> make -j64 release ``` Setup ``` $> TEST_TMPDIR=/dev/shm/db_bench ./db_bench -benchmarks="filluniquerandom" -key_size=32 -value_size=512 -num=1000000 -compression_type=none ``` Run ``` TEST_TMPDIR=/dev/shm/db_bench ./db_bench -use_existing_db=1 -benchmarks="newiterator,seekrandom" -cache_size=10485760000 ``` Before the change ``` DB path: [/dev/shm/db_bench/dbbench] newiterator : 0.552 micros/op 1810157 ops/sec 0.552 seconds 1000000 operations; DB path: [/dev/shm/db_bench/dbbench] seekrandom : 4.502 micros/op 222143 ops/sec 4.502 seconds 1000000 operations; (0 of 1000000 found) ``` After the change ``` DB path: [/dev/shm/db_bench/dbbench] newiterator : 0.520 micros/op 1924401 ops/sec 0.520 seconds 1000000 operations; DB path: [/dev/shm/db_bench/dbbench] seekrandom : 4.532 micros/op 220657 ops/sec 4.532 seconds 1000000 operations; (0 of 1000000 found) ``` Reviewed By: pdillinger Differential Revision: D54332713 Pulled By: jaykorean fbshipit-source-id: b28d897ad519e58b1ca82eb068a6319544a4fae5
127 lines
4.8 KiB
C++
127 lines
4.8 KiB
C++
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
// (found in the LICENSE.Apache file in the root directory).
|
|
//
|
|
// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
|
|
// Use of this source code is governed by a BSD-style license that can be
|
|
// found in the LICENSE file. See the AUTHORS file for names of contributors.
|
|
|
|
#pragma once
|
|
#include <stdint.h>
|
|
|
|
#include <string>
|
|
|
|
#include "db/db_impl/db_impl.h"
|
|
#include "db/db_iter.h"
|
|
#include "db/range_del_aggregator.h"
|
|
#include "memory/arena.h"
|
|
#include "options/cf_options.h"
|
|
#include "rocksdb/db.h"
|
|
#include "rocksdb/iterator.h"
|
|
#include "util/autovector.h"
|
|
|
|
namespace ROCKSDB_NAMESPACE {
|
|
|
|
class Arena;
|
|
class Version;
|
|
|
|
// A wrapper iterator which wraps DB Iterator and the arena, with which the DB
|
|
// iterator is supposed to be allocated. This class is used as an entry point of
|
|
// a iterator hierarchy whose memory can be allocated inline. In that way,
|
|
// accessing the iterator tree can be more cache friendly. It is also faster
|
|
// to allocate.
|
|
// When using the class's Iterator interface, the behavior is exactly
|
|
// the same as the inner DBIter.
|
|
class ArenaWrappedDBIter : public Iterator {
|
|
public:
|
|
~ArenaWrappedDBIter() override {
|
|
if (db_iter_ != nullptr) {
|
|
db_iter_->~DBIter();
|
|
} else {
|
|
assert(false);
|
|
}
|
|
}
|
|
|
|
// Get the arena to be used to allocate memory for DBIter to be wrapped,
|
|
// as well as child iterators in it.
|
|
virtual Arena* GetArena() { return &arena_; }
|
|
|
|
const ReadOptions& GetReadOptions() { return read_options_; }
|
|
|
|
// Set the internal iterator wrapped inside the DB Iterator. Usually it is
|
|
// a merging iterator.
|
|
virtual void SetIterUnderDBIter(InternalIterator* iter) {
|
|
db_iter_->SetIter(iter);
|
|
}
|
|
|
|
void SetMemtableRangetombstoneIter(TruncatedRangeDelIterator** iter) {
|
|
memtable_range_tombstone_iter_ = iter;
|
|
}
|
|
|
|
bool Valid() const override { return db_iter_->Valid(); }
|
|
void SeekToFirst() override { db_iter_->SeekToFirst(); }
|
|
void SeekToLast() override { db_iter_->SeekToLast(); }
|
|
// 'target' does not contain timestamp, even if user timestamp feature is
|
|
// enabled.
|
|
void Seek(const Slice& target) override { db_iter_->Seek(target); }
|
|
void SeekForPrev(const Slice& target) override {
|
|
db_iter_->SeekForPrev(target);
|
|
}
|
|
void Next() override { db_iter_->Next(); }
|
|
void Prev() override { db_iter_->Prev(); }
|
|
Slice key() const override { return db_iter_->key(); }
|
|
Slice value() const override { return db_iter_->value(); }
|
|
const WideColumns& columns() const override { return db_iter_->columns(); }
|
|
Status status() const override { return db_iter_->status(); }
|
|
Slice timestamp() const override { return db_iter_->timestamp(); }
|
|
bool IsBlob() const { return db_iter_->IsBlob(); }
|
|
|
|
Status GetProperty(std::string prop_name, std::string* prop) override;
|
|
|
|
Status Refresh() override;
|
|
Status Refresh(const Snapshot*) override;
|
|
|
|
void Init(Env* env, const ReadOptions& read_options,
|
|
const ImmutableOptions& ioptions,
|
|
const MutableCFOptions& mutable_cf_options, const Version* version,
|
|
const SequenceNumber& sequence,
|
|
uint64_t max_sequential_skip_in_iterations, uint64_t version_number,
|
|
ReadCallback* read_callback, ColumnFamilyHandleImpl* cfh,
|
|
bool expose_blob_index, bool allow_refresh);
|
|
|
|
// Store some parameters so we can refresh the iterator at a later point
|
|
// with these same params
|
|
void StoreRefreshInfo(ColumnFamilyHandleImpl* cfh,
|
|
ReadCallback* read_callback, bool expose_blob_index) {
|
|
cfh_ = cfh;
|
|
read_callback_ = read_callback;
|
|
expose_blob_index_ = expose_blob_index;
|
|
}
|
|
|
|
private:
|
|
DBIter* db_iter_ = nullptr;
|
|
Arena arena_;
|
|
uint64_t sv_number_;
|
|
ColumnFamilyHandleImpl* cfh_ = nullptr;
|
|
ReadOptions read_options_;
|
|
ReadCallback* read_callback_;
|
|
bool expose_blob_index_ = false;
|
|
bool allow_refresh_ = true;
|
|
// If this is nullptr, it means the mutable memtable does not contain range
|
|
// tombstone when added under this DBIter.
|
|
TruncatedRangeDelIterator** memtable_range_tombstone_iter_ = nullptr;
|
|
};
|
|
|
|
// Generate the arena wrapped iterator class.
|
|
// `cfh` is used for reneweal. If left null, renewal will not
|
|
// be supported.
|
|
ArenaWrappedDBIter* NewArenaWrappedDbIterator(
|
|
Env* env, const ReadOptions& read_options, const ImmutableOptions& ioptions,
|
|
const MutableCFOptions& mutable_cf_options, const Version* version,
|
|
const SequenceNumber& sequence, uint64_t max_sequential_skip_in_iterations,
|
|
uint64_t version_number, ReadCallback* read_callback,
|
|
ColumnFamilyHandleImpl* cfh = nullptr, bool expose_blob_index = false,
|
|
bool allow_refresh = true);
|
|
} // namespace ROCKSDB_NAMESPACE
|