rocksdb/utilities/transactions/optimistic_transaction_db_impl.h
Peter Dillinger 17bc27741f Improve memory efficiency of many OptimisticTransactionDBs (#11439)
Summary:
Currently it's easy to use a ton of memory with many small OptimisticTransactionDB instances, because each one by default allocates a million mutexes (40 bytes each on my compiler) for validating transactions. It even puts a lot of pressure on the allocator by allocating each one individually!

In this change:
* Create a new object and option that enables sharing these buckets of mutexes between instances. This is generally good for load balancing potential contention as various DBs become hotter or colder with txn writes. About the only cases where this sharing wouldn't make sense (e.g. each DB usually written by one thread) are cases that would be better off with OccValidationPolicy::kValidateSerial which doesn't use the buckets anyway.
* Allocate the mutexes in a contiguous array, for efficiency
* Add an option to ensure the mutexes are cache-aligned. In several other places we use cache-aligned mutexes but OptimisticTransactionDB historically does not. It should be a space-time trade-off the user can choose.
* Provide some visibility into the memory used by the mutex buckets with an ApproximateMemoryUsage() function (also used in unit testing)
* Share code with other users of "striped" mutexes, appropriate refactoring for customization & efficiency (e.g. using FastRange instead of modulus)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11439

Test Plan: unit tests added. Ran sized-up versions of stress test in unit test, including a before-and-after performance test showing no consistent difference. (NOTE: OptimisticTransactionDB not currently covered by db_stress!)

Reviewed By: ltamasi

Differential Revision: D45796393

Pulled By: pdillinger

fbshipit-source-id: ae2b3a26ad91ceeec15debcdc63ff48df6736a54
2023-05-24 11:57:15 -07:00

112 lines
3.7 KiB
C++

// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
// This source code is licensed under both the GPLv2 (found in the
// COPYING file in the root directory) and Apache 2.0 License
// (found in the LICENSE.Apache file in the root directory).
#pragma once
#include <algorithm>
#include <cstdint>
#include <memory>
#include <vector>
#include "rocksdb/db.h"
#include "rocksdb/options.h"
#include "rocksdb/utilities/optimistic_transaction_db.h"
#include "util/cast_util.h"
#include "util/mutexlock.h"
namespace ROCKSDB_NAMESPACE {
class OccLockBucketsImplBase : public OccLockBuckets {
public:
virtual port::Mutex& GetLockBucket(const Slice& key, uint64_t seed) = 0;
};
template <bool cache_aligned>
class OccLockBucketsImpl : public OccLockBucketsImplBase {
public:
explicit OccLockBucketsImpl(size_t bucket_count) : locks_(bucket_count) {}
port::Mutex& GetLockBucket(const Slice& key, uint64_t seed) override {
return locks_.Get(key, seed);
}
size_t ApproximateMemoryUsage() const override {
return locks_.ApproximateMemoryUsage();
}
private:
// TODO: investigate optionally using folly::MicroLock to majorly save space
using M = std::conditional_t<cache_aligned, CacheAlignedWrapper<port::Mutex>,
port::Mutex>;
Striped<M> locks_;
};
class OptimisticTransactionDBImpl : public OptimisticTransactionDB {
public:
explicit OptimisticTransactionDBImpl(
DB* db, const OptimisticTransactionDBOptions& occ_options,
bool take_ownership = true)
: OptimisticTransactionDB(db),
db_owner_(take_ownership),
validate_policy_(occ_options.validate_policy) {
if (validate_policy_ == OccValidationPolicy::kValidateParallel) {
auto bucketed_locks = occ_options.shared_lock_buckets;
if (!bucketed_locks) {
uint32_t bucket_count = std::max(16u, occ_options.occ_lock_buckets);
bucketed_locks = MakeSharedOccLockBuckets(bucket_count);
}
bucketed_locks_ = static_cast_with_check<OccLockBucketsImplBase>(
std::move(bucketed_locks));
}
}
~OptimisticTransactionDBImpl() {
// Prevent this stackable from destroying
// base db
if (!db_owner_) {
db_ = nullptr;
}
}
Transaction* BeginTransaction(const WriteOptions& write_options,
const OptimisticTransactionOptions& txn_options,
Transaction* old_txn) override;
// Transactional `DeleteRange()` is not yet supported.
using StackableDB::DeleteRange;
virtual Status DeleteRange(const WriteOptions&, ColumnFamilyHandle*,
const Slice&, const Slice&) override {
return Status::NotSupported();
}
// Range deletions also must not be snuck into `WriteBatch`es as they are
// incompatible with `OptimisticTransactionDB`.
virtual Status Write(const WriteOptions& write_opts,
WriteBatch* batch) override {
if (batch->HasDeleteRange()) {
return Status::NotSupported();
}
return OptimisticTransactionDB::Write(write_opts, batch);
}
OccValidationPolicy GetValidatePolicy() const { return validate_policy_; }
port::Mutex& GetLockBucket(const Slice& key, uint64_t seed) {
return bucketed_locks_->GetLockBucket(key, seed);
}
private:
std::shared_ptr<OccLockBucketsImplBase> bucketed_locks_;
bool db_owner_;
const OccValidationPolicy validate_policy_;
void ReinitializeTransaction(Transaction* txn,
const WriteOptions& write_options,
const OptimisticTransactionOptions& txn_options =
OptimisticTransactionOptions());
};
} // namespace ROCKSDB_NAMESPACE