2016-10-21 00:05:32 +00:00
|
|
|
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.
|
2017-07-15 23:03:42 +00:00
|
|
|
// This source code is licensed under both the GPLv2 (found in the
|
|
|
|
// COPYING file in the root directory) and Apache 2.0 License
|
|
|
|
// (found in the LICENSE.Apache file in the root directory).
|
2016-10-21 00:05:32 +00:00
|
|
|
|
2017-02-28 19:05:08 +00:00
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
#include "db/external_sst_file_ingestion_job.h"
|
|
|
|
|
|
|
|
#include <algorithm>
|
2019-06-21 17:12:29 +00:00
|
|
|
#include <cinttypes>
|
2016-10-21 00:05:32 +00:00
|
|
|
#include <string>
|
2019-06-21 17:12:29 +00:00
|
|
|
#include <unordered_set>
|
2016-10-21 00:05:32 +00:00
|
|
|
#include <vector>
|
|
|
|
|
2019-06-21 17:12:29 +00:00
|
|
|
#include "db/db_impl/db_impl.h"
|
2016-10-21 00:05:32 +00:00
|
|
|
#include "db/version_edit.h"
|
2019-05-30 03:44:08 +00:00
|
|
|
#include "file/file_util.h"
|
2019-09-16 17:31:27 +00:00
|
|
|
#include "file/random_access_file_reader.h"
|
2021-09-29 11:01:57 +00:00
|
|
|
#include "logging/logging.h"
|
2017-02-03 00:38:40 +00:00
|
|
|
#include "table/merging_iterator.h"
|
2016-10-21 00:05:32 +00:00
|
|
|
#include "table/scoped_arena_iterator.h"
|
|
|
|
#include "table/sst_file_writer_collectors.h"
|
|
|
|
#include "table/table_builder.h"
|
2022-05-19 18:04:21 +00:00
|
|
|
#include "table/unique_id_impl.h"
|
2019-05-31 00:39:43 +00:00
|
|
|
#include "test_util/sync_point.h"
|
2016-10-21 00:05:32 +00:00
|
|
|
#include "util/stop_watch.h"
|
|
|
|
|
2020-02-20 20:07:53 +00:00
|
|
|
namespace ROCKSDB_NAMESPACE {
|
2016-10-21 00:05:32 +00:00
|
|
|
|
|
|
|
Status ExternalSstFileIngestionJob::Prepare(
|
2018-07-27 21:02:07 +00:00
|
|
|
const std::vector<std::string>& external_files_paths,
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
const std::vector<std::string>& files_checksums,
|
|
|
|
const std::vector<std::string>& files_checksum_func_names,
|
2021-10-08 17:29:06 +00:00
|
|
|
const Temperature& file_temperature, uint64_t next_file_number,
|
|
|
|
SuperVersion* sv) {
|
2016-10-21 00:05:32 +00:00
|
|
|
Status status;
|
|
|
|
|
|
|
|
// Read the information of files we are ingesting
|
|
|
|
for (const std::string& file_path : external_files_paths) {
|
|
|
|
IngestedFileInfo file_to_ingest;
|
New stable, fixed-length cache keys (#9126)
Summary:
This change standardizes on a new 16-byte cache key format for
block cache (incl compressed and secondary) and persistent cache (but
not table cache and row cache).
The goal is a really fast cache key with practically ideal stability and
uniqueness properties without external dependencies (e.g. from FileSystem).
A fixed key size of 16 bytes should enable future optimizations to the
concurrent hash table for block cache, which is a heavy CPU user /
bottleneck, but there appears to be measurable performance improvement
even with no changes to LRUCache.
This change replaces a lot of disjointed and ugly code handling cache
keys with calls to a simple, clean new internal API (cache_key.h).
(Preserving the old cache key logic under an option would be very ugly
and likely negate the performance gain of the new approach. Complete
replacement carries some inherent risk, but I think that's acceptable
with sufficient analysis and testing.)
The scheme for encoding new cache keys is complicated but explained
in cache_key.cc.
Also: EndianSwapValue is moved to math.h to be next to other bit
operations. (Explains some new include "math.h".) ReverseBits operation
added and unit tests added to hash_test for both.
Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126
Test Plan:
### Basic correctness
Several tests needed updates to work with the new functionality, mostly
because we are no longer relying on filesystem for stable cache keys
so table builders & readers need more context info to agree on cache
keys. This functionality is so core, a huge number of existing tests
exercise the cache key functionality.
### Performance
Create db with
`TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters`
And test performance with
`TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4`
using DEBUG_LEVEL=0 and simultaneous before & after runs.
Before ops/sec, avg over 100 runs: 121924
After ops/sec, avg over 100 runs: 125385 (+2.8%)
### Collision probability
I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity
over many months, by making some pessimistic simplifying assumptions:
* Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys)
* All of every file is cached for its entire lifetime
We use a simple table with skewed address assignment and replacement on address collision
to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output
with `./cache_bench -stress_cache_key -sck_keep_bits=40`:
```
Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day
Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached)
```
These come from default settings of 2.5M files per day of 32 MB each, and
`-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of
the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation
is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality.
More default assumptions, relatively pessimistic:
* 100 DBs in same process (doesn't matter much)
* Re-open DB in same process (new session ID related to old session ID) on average
every 100 files generated
* Restart process (all new session IDs unrelated to old) 24 times per day
After enough data, we get a result at the end:
```
(keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected)
```
If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data:
```
(keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected)
(keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected)
```
The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases:
```
197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected)
```
I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data.
Reviewed By: zhichao-cao
Differential Revision: D33171746
Pulled By: pdillinger
fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
2021-12-17 01:13:55 +00:00
|
|
|
status =
|
|
|
|
GetIngestedFileInfo(file_path, next_file_number++, &file_to_ingest, sv);
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
2021-04-01 02:12:29 +00:00
|
|
|
if (file_to_ingest.cf_id !=
|
2016-12-05 22:16:23 +00:00
|
|
|
TablePropertiesCollectorFactory::Context::kUnknownColumnFamily &&
|
2021-04-01 02:12:29 +00:00
|
|
|
file_to_ingest.cf_id != cfd_->GetID()) {
|
2016-12-05 22:16:23 +00:00
|
|
|
return Status::InvalidArgument(
|
2020-02-28 22:10:51 +00:00
|
|
|
"External file column family id don't match");
|
2016-12-05 22:16:23 +00:00
|
|
|
}
|
2021-04-01 02:12:29 +00:00
|
|
|
|
|
|
|
if (file_to_ingest.num_entries == 0 &&
|
|
|
|
file_to_ingest.num_range_deletions == 0) {
|
|
|
|
return Status::InvalidArgument("File contain no entries");
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!file_to_ingest.smallest_internal_key.Valid() ||
|
|
|
|
!file_to_ingest.largest_internal_key.Valid()) {
|
|
|
|
return Status::Corruption("Generated table have corrupted keys");
|
|
|
|
}
|
|
|
|
|
|
|
|
files_to_ingest_.emplace_back(std::move(file_to_ingest));
|
2016-12-05 22:16:23 +00:00
|
|
|
}
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
const Comparator* ucmp = cfd_->internal_comparator().user_comparator();
|
|
|
|
auto num_files = files_to_ingest_.size();
|
|
|
|
if (num_files == 0) {
|
|
|
|
return Status::InvalidArgument("The list of files is empty");
|
|
|
|
} else if (num_files > 1) {
|
2020-02-28 22:10:51 +00:00
|
|
|
// Verify that passed files don't have overlapping ranges
|
2016-10-21 00:05:32 +00:00
|
|
|
autovector<const IngestedFileInfo*> sorted_files;
|
|
|
|
for (size_t i = 0; i < num_files; i++) {
|
|
|
|
sorted_files.push_back(&files_to_ingest_[i]);
|
|
|
|
}
|
|
|
|
|
|
|
|
std::sort(
|
|
|
|
sorted_files.begin(), sorted_files.end(),
|
|
|
|
[&ucmp](const IngestedFileInfo* info1, const IngestedFileInfo* info2) {
|
2019-08-15 03:58:59 +00:00
|
|
|
return sstableKeyCompare(ucmp, info1->smallest_internal_key,
|
|
|
|
info2->smallest_internal_key) < 0;
|
2016-10-21 00:05:32 +00:00
|
|
|
});
|
|
|
|
|
2020-06-02 22:02:44 +00:00
|
|
|
for (size_t i = 0; i + 1 < num_files; i++) {
|
2019-08-15 03:58:59 +00:00
|
|
|
if (sstableKeyCompare(ucmp, sorted_files[i]->largest_internal_key,
|
|
|
|
sorted_files[i + 1]->smallest_internal_key) >= 0) {
|
2019-09-13 21:48:18 +00:00
|
|
|
files_overlap_ = true;
|
|
|
|
break;
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2021-10-08 17:29:06 +00:00
|
|
|
// Hanlde the file temperature
|
|
|
|
for (size_t i = 0; i < num_files; i++) {
|
|
|
|
files_to_ingest_[i].file_temperature = file_temperature;
|
|
|
|
}
|
|
|
|
|
2019-09-13 21:48:18 +00:00
|
|
|
if (ingestion_options_.ingest_behind && files_overlap_) {
|
|
|
|
return Status::NotSupported("Files have overlapping ranges");
|
|
|
|
}
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
// Copy/Move external files into DB
|
2019-06-21 17:12:29 +00:00
|
|
|
std::unordered_set<size_t> ingestion_path_ids;
|
2016-10-21 00:05:32 +00:00
|
|
|
for (IngestedFileInfo& f : files_to_ingest_) {
|
2019-05-24 04:54:23 +00:00
|
|
|
f.copy_file = false;
|
2016-10-21 00:05:32 +00:00
|
|
|
const std::string path_outside_db = f.external_file_path;
|
2022-11-02 21:34:24 +00:00
|
|
|
const std::string path_inside_db = TableFileName(
|
|
|
|
cfd_->ioptions()->cf_paths, f.fd.GetNumber(), f.fd.GetPathId());
|
2016-10-21 00:05:32 +00:00
|
|
|
if (ingestion_options_.move_files) {
|
Introduce a new storage specific Env API (#5761)
Summary:
The current Env API encompasses both storage/file operations, as well as OS related operations. Most of the APIs return a Status, which does not have enough metadata about an error, such as whether its retry-able or not, scope (i.e fault domain) of the error etc., that may be required in order to properly handle a storage error. The file APIs also do not provide enough control over the IO SLA, such as timeout, prioritization, hinting about placement and redundancy etc.
This PR separates out the file/storage APIs from Env into a new FileSystem class. The APIs are updated to return an IOStatus with metadata about the error, as well as to take an IOOptions structure as input in order to allow more control over the IO.
The user can set both ```options.env``` and ```options.file_system``` to specify that RocksDB should use the former for OS related operations and the latter for storage operations. Internally, a ```CompositeEnvWrapper``` has been introduced that inherits from ```Env``` and redirects individual methods to either an ```Env``` implementation or the ```FileSystem``` as appropriate. When options are sanitized during ```DB::Open```, ```options.env``` is replaced with a newly allocated ```CompositeEnvWrapper``` instance if both env and file_system have been specified. This way, the rest of the RocksDB code can continue to function as before.
This PR also ports PosixEnv to the new API by splitting it into two - PosixEnv and PosixFileSystem. PosixEnv is defined as a sub-class of CompositeEnvWrapper, and threading/time functions are overridden with Posix specific implementations in order to avoid an extra level of indirection.
The ```CompositeEnvWrapper``` translates ```IOStatus``` return code to ```Status```, and sets the severity to ```kSoftError``` if the io_status is retryable. The error handling code in RocksDB can then recover the DB automatically.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5761
Differential Revision: D18868376
Pulled By: anand1976
fbshipit-source-id: 39efe18a162ea746fabac6360ff529baba48486f
2019-12-13 22:47:08 +00:00
|
|
|
status =
|
|
|
|
fs_->LinkFile(path_outside_db, path_inside_db, IOOptions(), nullptr);
|
2019-06-21 17:12:29 +00:00
|
|
|
if (status.ok()) {
|
|
|
|
// It is unsafe to assume application had sync the file and file
|
|
|
|
// directory before ingest the file. For integrity of RocksDB we need
|
|
|
|
// to sync the file.
|
Introduce a new storage specific Env API (#5761)
Summary:
The current Env API encompasses both storage/file operations, as well as OS related operations. Most of the APIs return a Status, which does not have enough metadata about an error, such as whether its retry-able or not, scope (i.e fault domain) of the error etc., that may be required in order to properly handle a storage error. The file APIs also do not provide enough control over the IO SLA, such as timeout, prioritization, hinting about placement and redundancy etc.
This PR separates out the file/storage APIs from Env into a new FileSystem class. The APIs are updated to return an IOStatus with metadata about the error, as well as to take an IOOptions structure as input in order to allow more control over the IO.
The user can set both ```options.env``` and ```options.file_system``` to specify that RocksDB should use the former for OS related operations and the latter for storage operations. Internally, a ```CompositeEnvWrapper``` has been introduced that inherits from ```Env``` and redirects individual methods to either an ```Env``` implementation or the ```FileSystem``` as appropriate. When options are sanitized during ```DB::Open```, ```options.env``` is replaced with a newly allocated ```CompositeEnvWrapper``` instance if both env and file_system have been specified. This way, the rest of the RocksDB code can continue to function as before.
This PR also ports PosixEnv to the new API by splitting it into two - PosixEnv and PosixFileSystem. PosixEnv is defined as a sub-class of CompositeEnvWrapper, and threading/time functions are overridden with Posix specific implementations in order to avoid an extra level of indirection.
The ```CompositeEnvWrapper``` translates ```IOStatus``` return code to ```Status```, and sets the severity to ```kSoftError``` if the io_status is retryable. The error handling code in RocksDB can then recover the DB automatically.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5761
Differential Revision: D18868376
Pulled By: anand1976
fbshipit-source-id: 39efe18a162ea746fabac6360ff529baba48486f
2019-12-13 22:47:08 +00:00
|
|
|
std::unique_ptr<FSWritableFile> file_to_sync;
|
2021-05-19 02:32:55 +00:00
|
|
|
Status s = fs_->ReopenWritableFile(path_inside_db, env_options_,
|
|
|
|
&file_to_sync, nullptr);
|
|
|
|
TEST_SYNC_POINT_CALLBACK("ExternalSstFileIngestionJob::Prepare:Reopen",
|
|
|
|
&s);
|
|
|
|
// Some file systems (especially remote/distributed) don't support
|
|
|
|
// reopening a file for writing and don't require reopening and
|
|
|
|
// syncing the file. Ignore the NotSupported error in that case.
|
|
|
|
if (!s.IsNotSupported()) {
|
|
|
|
status = s;
|
|
|
|
if (status.ok()) {
|
|
|
|
TEST_SYNC_POINT(
|
|
|
|
"ExternalSstFileIngestionJob::BeforeSyncIngestedFile");
|
|
|
|
status = SyncIngestedFile(file_to_sync.get());
|
|
|
|
TEST_SYNC_POINT(
|
|
|
|
"ExternalSstFileIngestionJob::AfterSyncIngestedFile");
|
|
|
|
if (!status.ok()) {
|
|
|
|
ROCKS_LOG_WARN(db_options_.info_log,
|
|
|
|
"Failed to sync ingested file %s: %s",
|
|
|
|
path_inside_db.c_str(), status.ToString().c_str());
|
|
|
|
}
|
2019-06-21 17:12:29 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
} else if (status.IsNotSupported() &&
|
|
|
|
ingestion_options_.failed_move_fall_back_to_copy) {
|
2019-05-24 04:54:23 +00:00
|
|
|
// Original file is on a different FS, use copy instead of hard linking.
|
2018-04-13 17:47:54 +00:00
|
|
|
f.copy_file = true;
|
2022-05-18 18:23:12 +00:00
|
|
|
ROCKS_LOG_INFO(db_options_.info_log,
|
|
|
|
"Triy to link file %s but it's not supported : %s",
|
|
|
|
path_outside_db.c_str(), status.ToString().c_str());
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
} else {
|
2019-05-24 04:54:23 +00:00
|
|
|
f.copy_file = true;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (f.copy_file) {
|
|
|
|
TEST_SYNC_POINT_CALLBACK("ExternalSstFileIngestionJob::Prepare:CopyFile",
|
|
|
|
nullptr);
|
2019-06-21 17:12:29 +00:00
|
|
|
// CopyFile also sync the new file.
|
2022-02-19 02:18:49 +00:00
|
|
|
status =
|
|
|
|
CopyFile(fs_.get(), path_outside_db, path_inside_db, 0,
|
|
|
|
db_options_.use_fsync, io_tracer_, Temperature::kUnknown);
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
2018-04-13 17:47:54 +00:00
|
|
|
TEST_SYNC_POINT("ExternalSstFileIngestionJob::Prepare:FileAdded");
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
f.internal_file_path = path_inside_db;
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
// Initialize the checksum information of ingested files.
|
|
|
|
f.file_checksum = kUnknownFileChecksum;
|
|
|
|
f.file_checksum_func_name = kUnknownFileChecksumFuncName;
|
2019-06-21 17:12:29 +00:00
|
|
|
ingestion_path_ids.insert(f.fd.GetPathId());
|
|
|
|
}
|
|
|
|
|
|
|
|
TEST_SYNC_POINT("ExternalSstFileIngestionJob::BeforeSyncDir");
|
|
|
|
if (status.ok()) {
|
|
|
|
for (auto path_id : ingestion_path_ids) {
|
2021-11-03 19:20:19 +00:00
|
|
|
status = directories_->GetDataDir(path_id)->FsyncWithDirOptions(
|
|
|
|
IOOptions(), nullptr,
|
|
|
|
DirFsyncOptions(DirFsyncOptions::FsyncReason::kNewFileSynced));
|
2019-06-21 17:12:29 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
ROCKS_LOG_WARN(db_options_.info_log,
|
|
|
|
"Failed to sync directory %" ROCKSDB_PRIszt
|
|
|
|
" while ingest file: %s",
|
|
|
|
path_id, status.ToString().c_str());
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
2019-06-21 17:12:29 +00:00
|
|
|
TEST_SYNC_POINT("ExternalSstFileIngestionJob::AfterSyncDir");
|
2016-10-21 00:05:32 +00:00
|
|
|
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
// Generate and check the sst file checksum. Note that, if
|
|
|
|
// IngestExternalFileOptions::write_global_seqno is true, we will not update
|
|
|
|
// the checksum information in the files_to_ingests_ here, since the file is
|
|
|
|
// upadted with the new global_seqno. After global_seqno is updated, DB will
|
|
|
|
// generate the new checksum and store it in the Manifest. In all other cases
|
|
|
|
// if ingestion_options_.write_global_seqno == true and
|
|
|
|
// verify_file_checksum is false, we only check the checksum function name.
|
|
|
|
if (status.ok() && db_options_.file_checksum_gen_factory != nullptr) {
|
|
|
|
if (ingestion_options_.verify_file_checksum == false &&
|
|
|
|
files_checksums.size() == files_to_ingest_.size() &&
|
|
|
|
files_checksum_func_names.size() == files_to_ingest_.size()) {
|
|
|
|
// Only when verify_file_checksum == false and the checksum for ingested
|
|
|
|
// files are provided, DB will use the provided checksum and does not
|
|
|
|
// generate the checksum for ingested files.
|
|
|
|
need_generate_file_checksum_ = false;
|
|
|
|
} else {
|
|
|
|
need_generate_file_checksum_ = true;
|
|
|
|
}
|
|
|
|
FileChecksumGenContext gen_context;
|
|
|
|
std::unique_ptr<FileChecksumGenerator> file_checksum_gen =
|
|
|
|
db_options_.file_checksum_gen_factory->CreateFileChecksumGenerator(
|
|
|
|
gen_context);
|
|
|
|
std::vector<std::string> generated_checksums;
|
|
|
|
std::vector<std::string> generated_checksum_func_names;
|
|
|
|
// Step 1: generate the checksum for ingested sst file.
|
|
|
|
if (need_generate_file_checksum_) {
|
|
|
|
for (size_t i = 0; i < files_to_ingest_.size(); i++) {
|
2020-10-28 23:46:04 +00:00
|
|
|
std::string generated_checksum;
|
|
|
|
std::string generated_checksum_func_name;
|
|
|
|
std::string requested_checksum_func_name;
|
2022-02-17 07:17:03 +00:00
|
|
|
// TODO: rate limit file reads for checksum calculation during file
|
|
|
|
// ingestion.
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
IOStatus io_s = GenerateOneFileChecksum(
|
2020-08-13 00:28:10 +00:00
|
|
|
fs_.get(), files_to_ingest_[i].internal_file_path,
|
2020-10-28 23:46:04 +00:00
|
|
|
db_options_.file_checksum_gen_factory.get(),
|
|
|
|
requested_checksum_func_name, &generated_checksum,
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
&generated_checksum_func_name,
|
|
|
|
ingestion_options_.verify_checksums_readahead_size,
|
2020-12-27 06:05:42 +00:00
|
|
|
db_options_.allow_mmap_reads, io_tracer_,
|
2022-02-17 07:17:03 +00:00
|
|
|
db_options_.rate_limiter.get(),
|
|
|
|
Env::IO_TOTAL /* rate_limiter_priority */);
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
if (!io_s.ok()) {
|
|
|
|
status = io_s;
|
|
|
|
ROCKS_LOG_WARN(db_options_.info_log,
|
|
|
|
"Sst file checksum generation of file: %s failed: %s",
|
|
|
|
files_to_ingest_[i].internal_file_path.c_str(),
|
|
|
|
status.ToString().c_str());
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
if (ingestion_options_.write_global_seqno == false) {
|
|
|
|
files_to_ingest_[i].file_checksum = generated_checksum;
|
|
|
|
files_to_ingest_[i].file_checksum_func_name =
|
|
|
|
generated_checksum_func_name;
|
|
|
|
}
|
|
|
|
generated_checksums.push_back(generated_checksum);
|
|
|
|
generated_checksum_func_names.push_back(generated_checksum_func_name);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
// Step 2: based on the verify_file_checksum and ingested checksum
|
|
|
|
// information, do the verification.
|
|
|
|
if (status.ok()) {
|
|
|
|
if (files_checksums.size() == files_to_ingest_.size() &&
|
|
|
|
files_checksum_func_names.size() == files_to_ingest_.size()) {
|
|
|
|
// Verify the checksum and checksum function name.
|
|
|
|
if (ingestion_options_.verify_file_checksum) {
|
|
|
|
for (size_t i = 0; i < files_to_ingest_.size(); i++) {
|
|
|
|
if (files_checksum_func_names[i] !=
|
|
|
|
generated_checksum_func_names[i]) {
|
|
|
|
status = Status::InvalidArgument(
|
|
|
|
"Checksum function name does not match with the checksum "
|
|
|
|
"function name of this DB");
|
|
|
|
ROCKS_LOG_WARN(
|
|
|
|
db_options_.info_log,
|
|
|
|
"Sst file checksum verification of file: %s failed: %s",
|
|
|
|
external_files_paths[i].c_str(), status.ToString().c_str());
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
if (files_checksums[i] != generated_checksums[i]) {
|
|
|
|
status = Status::Corruption(
|
|
|
|
"Ingested checksum does not match with the generated "
|
|
|
|
"checksum");
|
|
|
|
ROCKS_LOG_WARN(
|
|
|
|
db_options_.info_log,
|
|
|
|
"Sst file checksum verification of file: %s failed: %s",
|
|
|
|
files_to_ingest_[i].internal_file_path.c_str(),
|
|
|
|
status.ToString().c_str());
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
// If verify_file_checksum is not enabled, we only verify the
|
|
|
|
// checksum function name. If it does not match, fail the ingestion.
|
|
|
|
// If matches, we trust the ingested checksum information and store
|
|
|
|
// in the Manifest.
|
|
|
|
for (size_t i = 0; i < files_to_ingest_.size(); i++) {
|
|
|
|
if (files_checksum_func_names[i] != file_checksum_gen->Name()) {
|
|
|
|
status = Status::InvalidArgument(
|
|
|
|
"Checksum function name does not match with the checksum "
|
|
|
|
"function name of this DB");
|
|
|
|
ROCKS_LOG_WARN(
|
|
|
|
db_options_.info_log,
|
|
|
|
"Sst file checksum verification of file: %s failed: %s",
|
|
|
|
external_files_paths[i].c_str(), status.ToString().c_str());
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
files_to_ingest_[i].file_checksum = files_checksums[i];
|
|
|
|
files_to_ingest_[i].file_checksum_func_name =
|
|
|
|
files_checksum_func_names[i];
|
|
|
|
}
|
|
|
|
}
|
|
|
|
} else if (files_checksums.size() != files_checksum_func_names.size() ||
|
|
|
|
(files_checksums.size() == files_checksum_func_names.size() &&
|
|
|
|
files_checksums.size() != 0)) {
|
|
|
|
// The checksum or checksum function name vector are not both empty
|
|
|
|
// and they are incomplete.
|
|
|
|
status = Status::InvalidArgument(
|
|
|
|
"The checksum information of ingested sst files are nonempty and "
|
|
|
|
"the size of checksums or the size of the checksum function "
|
|
|
|
"names "
|
|
|
|
"does not match with the number of ingested sst files");
|
|
|
|
ROCKS_LOG_WARN(
|
|
|
|
db_options_.info_log,
|
|
|
|
"The ingested sst files checksum information is incomplete: %s",
|
|
|
|
status.ToString().c_str());
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2019-06-21 17:12:29 +00:00
|
|
|
// TODO: The following is duplicated with Cleanup().
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!status.ok()) {
|
2021-01-26 06:07:26 +00:00
|
|
|
IOOptions io_opts;
|
2016-10-21 00:05:32 +00:00
|
|
|
// We failed, remove all files that we copied into the db
|
|
|
|
for (IngestedFileInfo& f : files_to_ingest_) {
|
2018-04-13 17:47:54 +00:00
|
|
|
if (f.internal_file_path.empty()) {
|
2019-08-31 01:27:43 +00:00
|
|
|
continue;
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
2021-01-26 06:07:26 +00:00
|
|
|
Status s = fs_->DeleteFile(f.internal_file_path, io_opts, nullptr);
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!s.ok()) {
|
2017-03-16 02:22:52 +00:00
|
|
|
ROCKS_LOG_WARN(db_options_.info_log,
|
|
|
|
"AddFile() clean up for file %s failed : %s",
|
|
|
|
f.internal_file_path.c_str(), s.ToString().c_str());
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
2018-02-28 01:08:34 +00:00
|
|
|
Status ExternalSstFileIngestionJob::NeedsFlush(bool* flush_needed,
|
|
|
|
SuperVersion* super_version) {
|
|
|
|
autovector<Range> ranges;
|
2022-09-30 23:13:03 +00:00
|
|
|
autovector<std::string> keys;
|
|
|
|
size_t ts_sz = cfd_->user_comparator()->timestamp_size();
|
|
|
|
if (ts_sz) {
|
|
|
|
// Check all ranges [begin, end] inclusively. Add maximum
|
|
|
|
// timestamp to include all `begin` keys, and add minimal timestamp to
|
|
|
|
// include all `end` keys.
|
|
|
|
for (const IngestedFileInfo& file_to_ingest : files_to_ingest_) {
|
|
|
|
std::string begin_str;
|
|
|
|
std::string end_str;
|
|
|
|
AppendUserKeyWithMaxTimestamp(
|
|
|
|
&begin_str, file_to_ingest.smallest_internal_key.user_key(), ts_sz);
|
|
|
|
AppendKeyWithMinTimestamp(
|
|
|
|
&end_str, file_to_ingest.largest_internal_key.user_key(), ts_sz);
|
|
|
|
keys.emplace_back(std::move(begin_str));
|
|
|
|
keys.emplace_back(std::move(end_str));
|
|
|
|
}
|
|
|
|
for (size_t i = 0; i < files_to_ingest_.size(); ++i) {
|
|
|
|
ranges.emplace_back(keys[2 * i], keys[2 * i + 1]);
|
|
|
|
}
|
|
|
|
} else {
|
|
|
|
for (const IngestedFileInfo& file_to_ingest : files_to_ingest_) {
|
|
|
|
ranges.emplace_back(file_to_ingest.smallest_internal_key.user_key(),
|
|
|
|
file_to_ingest.largest_internal_key.user_key());
|
|
|
|
}
|
2018-02-28 01:08:34 +00:00
|
|
|
}
|
2020-10-28 17:11:13 +00:00
|
|
|
Status status = cfd_->RangesOverlapWithMemtables(
|
|
|
|
ranges, super_version, db_options_.allow_data_in_errors, flush_needed);
|
2016-10-21 00:05:32 +00:00
|
|
|
if (status.ok() && *flush_needed &&
|
|
|
|
!ingestion_options_.allow_blocking_flush) {
|
|
|
|
status = Status::InvalidArgument("External file requires flush");
|
|
|
|
}
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
2017-06-24 21:06:43 +00:00
|
|
|
// REQUIRES: we have become the only writer by entering both write_thread_ and
|
|
|
|
// nonmem_write_thread_
|
2016-10-21 00:05:32 +00:00
|
|
|
Status ExternalSstFileIngestionJob::Run() {
|
|
|
|
Status status;
|
2018-02-28 01:08:34 +00:00
|
|
|
SuperVersion* super_version = cfd_->GetSuperVersion();
|
2016-10-21 00:05:32 +00:00
|
|
|
#ifndef NDEBUG
|
|
|
|
// We should never run the job with a memtable that is overlapping
|
|
|
|
// with the files we are ingesting
|
|
|
|
bool need_flush = false;
|
2018-02-28 01:08:34 +00:00
|
|
|
status = NeedsFlush(&need_flush, super_version);
|
2020-12-10 05:19:55 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
if (need_flush) {
|
|
|
|
return Status::TryAgain();
|
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
assert(status.ok() && need_flush == false);
|
|
|
|
#endif
|
|
|
|
|
|
|
|
bool force_global_seqno = false;
|
2017-04-26 20:28:39 +00:00
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
if (ingestion_options_.snapshot_consistency && !db_snapshots_->empty()) {
|
|
|
|
// We need to assign a global sequence number to all the files even
|
2020-02-28 22:10:51 +00:00
|
|
|
// if the don't overlap with any ranges since we have snapshots
|
2016-10-21 00:05:32 +00:00
|
|
|
force_global_seqno = true;
|
|
|
|
}
|
2017-11-11 01:18:01 +00:00
|
|
|
// It is safe to use this instead of LastAllocatedSequence since we are
|
2017-06-24 21:06:43 +00:00
|
|
|
// the only active writer, and hence they are equal
|
2019-09-13 21:48:18 +00:00
|
|
|
SequenceNumber last_seqno = versions_->LastSequence();
|
2016-10-21 00:05:32 +00:00
|
|
|
edit_.SetColumnFamily(cfd_->GetID());
|
|
|
|
// The levels that the files will be ingested into
|
2017-04-26 20:28:39 +00:00
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
for (IngestedFileInfo& f : files_to_ingest_) {
|
2017-04-26 20:28:39 +00:00
|
|
|
SequenceNumber assigned_seqno = 0;
|
2017-05-17 18:32:26 +00:00
|
|
|
if (ingestion_options_.ingest_behind) {
|
|
|
|
status = CheckLevelForIngestedBehindFile(&f);
|
|
|
|
} else {
|
|
|
|
status = AssignLevelAndSeqnoForIngestedFile(
|
2019-09-13 21:48:18 +00:00
|
|
|
super_version, force_global_seqno, cfd_->ioptions()->compaction_style,
|
|
|
|
last_seqno, &f, &assigned_seqno);
|
2017-05-17 18:32:26 +00:00
|
|
|
}
|
2021-04-20 20:59:24 +00:00
|
|
|
|
|
|
|
// Modify the smallest/largest internal key to include the sequence number
|
|
|
|
// that we just learned. Only overwrite sequence number zero. There could
|
|
|
|
// be a nonzero sequence number already to indicate a range tombstone's
|
|
|
|
// exclusive endpoint.
|
|
|
|
ParsedInternalKey smallest_parsed, largest_parsed;
|
|
|
|
if (status.ok()) {
|
|
|
|
status = ParseInternalKey(*f.smallest_internal_key.rep(),
|
|
|
|
&smallest_parsed, false /* log_err_key */);
|
|
|
|
}
|
|
|
|
if (status.ok()) {
|
|
|
|
status = ParseInternalKey(*f.largest_internal_key.rep(), &largest_parsed,
|
|
|
|
false /* log_err_key */);
|
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
2021-04-20 20:59:24 +00:00
|
|
|
if (smallest_parsed.sequence == 0) {
|
|
|
|
UpdateInternalKey(f.smallest_internal_key.rep(), assigned_seqno,
|
|
|
|
smallest_parsed.type);
|
|
|
|
}
|
|
|
|
if (largest_parsed.sequence == 0) {
|
|
|
|
UpdateInternalKey(f.largest_internal_key.rep(), assigned_seqno,
|
|
|
|
largest_parsed.type);
|
|
|
|
}
|
|
|
|
|
2017-04-26 20:28:39 +00:00
|
|
|
status = AssignGlobalSeqnoForIngestedFile(&f, assigned_seqno);
|
|
|
|
TEST_SYNC_POINT_CALLBACK("ExternalSstFileIngestionJob::Run",
|
|
|
|
&assigned_seqno);
|
2019-09-13 21:48:18 +00:00
|
|
|
if (assigned_seqno > last_seqno) {
|
|
|
|
assert(assigned_seqno == last_seqno + 1);
|
|
|
|
last_seqno = assigned_seqno;
|
|
|
|
++consumed_seqno_count_;
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
2019-10-14 22:19:31 +00:00
|
|
|
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
status = GenerateChecksumForIngestedFile(&f);
|
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
2019-11-23 00:01:21 +00:00
|
|
|
// We use the import time as the ancester time. This is the time the data
|
|
|
|
// is written to the database.
|
|
|
|
int64_t temp_current_time = 0;
|
2019-11-27 05:38:38 +00:00
|
|
|
uint64_t current_time = kUnknownFileCreationTime;
|
|
|
|
uint64_t oldest_ancester_time = kUnknownOldestAncesterTime;
|
2021-01-26 06:07:26 +00:00
|
|
|
if (clock_->GetCurrentTime(&temp_current_time).ok()) {
|
2019-11-27 05:38:38 +00:00
|
|
|
current_time = oldest_ancester_time =
|
|
|
|
static_cast<uint64_t>(temp_current_time);
|
2019-11-23 00:01:21 +00:00
|
|
|
}
|
Record and use the tail size to prefetch table tail (#11406)
Summary:
**Context:**
We prefetch the tail part of a SST file (i.e, the blocks after data blocks till the end of the file) during each SST file open in hope to prefetch all the stuff at once ahead of time for later read e.g, footer, meta index, filter/index etc. The existing approach to estimate the tail size to prefetch is through `TailPrefetchStats` heuristics introduced in https://github.com/facebook/rocksdb/pull/4156, which has caused small reads in unlucky case (e.g, small read into the tail buffer during table open in thread 1 under the same BlockBasedTableFactory object can make thread 2's tail prefetching use a small size that it shouldn't) and is hard to debug. Therefore we decide to record the exact tail size and use it directly to prefetch tail of the SST instead of relying heuristics.
**Summary:**
- Obtain and record in manifest the tail size in `BlockBasedTableBuilder::Finish()`
- For backward compatibility, we fall back to TailPrefetchStats and last to simple heuristics that the tail size is a linear portion of the file size - see PR conversation for more.
- Make`tail_start_offset` part of the table properties and deduct tail size to record in manifest for external files (e.g, file ingestion, import CF) and db repair (with no access to manifest).
Pull Request resolved: https://github.com/facebook/rocksdb/pull/11406
Test Plan:
1. New UT
2. db bench
Note: db bench on /tmp/ where direct read is supported is too slow to finish and the default pinning setting in db bench is not helpful to profile # sst read of Get. Therefore I hacked the following to obtain the following comparison.
```
diff --git a/table/block_based/block_based_table_reader.cc b/table/block_based/block_based_table_reader.cc
index bd5669f0f..791484c1f 100644
--- a/table/block_based/block_based_table_reader.cc
+++ b/table/block_based/block_based_table_reader.cc
@@ -838,7 +838,7 @@ Status BlockBasedTable::PrefetchTail(
&tail_prefetch_size);
// Try file system prefetch
- if (!file->use_direct_io() && !force_direct_prefetch) {
+ if (false && !file->use_direct_io() && !force_direct_prefetch) {
if (!file->Prefetch(prefetch_off, prefetch_len, ro.rate_limiter_priority)
.IsNotSupported()) {
prefetch_buffer->reset(new FilePrefetchBuffer(
diff --git a/tools/db_bench_tool.cc b/tools/db_bench_tool.cc
index ea40f5fa0..39a0ac385 100644
--- a/tools/db_bench_tool.cc
+++ b/tools/db_bench_tool.cc
@@ -4191,6 +4191,8 @@ class Benchmark {
std::shared_ptr<TableFactory>(NewCuckooTableFactory(table_options));
} else {
BlockBasedTableOptions block_based_options;
+ block_based_options.metadata_cache_options.partition_pinning =
+ PinningTier::kAll;
block_based_options.checksum =
static_cast<ChecksumType>(FLAGS_checksum_type);
if (FLAGS_use_hash_search) {
```
Create DB
```
./db_bench --bloom_bits=3 --use_existing_db=1 --seed=1682546046158958 --partition_index_and_filters=1 --statistics=1 -db=/dev/shm/testdb/ -benchmarks=readrandom -key_size=3200 -value_size=512 -num=1000000 -write_buffer_size=6550000 -disable_auto_compactions=false -target_file_size_base=6550000 -compression_type=none
```
ReadRandom
```
./db_bench --bloom_bits=3 --use_existing_db=1 --seed=1682546046158958 --partition_index_and_filters=1 --statistics=1 -db=/dev/shm/testdb/ -benchmarks=readrandom -key_size=3200 -value_size=512 -num=1000000 -write_buffer_size=6550000 -disable_auto_compactions=false -target_file_size_base=6550000 -compression_type=none
```
(a) Existing (Use TailPrefetchStats for tail size + use seperate prefetch buffer in PartitionedFilter/IndexReader::CacheDependencies())
```
rocksdb.table.open.prefetch.tail.hit COUNT : 3395
rocksdb.sst.read.micros P50 : 5.655570 P95 : 9.931396 P99 : 14.845454 P100 : 585.000000 COUNT : 999905 SUM : 6590614
```
(b) This PR (Record tail size + use the same tail buffer in PartitionedFilter/IndexReader::CacheDependencies())
```
rocksdb.table.open.prefetch.tail.hit COUNT : 14257
rocksdb.sst.read.micros P50 : 5.173347 P95 : 9.015017 P99 : 12.912610 P100 : 228.000000 COUNT : 998547 SUM : 5976540
```
As we can see, we increase the prefetch tail hit count and decrease SST read count with this PR
3. Test backward compatibility by stepping through reading with post-PR code on a db generated pre-PR.
Reviewed By: pdillinger
Differential Revision: D45413346
Pulled By: hx235
fbshipit-source-id: 7d5e36a60a72477218f79905168d688452a4c064
2023-05-08 20:14:28 +00:00
|
|
|
uint64_t tail_size = 0;
|
|
|
|
bool contain_no_data_blocks = f.table_properties.num_entries > 0 &&
|
|
|
|
(f.table_properties.num_entries ==
|
|
|
|
f.table_properties.num_range_deletions);
|
|
|
|
if (f.table_properties.tail_start_offset > 0 || contain_no_data_blocks) {
|
|
|
|
uint64_t file_size = f.fd.GetFileSize();
|
|
|
|
assert(f.table_properties.tail_start_offset <= file_size);
|
|
|
|
tail_size = file_size - f.table_properties.tail_start_offset;
|
|
|
|
}
|
|
|
|
|
2021-10-08 17:29:06 +00:00
|
|
|
FileMetaData f_metadata(
|
|
|
|
f.fd.GetNumber(), f.fd.GetPathId(), f.fd.GetFileSize(),
|
|
|
|
f.smallest_internal_key, f.largest_internal_key, f.assigned_seqno,
|
2021-12-03 22:42:05 +00:00
|
|
|
f.assigned_seqno, false, f.file_temperature, kInvalidBlobFileNumber,
|
Sort L0 files by newly introduced epoch_num (#10922)
Summary:
**Context:**
Sorting L0 files by `largest_seqno` has at least two inconvenience:
- File ingestion and compaction involving ingested files can create files of overlapping seqno range with the existing files. `force_consistency_check=true` will catch such overlap seqno range even those harmless overlap.
- For example, consider the following sequence of events ("key@n" indicates key at seqno "n")
- insert k1@1 to memtable m1
- ingest file s1 with k2@2, ingest file s2 with k3@3
- insert k4@4 to m1
- compact files s1, s2 and result in new file s3 of seqno range [2, 3]
- flush m1 and result in new file s4 of seqno range [1, 4]. And `force_consistency_check=true` will think s4 and s3 has file reordering corruption that might cause retuning an old value of k1
- However such caught corruption is a false positive since s1, s2 will not have overlapped keys with k1 or whatever inserted into m1 before ingest file s1 by the requirement of file ingestion (otherwise the m1 will be flushed first before any of the file ingestion completes). Therefore there in fact isn't any file reordering corruption.
- Single delete can decrease a file's largest seqno and ordering by `largest_seqno` can introduce a wrong ordering hence file reordering corruption
- For example, consider the following sequence of events ("key@n" indicates key at seqno "n", Credit to ajkr for this example)
- an existing SST s1 contains only k1@1
- insert k1@2 to memtable m1
- ingest file s2 with k3@3, ingest file s3 with k4@4
- insert single delete k5@5 in m1
- flush m1 and result in new file s4 of seqno range [2, 5]
- compact s1, s2, s3 and result in new file s5 of seqno range [1, 4]
- compact s4 and result in new file s6 of seqno range [2] due to single delete
- By the last step, we have file ordering by largest seqno (">" means "newer") : s5 > s6 while s6 contains a newer version of the k1's value (i.e, k1@2) than s5, which is a real reordering corruption. While this can be caught by `force_consistency_check=true`, there isn't a good way to prevent this from happening if ordering by `largest_seqno`
Therefore, we are redesigning the sorting criteria of L0 files and avoid above inconvenience. Credit to ajkr , we now introduce `epoch_num` which describes the order of a file being flushed or ingested/imported (compaction output file will has the minimum `epoch_num` among input files'). This will avoid the above inconvenience in the following ways:
- In the first case above, there will no longer be overlap seqno range check in `force_consistency_check=true` but `epoch_number` ordering check. This will result in file ordering s1 < s2 < s4 (pre-compaction) and s3 < s4 (post-compaction) which won't trigger false positive corruption. See test class `DBCompactionTestL0FilesMisorderCorruption*` for more.
- In the second case above, this will result in file ordering s1 < s2 < s3 < s4 (pre-compacting s1, s2, s3), s5 < s4 (post-compacting s1, s2, s3), s5 < s6 (post-compacting s4), which are correct file ordering without causing any corruption.
**Summary:**
- Introduce `epoch_number` stored per `ColumnFamilyData` and sort CF's L0 files by their assigned `epoch_number` instead of `largest_seqno`.
- `epoch_number` is increased and assigned upon `VersionEdit::AddFile()` for flush (or similarly for WriteLevel0TableForRecovery) and file ingestion (except for allow_behind_true, which will always get assigned as the `kReservedEpochNumberForFileIngestedBehind`)
- Compaction output file is assigned with the minimum `epoch_number` among input files'
- Refit level: reuse refitted file's epoch_number
- Other paths needing `epoch_number` treatment:
- Import column families: reuse file's epoch_number if exists. If not, assign one based on `NewestFirstBySeqNo`
- Repair: reuse file's epoch_number if exists. If not, assign one based on `NewestFirstBySeqNo`.
- Assigning new epoch_number to a file and adding this file to LSM tree should be atomic. This is guaranteed by us assigning epoch_number right upon `VersionEdit::AddFile()` where this version edit will be apply to LSM tree shape right after by holding the db mutex (e.g, flush, file ingestion, import column family) or by there is only 1 ongoing edit per CF (e.g, WriteLevel0TableForRecovery, Repair).
- Assigning the minimum input epoch number to compaction output file won't misorder L0 files (even through later `Refit(target_level=0)`). It's due to for every key "k" in the input range, a legit compaction will cover a continuous epoch number range of that key. As long as we assign the key "k" the minimum input epoch number, it won't become newer or older than the versions of this key that aren't included in this compaction hence no misorder.
- Persist `epoch_number` of each file in manifest and recover `epoch_number` on db recovery
- Backward compatibility with old db without `epoch_number` support is guaranteed by assigning `epoch_number` to recovered files by `NewestFirstBySeqno` order. See `VersionStorageInfo::RecoverEpochNumbers()` for more
- Forward compatibility with manifest is guaranteed by flexibility of `NewFileCustomTag`
- Replace `force_consistent_check` on L0 with `epoch_number` and remove false positive check like case 1 with `largest_seqno` above
- Due to backward compatibility issue, we might encounter files with missing epoch number at the beginning of db recovery. We will still use old L0 sorting mechanism (`NewestFirstBySeqno`) to check/sort them till we infer their epoch number. See usages of `EpochNumberRequirement`.
- Remove fix https://github.com/facebook/rocksdb/pull/5958#issue-511150930 and their outdated tests to file reordering corruption because such fix can be replaced by this PR.
- Misc:
- update existing tests with `epoch_number` so make check will pass
- update https://github.com/facebook/rocksdb/pull/5958#issue-511150930 tests to verify corruption is fixed using `epoch_number` and cover universal/fifo compaction/CompactRange/CompactFile cases
- assert db_mutex is held for a few places before calling ColumnFamilyData::NewEpochNumber()
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10922
Test Plan:
- `make check`
- New unit tests under `db/db_compaction_test.cc`, `db/db_test2.cc`, `db/version_builder_test.cc`, `db/repair_test.cc`
- Updated tests (i.e, `DBCompactionTestL0FilesMisorderCorruption*`) under https://github.com/facebook/rocksdb/pull/5958#issue-511150930
- [Ongoing] Compatibility test: manually run https://github.com/ajkr/rocksdb/commit/36a5686ec012f35a4371e409aa85c404ca1c210d (with file ingestion off for running the `.orig` binary to prevent this bug affecting upgrade/downgrade formality checking) for 1 hour on `simple black/white box`, `cf_consistency/txn/enable_ts with whitebox + test_best_efforts_recovery with blackbox`
- [Ongoing] normal db stress test
- [Ongoing] db stress test with aggressive value https://github.com/facebook/rocksdb/pull/10761
Reviewed By: ajkr
Differential Revision: D41063187
Pulled By: hx235
fbshipit-source-id: 826cb23455de7beaabe2d16c57682a82733a32a9
2022-12-13 21:29:37 +00:00
|
|
|
oldest_ancester_time, current_time,
|
|
|
|
ingestion_options_.ingest_behind
|
|
|
|
? kReservedEpochNumberForFileIngestedBehind
|
|
|
|
: cfd_->NewEpochNumber(),
|
2023-06-22 04:49:01 +00:00
|
|
|
f.file_checksum, f.file_checksum_func_name, f.unique_id, 0, tail_size,
|
|
|
|
static_cast<bool>(
|
|
|
|
f.table_properties.user_defined_timestamps_persisted));
|
2021-10-08 17:29:06 +00:00
|
|
|
f_metadata.temperature = f.file_temperature;
|
|
|
|
edit_.AddFile(f.picked_level, f_metadata);
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
Add missing range conflict check between file ingestion and RefitLevel() (#10988)
Summary:
**Context:**
File ingestion never checks whether the key range it acts on overlaps with an ongoing RefitLevel() (used in `CompactRange()` with `change_level=true`). That's because RefitLevel() doesn't register and make its key range known to file ingestion. Though it checks overlapping with other compactions by https://github.com/facebook/rocksdb/blob/7.8.fb/db/external_sst_file_ingestion_job.cc#L998.
RefitLevel() (used in `CompactRange()` with `change_level=true`) doesn't check whether the key range it acts on overlaps with an ongoing file ingestion. That's because file ingestion does not register and make its key range known to other compactions.
- Note that non-refitlevel-compaction (e.g, manual compaction w/o RefitLevel() or general compaction) also does not check key range overlap with ongoing file ingestion for the same reason.
- But it's fine. Credited to cbi42's discovery, `WaitForIngestFile` was called by background and foreground compactions. They were introduced in https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.
- Regardless, this PR registers file ingestion like a compaction is a general approach that will also add range conflict check between file ingestion and non-refitlevel-compaction, though it has not been the issue motivated this PR.
Above are bugs resulting in two bad consequences:
- If file ingestion and RefitLevel() creates files in the same level, then range-overlapped files will be created at that level and caught as corruption by `force_consistency_checks=true`
- If file ingestion and RefitLevel() creates file in different levels, then with one further compaction on the ingested file, it can result in two same keys both with seqno 0 in two different levels. Then with iterator's [optimization](https://github.com/facebook/rocksdb/blame/c62f3221698fd273b673d4f7e54eabb8329a4369/db/db_iter.cc#L342-L343) that assumes no two same keys both with seqno 0, it will either break this assertion in debug build or, even worst, return value of this same key for the key after it, which is the wrong value to return, in release build.
Therefore we decide to introduce range conflict check for file ingestion and RefitLevel() inspired from the existing range conflict check among compactions.
**Summary:**
- Treat file ingestion job and RefitLevel() as `Compaction` of new compaction reasons: `CompactionReason::kExternalSstIngestion` and `CompactionReason::kRefitLevel` and register/unregister them. File ingestion is treated as compaction from L0 to different levels and RefitLevel() as compaction from source level to target level.
- Check for `RangeOverlapWithCompaction` with other ongoing compactions, `RegisterCompaction()` on this "compaction" before changing the LSM state in `VersionStorageInfo`, and `UnregisterCompaction()` after changing.
- Replace scattered fixes (https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.) that prevents overlapping between file ingestion and non-refit-level compaction with this fix cuz those practices are easy to overlook.
- Misc: logic cleanup, see PR comments
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10988
Test Plan:
- New unit test `DBCompactionTestWithOngoingFileIngestionParam*` that failed pre-fix and passed afterwards.
- Made compatible with existing tests, see PR comments
- make check
- [Ongoing] Stress test rehearsal with normal value and aggressive CI value https://github.com/facebook/rocksdb/pull/10761
Reviewed By: cbi42
Differential Revision: D41535685
Pulled By: hx235
fbshipit-source-id: 549833a577ba1496d20a870583d4caa737da1258
2022-12-29 23:05:36 +00:00
|
|
|
|
|
|
|
CreateEquivalentFileIngestingCompactions();
|
2016-10-21 00:05:32 +00:00
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
Add missing range conflict check between file ingestion and RefitLevel() (#10988)
Summary:
**Context:**
File ingestion never checks whether the key range it acts on overlaps with an ongoing RefitLevel() (used in `CompactRange()` with `change_level=true`). That's because RefitLevel() doesn't register and make its key range known to file ingestion. Though it checks overlapping with other compactions by https://github.com/facebook/rocksdb/blob/7.8.fb/db/external_sst_file_ingestion_job.cc#L998.
RefitLevel() (used in `CompactRange()` with `change_level=true`) doesn't check whether the key range it acts on overlaps with an ongoing file ingestion. That's because file ingestion does not register and make its key range known to other compactions.
- Note that non-refitlevel-compaction (e.g, manual compaction w/o RefitLevel() or general compaction) also does not check key range overlap with ongoing file ingestion for the same reason.
- But it's fine. Credited to cbi42's discovery, `WaitForIngestFile` was called by background and foreground compactions. They were introduced in https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.
- Regardless, this PR registers file ingestion like a compaction is a general approach that will also add range conflict check between file ingestion and non-refitlevel-compaction, though it has not been the issue motivated this PR.
Above are bugs resulting in two bad consequences:
- If file ingestion and RefitLevel() creates files in the same level, then range-overlapped files will be created at that level and caught as corruption by `force_consistency_checks=true`
- If file ingestion and RefitLevel() creates file in different levels, then with one further compaction on the ingested file, it can result in two same keys both with seqno 0 in two different levels. Then with iterator's [optimization](https://github.com/facebook/rocksdb/blame/c62f3221698fd273b673d4f7e54eabb8329a4369/db/db_iter.cc#L342-L343) that assumes no two same keys both with seqno 0, it will either break this assertion in debug build or, even worst, return value of this same key for the key after it, which is the wrong value to return, in release build.
Therefore we decide to introduce range conflict check for file ingestion and RefitLevel() inspired from the existing range conflict check among compactions.
**Summary:**
- Treat file ingestion job and RefitLevel() as `Compaction` of new compaction reasons: `CompactionReason::kExternalSstIngestion` and `CompactionReason::kRefitLevel` and register/unregister them. File ingestion is treated as compaction from L0 to different levels and RefitLevel() as compaction from source level to target level.
- Check for `RangeOverlapWithCompaction` with other ongoing compactions, `RegisterCompaction()` on this "compaction" before changing the LSM state in `VersionStorageInfo`, and `UnregisterCompaction()` after changing.
- Replace scattered fixes (https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.) that prevents overlapping between file ingestion and non-refit-level compaction with this fix cuz those practices are easy to overlook.
- Misc: logic cleanup, see PR comments
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10988
Test Plan:
- New unit test `DBCompactionTestWithOngoingFileIngestionParam*` that failed pre-fix and passed afterwards.
- Made compatible with existing tests, see PR comments
- make check
- [Ongoing] Stress test rehearsal with normal value and aggressive CI value https://github.com/facebook/rocksdb/pull/10761
Reviewed By: cbi42
Differential Revision: D41535685
Pulled By: hx235
fbshipit-source-id: 549833a577ba1496d20a870583d4caa737da1258
2022-12-29 23:05:36 +00:00
|
|
|
void ExternalSstFileIngestionJob::CreateEquivalentFileIngestingCompactions() {
|
|
|
|
// A map from output level to input of compactions equivalent to this
|
|
|
|
// ingestion job.
|
|
|
|
// TODO: simplify below logic to creating compaction per ingested file
|
|
|
|
// instead of per output level, once we figure out how to treat ingested files
|
|
|
|
// with adjacent range deletion tombstones to same output level in the same
|
|
|
|
// job as non-overlapping compactions.
|
|
|
|
std::map<int, CompactionInputFiles>
|
|
|
|
output_level_to_file_ingesting_compaction_input;
|
|
|
|
|
|
|
|
for (const auto& pair : edit_.GetNewFiles()) {
|
|
|
|
int output_level = pair.first;
|
|
|
|
const FileMetaData& f_metadata = pair.second;
|
|
|
|
|
|
|
|
CompactionInputFiles& input =
|
|
|
|
output_level_to_file_ingesting_compaction_input[output_level];
|
|
|
|
if (input.files.empty()) {
|
|
|
|
// Treat the source level of ingested files to be level 0
|
|
|
|
input.level = 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
compaction_input_metdatas_.push_back(new FileMetaData(f_metadata));
|
|
|
|
input.files.push_back(compaction_input_metdatas_.back());
|
|
|
|
}
|
|
|
|
|
|
|
|
for (const auto& pair : output_level_to_file_ingesting_compaction_input) {
|
|
|
|
int output_level = pair.first;
|
|
|
|
const CompactionInputFiles& input = pair.second;
|
|
|
|
|
|
|
|
const auto& mutable_cf_options = *(cfd_->GetLatestMutableCFOptions());
|
|
|
|
file_ingesting_compactions_.push_back(new Compaction(
|
|
|
|
cfd_->current()->storage_info(), *cfd_->ioptions(), mutable_cf_options,
|
|
|
|
mutable_db_options_, {input}, output_level,
|
|
|
|
MaxFileSizeForLevel(
|
|
|
|
mutable_cf_options, output_level,
|
|
|
|
cfd_->ioptions()->compaction_style) /* output file size
|
|
|
|
limit,
|
|
|
|
* not applicable
|
|
|
|
*/
|
|
|
|
,
|
|
|
|
LLONG_MAX /* max compaction bytes, not applicable */,
|
|
|
|
0 /* output path ID, not applicable */, mutable_cf_options.compression,
|
|
|
|
mutable_cf_options.compression_opts, Temperature::kUnknown,
|
|
|
|
0 /* max_subcompaction, not applicable */,
|
|
|
|
{} /* grandparents, not applicable */, false /* is manual */,
|
|
|
|
"" /* trim_ts */, -1 /* score, not applicable */,
|
|
|
|
false /* is deletion compaction, not applicable */,
|
|
|
|
files_overlap_ /* l0_files_might_overlap, not applicable */,
|
|
|
|
CompactionReason::kExternalSstIngestion));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
void ExternalSstFileIngestionJob::RegisterRange() {
|
|
|
|
for (const auto& c : file_ingesting_compactions_) {
|
|
|
|
cfd_->compaction_picker()->RegisterCompaction(c);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
void ExternalSstFileIngestionJob::UnregisterRange() {
|
|
|
|
for (const auto& c : file_ingesting_compactions_) {
|
|
|
|
cfd_->compaction_picker()->UnregisterCompaction(c);
|
|
|
|
delete c;
|
|
|
|
}
|
|
|
|
file_ingesting_compactions_.clear();
|
|
|
|
|
|
|
|
for (const auto& f : compaction_input_metdatas_) {
|
|
|
|
delete f;
|
|
|
|
}
|
|
|
|
compaction_input_metdatas_.clear();
|
|
|
|
}
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
void ExternalSstFileIngestionJob::UpdateStats() {
|
|
|
|
// Update internal stats for new ingested files
|
|
|
|
uint64_t total_keys = 0;
|
|
|
|
uint64_t total_l0_files = 0;
|
2021-01-26 06:07:26 +00:00
|
|
|
uint64_t total_time = clock_->NowMicros() - job_start_time_;
|
2019-09-13 21:48:18 +00:00
|
|
|
|
|
|
|
EventLoggerStream stream = event_logger_->Log();
|
|
|
|
stream << "event"
|
|
|
|
<< "ingest_finished";
|
|
|
|
stream << "files_ingested";
|
|
|
|
stream.StartArray();
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
for (IngestedFileInfo& f : files_to_ingest_) {
|
2022-11-02 21:34:24 +00:00
|
|
|
InternalStats::CompactionStats stats(
|
|
|
|
CompactionReason::kExternalSstIngestion, 1);
|
2016-10-21 00:05:32 +00:00
|
|
|
stats.micros = total_time;
|
2018-10-05 03:44:43 +00:00
|
|
|
// If actual copy occurred for this file, then we need to count the file
|
2018-04-13 17:47:54 +00:00
|
|
|
// size as the actual bytes written. If the file was linked, then we ignore
|
|
|
|
// the bytes written for file metadata.
|
|
|
|
// TODO (yanqin) maybe account for file metadata bytes for exact accuracy?
|
|
|
|
if (f.copy_file) {
|
|
|
|
stats.bytes_written = f.fd.GetFileSize();
|
|
|
|
} else {
|
|
|
|
stats.bytes_moved = f.fd.GetFileSize();
|
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
stats.num_output_files = 1;
|
2019-03-20 00:24:09 +00:00
|
|
|
cfd_->internal_stats()->AddCompactionStats(f.picked_level,
|
|
|
|
Env::Priority::USER, stats);
|
2016-10-21 00:05:32 +00:00
|
|
|
cfd_->internal_stats()->AddCFStats(InternalStats::BYTES_INGESTED_ADD_FILE,
|
|
|
|
f.fd.GetFileSize());
|
|
|
|
total_keys += f.num_entries;
|
|
|
|
if (f.picked_level == 0) {
|
|
|
|
total_l0_files += 1;
|
|
|
|
}
|
2017-03-16 02:22:52 +00:00
|
|
|
ROCKS_LOG_INFO(
|
|
|
|
db_options_.info_log,
|
2016-10-21 00:05:32 +00:00
|
|
|
"[AddFile] External SST file %s was ingested in L%d with path %s "
|
|
|
|
"(global_seqno=%" PRIu64 ")\n",
|
|
|
|
f.external_file_path.c_str(), f.picked_level,
|
|
|
|
f.internal_file_path.c_str(), f.assigned_seqno);
|
2019-09-13 21:48:18 +00:00
|
|
|
stream << "file" << f.internal_file_path << "level" << f.picked_level;
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
2019-09-13 21:48:18 +00:00
|
|
|
stream.EndArray();
|
|
|
|
|
|
|
|
stream << "lsm_state";
|
|
|
|
stream.StartArray();
|
|
|
|
auto vstorage = cfd_->current()->storage_info();
|
|
|
|
for (int level = 0; level < vstorage->num_levels(); ++level) {
|
|
|
|
stream << vstorage->NumLevelFiles(level);
|
|
|
|
}
|
|
|
|
stream.EndArray();
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
cfd_->internal_stats()->AddCFStats(InternalStats::INGESTED_NUM_KEYS_TOTAL,
|
|
|
|
total_keys);
|
|
|
|
cfd_->internal_stats()->AddCFStats(InternalStats::INGESTED_NUM_FILES_TOTAL,
|
|
|
|
files_to_ingest_.size());
|
|
|
|
cfd_->internal_stats()->AddCFStats(
|
|
|
|
InternalStats::INGESTED_LEVEL0_NUM_FILES_TOTAL, total_l0_files);
|
|
|
|
}
|
|
|
|
|
|
|
|
void ExternalSstFileIngestionJob::Cleanup(const Status& status) {
|
2021-01-26 06:07:26 +00:00
|
|
|
IOOptions io_opts;
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
// We failed to add the files to the database
|
|
|
|
// remove all the files we copied
|
|
|
|
for (IngestedFileInfo& f : files_to_ingest_) {
|
2019-08-31 01:27:43 +00:00
|
|
|
if (f.internal_file_path.empty()) {
|
|
|
|
continue;
|
|
|
|
}
|
2021-01-26 06:07:26 +00:00
|
|
|
Status s = fs_->DeleteFile(f.internal_file_path, io_opts, nullptr);
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!s.ok()) {
|
2017-03-16 02:22:52 +00:00
|
|
|
ROCKS_LOG_WARN(db_options_.info_log,
|
|
|
|
"AddFile() clean up for file %s failed : %s",
|
|
|
|
f.internal_file_path.c_str(), s.ToString().c_str());
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
}
|
2019-09-13 21:48:18 +00:00
|
|
|
consumed_seqno_count_ = 0;
|
|
|
|
files_overlap_ = false;
|
2016-10-21 00:05:32 +00:00
|
|
|
} else if (status.ok() && ingestion_options_.move_files) {
|
|
|
|
// The files were moved and added successfully, remove original file links
|
|
|
|
for (IngestedFileInfo& f : files_to_ingest_) {
|
2021-01-26 06:07:26 +00:00
|
|
|
Status s = fs_->DeleteFile(f.external_file_path, io_opts, nullptr);
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!s.ok()) {
|
2017-03-16 02:22:52 +00:00
|
|
|
ROCKS_LOG_WARN(
|
|
|
|
db_options_.info_log,
|
2016-10-21 00:05:32 +00:00
|
|
|
"%s was added to DB successfully but failed to remove original "
|
|
|
|
"file link : %s",
|
|
|
|
f.external_file_path.c_str(), s.ToString().c_str());
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
Status ExternalSstFileIngestionJob::GetIngestedFileInfo(
|
New stable, fixed-length cache keys (#9126)
Summary:
This change standardizes on a new 16-byte cache key format for
block cache (incl compressed and secondary) and persistent cache (but
not table cache and row cache).
The goal is a really fast cache key with practically ideal stability and
uniqueness properties without external dependencies (e.g. from FileSystem).
A fixed key size of 16 bytes should enable future optimizations to the
concurrent hash table for block cache, which is a heavy CPU user /
bottleneck, but there appears to be measurable performance improvement
even with no changes to LRUCache.
This change replaces a lot of disjointed and ugly code handling cache
keys with calls to a simple, clean new internal API (cache_key.h).
(Preserving the old cache key logic under an option would be very ugly
and likely negate the performance gain of the new approach. Complete
replacement carries some inherent risk, but I think that's acceptable
with sufficient analysis and testing.)
The scheme for encoding new cache keys is complicated but explained
in cache_key.cc.
Also: EndianSwapValue is moved to math.h to be next to other bit
operations. (Explains some new include "math.h".) ReverseBits operation
added and unit tests added to hash_test for both.
Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126
Test Plan:
### Basic correctness
Several tests needed updates to work with the new functionality, mostly
because we are no longer relying on filesystem for stable cache keys
so table builders & readers need more context info to agree on cache
keys. This functionality is so core, a huge number of existing tests
exercise the cache key functionality.
### Performance
Create db with
`TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters`
And test performance with
`TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4`
using DEBUG_LEVEL=0 and simultaneous before & after runs.
Before ops/sec, avg over 100 runs: 121924
After ops/sec, avg over 100 runs: 125385 (+2.8%)
### Collision probability
I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity
over many months, by making some pessimistic simplifying assumptions:
* Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys)
* All of every file is cached for its entire lifetime
We use a simple table with skewed address assignment and replacement on address collision
to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output
with `./cache_bench -stress_cache_key -sck_keep_bits=40`:
```
Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day
Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached)
```
These come from default settings of 2.5M files per day of 32 MB each, and
`-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of
the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation
is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality.
More default assumptions, relatively pessimistic:
* 100 DBs in same process (doesn't matter much)
* Re-open DB in same process (new session ID related to old session ID) on average
every 100 files generated
* Restart process (all new session IDs unrelated to old) 24 times per day
After enough data, we get a result at the end:
```
(keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected)
```
If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data:
```
(keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected)
(keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected)
```
The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases:
```
197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected)
```
I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data.
Reviewed By: zhichao-cao
Differential Revision: D33171746
Pulled By: pdillinger
fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
2021-12-17 01:13:55 +00:00
|
|
|
const std::string& external_file, uint64_t new_file_number,
|
|
|
|
IngestedFileInfo* file_to_ingest, SuperVersion* sv) {
|
2016-10-21 00:05:32 +00:00
|
|
|
file_to_ingest->external_file_path = external_file;
|
|
|
|
|
|
|
|
// Get external file size
|
Introduce a new storage specific Env API (#5761)
Summary:
The current Env API encompasses both storage/file operations, as well as OS related operations. Most of the APIs return a Status, which does not have enough metadata about an error, such as whether its retry-able or not, scope (i.e fault domain) of the error etc., that may be required in order to properly handle a storage error. The file APIs also do not provide enough control over the IO SLA, such as timeout, prioritization, hinting about placement and redundancy etc.
This PR separates out the file/storage APIs from Env into a new FileSystem class. The APIs are updated to return an IOStatus with metadata about the error, as well as to take an IOOptions structure as input in order to allow more control over the IO.
The user can set both ```options.env``` and ```options.file_system``` to specify that RocksDB should use the former for OS related operations and the latter for storage operations. Internally, a ```CompositeEnvWrapper``` has been introduced that inherits from ```Env``` and redirects individual methods to either an ```Env``` implementation or the ```FileSystem``` as appropriate. When options are sanitized during ```DB::Open```, ```options.env``` is replaced with a newly allocated ```CompositeEnvWrapper``` instance if both env and file_system have been specified. This way, the rest of the RocksDB code can continue to function as before.
This PR also ports PosixEnv to the new API by splitting it into two - PosixEnv and PosixFileSystem. PosixEnv is defined as a sub-class of CompositeEnvWrapper, and threading/time functions are overridden with Posix specific implementations in order to avoid an extra level of indirection.
The ```CompositeEnvWrapper``` translates ```IOStatus``` return code to ```Status```, and sets the severity to ```kSoftError``` if the io_status is retryable. The error handling code in RocksDB can then recover the DB automatically.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5761
Differential Revision: D18868376
Pulled By: anand1976
fbshipit-source-id: 39efe18a162ea746fabac6360ff529baba48486f
2019-12-13 22:47:08 +00:00
|
|
|
Status status = fs_->GetFileSize(external_file, IOOptions(),
|
|
|
|
&file_to_ingest->file_size, nullptr);
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
New stable, fixed-length cache keys (#9126)
Summary:
This change standardizes on a new 16-byte cache key format for
block cache (incl compressed and secondary) and persistent cache (but
not table cache and row cache).
The goal is a really fast cache key with practically ideal stability and
uniqueness properties without external dependencies (e.g. from FileSystem).
A fixed key size of 16 bytes should enable future optimizations to the
concurrent hash table for block cache, which is a heavy CPU user /
bottleneck, but there appears to be measurable performance improvement
even with no changes to LRUCache.
This change replaces a lot of disjointed and ugly code handling cache
keys with calls to a simple, clean new internal API (cache_key.h).
(Preserving the old cache key logic under an option would be very ugly
and likely negate the performance gain of the new approach. Complete
replacement carries some inherent risk, but I think that's acceptable
with sufficient analysis and testing.)
The scheme for encoding new cache keys is complicated but explained
in cache_key.cc.
Also: EndianSwapValue is moved to math.h to be next to other bit
operations. (Explains some new include "math.h".) ReverseBits operation
added and unit tests added to hash_test for both.
Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126
Test Plan:
### Basic correctness
Several tests needed updates to work with the new functionality, mostly
because we are no longer relying on filesystem for stable cache keys
so table builders & readers need more context info to agree on cache
keys. This functionality is so core, a huge number of existing tests
exercise the cache key functionality.
### Performance
Create db with
`TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters`
And test performance with
`TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4`
using DEBUG_LEVEL=0 and simultaneous before & after runs.
Before ops/sec, avg over 100 runs: 121924
After ops/sec, avg over 100 runs: 125385 (+2.8%)
### Collision probability
I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity
over many months, by making some pessimistic simplifying assumptions:
* Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys)
* All of every file is cached for its entire lifetime
We use a simple table with skewed address assignment and replacement on address collision
to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output
with `./cache_bench -stress_cache_key -sck_keep_bits=40`:
```
Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day
Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached)
```
These come from default settings of 2.5M files per day of 32 MB each, and
`-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of
the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation
is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality.
More default assumptions, relatively pessimistic:
* 100 DBs in same process (doesn't matter much)
* Re-open DB in same process (new session ID related to old session ID) on average
every 100 files generated
* Restart process (all new session IDs unrelated to old) 24 times per day
After enough data, we get a result at the end:
```
(keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected)
```
If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data:
```
(keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected)
(keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected)
```
The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases:
```
197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected)
```
I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data.
Reviewed By: zhichao-cao
Differential Revision: D33171746
Pulled By: pdillinger
fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
2021-12-17 01:13:55 +00:00
|
|
|
// Assign FD with number
|
|
|
|
file_to_ingest->fd =
|
|
|
|
FileDescriptor(new_file_number, 0, file_to_ingest->file_size);
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
// Create TableReader for external file
|
|
|
|
std::unique_ptr<TableReader> table_reader;
|
Introduce a new storage specific Env API (#5761)
Summary:
The current Env API encompasses both storage/file operations, as well as OS related operations. Most of the APIs return a Status, which does not have enough metadata about an error, such as whether its retry-able or not, scope (i.e fault domain) of the error etc., that may be required in order to properly handle a storage error. The file APIs also do not provide enough control over the IO SLA, such as timeout, prioritization, hinting about placement and redundancy etc.
This PR separates out the file/storage APIs from Env into a new FileSystem class. The APIs are updated to return an IOStatus with metadata about the error, as well as to take an IOOptions structure as input in order to allow more control over the IO.
The user can set both ```options.env``` and ```options.file_system``` to specify that RocksDB should use the former for OS related operations and the latter for storage operations. Internally, a ```CompositeEnvWrapper``` has been introduced that inherits from ```Env``` and redirects individual methods to either an ```Env``` implementation or the ```FileSystem``` as appropriate. When options are sanitized during ```DB::Open```, ```options.env``` is replaced with a newly allocated ```CompositeEnvWrapper``` instance if both env and file_system have been specified. This way, the rest of the RocksDB code can continue to function as before.
This PR also ports PosixEnv to the new API by splitting it into two - PosixEnv and PosixFileSystem. PosixEnv is defined as a sub-class of CompositeEnvWrapper, and threading/time functions are overridden with Posix specific implementations in order to avoid an extra level of indirection.
The ```CompositeEnvWrapper``` translates ```IOStatus``` return code to ```Status```, and sets the severity to ```kSoftError``` if the io_status is retryable. The error handling code in RocksDB can then recover the DB automatically.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5761
Differential Revision: D18868376
Pulled By: anand1976
fbshipit-source-id: 39efe18a162ea746fabac6360ff529baba48486f
2019-12-13 22:47:08 +00:00
|
|
|
std::unique_ptr<FSRandomAccessFile> sst_file;
|
2016-10-21 00:05:32 +00:00
|
|
|
std::unique_ptr<RandomAccessFileReader> sst_file_reader;
|
|
|
|
|
2022-11-02 21:34:24 +00:00
|
|
|
status =
|
|
|
|
fs_->NewRandomAccessFile(external_file, env_options_, &sst_file, nullptr);
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
2020-08-27 18:20:08 +00:00
|
|
|
sst_file_reader.reset(new RandomAccessFileReader(
|
|
|
|
std::move(sst_file), external_file, nullptr /*Env*/, io_tracer_));
|
2016-10-21 00:05:32 +00:00
|
|
|
|
2023-06-22 04:49:01 +00:00
|
|
|
// TODO(yuzhangyu): User-defined timestamps doesn't support external sst file
|
|
|
|
// ingestion. Pass in the correct `user_defined_timestamps_persisted` flag
|
|
|
|
// for creating `TableReaderOptions` when the support is there.
|
2016-10-21 00:05:32 +00:00
|
|
|
status = cfd_->ioptions()->table_factory->NewTableReader(
|
New stable, fixed-length cache keys (#9126)
Summary:
This change standardizes on a new 16-byte cache key format for
block cache (incl compressed and secondary) and persistent cache (but
not table cache and row cache).
The goal is a really fast cache key with practically ideal stability and
uniqueness properties without external dependencies (e.g. from FileSystem).
A fixed key size of 16 bytes should enable future optimizations to the
concurrent hash table for block cache, which is a heavy CPU user /
bottleneck, but there appears to be measurable performance improvement
even with no changes to LRUCache.
This change replaces a lot of disjointed and ugly code handling cache
keys with calls to a simple, clean new internal API (cache_key.h).
(Preserving the old cache key logic under an option would be very ugly
and likely negate the performance gain of the new approach. Complete
replacement carries some inherent risk, but I think that's acceptable
with sufficient analysis and testing.)
The scheme for encoding new cache keys is complicated but explained
in cache_key.cc.
Also: EndianSwapValue is moved to math.h to be next to other bit
operations. (Explains some new include "math.h".) ReverseBits operation
added and unit tests added to hash_test for both.
Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126
Test Plan:
### Basic correctness
Several tests needed updates to work with the new functionality, mostly
because we are no longer relying on filesystem for stable cache keys
so table builders & readers need more context info to agree on cache
keys. This functionality is so core, a huge number of existing tests
exercise the cache key functionality.
### Performance
Create db with
`TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters`
And test performance with
`TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4`
using DEBUG_LEVEL=0 and simultaneous before & after runs.
Before ops/sec, avg over 100 runs: 121924
After ops/sec, avg over 100 runs: 125385 (+2.8%)
### Collision probability
I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity
over many months, by making some pessimistic simplifying assumptions:
* Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys)
* All of every file is cached for its entire lifetime
We use a simple table with skewed address assignment and replacement on address collision
to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output
with `./cache_bench -stress_cache_key -sck_keep_bits=40`:
```
Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day
Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached)
```
These come from default settings of 2.5M files per day of 32 MB each, and
`-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of
the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation
is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality.
More default assumptions, relatively pessimistic:
* 100 DBs in same process (doesn't matter much)
* Re-open DB in same process (new session ID related to old session ID) on average
every 100 files generated
* Restart process (all new session IDs unrelated to old) 24 times per day
After enough data, we get a result at the end:
```
(keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected)
```
If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data:
```
(keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected)
(keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected)
```
The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases:
```
197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected)
```
I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data.
Reviewed By: zhichao-cao
Differential Revision: D33171746
Pulled By: pdillinger
fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
2021-12-17 01:13:55 +00:00
|
|
|
TableReaderOptions(
|
2022-01-21 19:36:36 +00:00
|
|
|
*cfd_->ioptions(), sv->mutable_cf_options.prefix_extractor,
|
New stable, fixed-length cache keys (#9126)
Summary:
This change standardizes on a new 16-byte cache key format for
block cache (incl compressed and secondary) and persistent cache (but
not table cache and row cache).
The goal is a really fast cache key with practically ideal stability and
uniqueness properties without external dependencies (e.g. from FileSystem).
A fixed key size of 16 bytes should enable future optimizations to the
concurrent hash table for block cache, which is a heavy CPU user /
bottleneck, but there appears to be measurable performance improvement
even with no changes to LRUCache.
This change replaces a lot of disjointed and ugly code handling cache
keys with calls to a simple, clean new internal API (cache_key.h).
(Preserving the old cache key logic under an option would be very ugly
and likely negate the performance gain of the new approach. Complete
replacement carries some inherent risk, but I think that's acceptable
with sufficient analysis and testing.)
The scheme for encoding new cache keys is complicated but explained
in cache_key.cc.
Also: EndianSwapValue is moved to math.h to be next to other bit
operations. (Explains some new include "math.h".) ReverseBits operation
added and unit tests added to hash_test for both.
Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126
Test Plan:
### Basic correctness
Several tests needed updates to work with the new functionality, mostly
because we are no longer relying on filesystem for stable cache keys
so table builders & readers need more context info to agree on cache
keys. This functionality is so core, a huge number of existing tests
exercise the cache key functionality.
### Performance
Create db with
`TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters`
And test performance with
`TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4`
using DEBUG_LEVEL=0 and simultaneous before & after runs.
Before ops/sec, avg over 100 runs: 121924
After ops/sec, avg over 100 runs: 125385 (+2.8%)
### Collision probability
I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity
over many months, by making some pessimistic simplifying assumptions:
* Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys)
* All of every file is cached for its entire lifetime
We use a simple table with skewed address assignment and replacement on address collision
to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output
with `./cache_bench -stress_cache_key -sck_keep_bits=40`:
```
Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day
Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached)
```
These come from default settings of 2.5M files per day of 32 MB each, and
`-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of
the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation
is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality.
More default assumptions, relatively pessimistic:
* 100 DBs in same process (doesn't matter much)
* Re-open DB in same process (new session ID related to old session ID) on average
every 100 files generated
* Restart process (all new session IDs unrelated to old) 24 times per day
After enough data, we get a result at the end:
```
(keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected)
```
If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data:
```
(keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected)
(keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected)
```
The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases:
```
197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected)
```
I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data.
Reviewed By: zhichao-cao
Differential Revision: D33171746
Pulled By: pdillinger
fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
2021-12-17 01:13:55 +00:00
|
|
|
env_options_, cfd_->internal_comparator(),
|
2023-04-25 19:08:23 +00:00
|
|
|
sv->mutable_cf_options.block_protection_bytes_per_key,
|
New stable, fixed-length cache keys (#9126)
Summary:
This change standardizes on a new 16-byte cache key format for
block cache (incl compressed and secondary) and persistent cache (but
not table cache and row cache).
The goal is a really fast cache key with practically ideal stability and
uniqueness properties without external dependencies (e.g. from FileSystem).
A fixed key size of 16 bytes should enable future optimizations to the
concurrent hash table for block cache, which is a heavy CPU user /
bottleneck, but there appears to be measurable performance improvement
even with no changes to LRUCache.
This change replaces a lot of disjointed and ugly code handling cache
keys with calls to a simple, clean new internal API (cache_key.h).
(Preserving the old cache key logic under an option would be very ugly
and likely negate the performance gain of the new approach. Complete
replacement carries some inherent risk, but I think that's acceptable
with sufficient analysis and testing.)
The scheme for encoding new cache keys is complicated but explained
in cache_key.cc.
Also: EndianSwapValue is moved to math.h to be next to other bit
operations. (Explains some new include "math.h".) ReverseBits operation
added and unit tests added to hash_test for both.
Fixes https://github.com/facebook/rocksdb/issues/7405 (presuming a root cause)
Pull Request resolved: https://github.com/facebook/rocksdb/pull/9126
Test Plan:
### Basic correctness
Several tests needed updates to work with the new functionality, mostly
because we are no longer relying on filesystem for stable cache keys
so table builders & readers need more context info to agree on cache
keys. This functionality is so core, a huge number of existing tests
exercise the cache key functionality.
### Performance
Create db with
`TEST_TMPDIR=/dev/shm ./db_bench -bloom_bits=10 -benchmarks=fillrandom -num=3000000 -partition_index_and_filters`
And test performance with
`TEST_TMPDIR=/dev/shm ./db_bench -readonly -use_existing_db -bloom_bits=10 -benchmarks=readrandom -num=3000000 -duration=30 -cache_index_and_filter_blocks -cache_size=250000 -threads=4`
using DEBUG_LEVEL=0 and simultaneous before & after runs.
Before ops/sec, avg over 100 runs: 121924
After ops/sec, avg over 100 runs: 125385 (+2.8%)
### Collision probability
I have built a tool, ./cache_bench -stress_cache_key to broadly simulate host-wide cache activity
over many months, by making some pessimistic simplifying assumptions:
* Every generated file has a cache entry for every byte offset in the file (contiguous range of cache keys)
* All of every file is cached for its entire lifetime
We use a simple table with skewed address assignment and replacement on address collision
to simulate files coming & going, with quite a variance (super-Poisson) in ages. Some output
with `./cache_bench -stress_cache_key -sck_keep_bits=40`:
```
Total cache or DBs size: 32TiB Writing 925.926 MiB/s or 76.2939TiB/day
Multiply by 9.22337e+18 to correct for simulation losses (but still assume whole file cached)
```
These come from default settings of 2.5M files per day of 32 MB each, and
`-sck_keep_bits=40` means that to represent a single file, we are only keeping 40 bits of
the 128-bit cache key. With file size of 2\*\*25 contiguous keys (pessimistic), our simulation
is about 2\*\*(128-40-25) or about 9 billion billion times more prone to collision than reality.
More default assumptions, relatively pessimistic:
* 100 DBs in same process (doesn't matter much)
* Re-open DB in same process (new session ID related to old session ID) on average
every 100 files generated
* Restart process (all new session IDs unrelated to old) 24 times per day
After enough data, we get a result at the end:
```
(keep 40 bits) 17 collisions after 2 x 90 days, est 10.5882 days between (9.76592e+19 corrected)
```
If we believe the (pessimistic) simulation and the mathematical generalization, we would need to run a billion machines all for 97 billion days to expect a cache key collision. To help verify that our generalization ("corrected") is robust, we can make our simulation more precise with `-sck_keep_bits=41` and `42`, which takes more running time to get enough data:
```
(keep 41 bits) 16 collisions after 4 x 90 days, est 22.5 days between (1.03763e+20 corrected)
(keep 42 bits) 19 collisions after 10 x 90 days, est 47.3684 days between (1.09224e+20 corrected)
```
The generalized prediction still holds. With the `-sck_randomize` option, we can see that we are beating "random" cache keys (except offsets still non-randomized) by a modest amount (roughly 20x less collision prone than random), which should make us reasonably comfortable even in "degenerate" cases:
```
197 collisions after 1 x 90 days, est 0.456853 days between (4.21372e+18 corrected)
```
I've run other tests to validate other conditions behave as expected, never behaving "worse than random" unless we start chopping off structured data.
Reviewed By: zhichao-cao
Differential Revision: D33171746
Pulled By: pdillinger
fbshipit-source-id: f16a57e369ed37be5e7e33525ace848d0537c88f
2021-12-17 01:13:55 +00:00
|
|
|
/*skip_filters*/ false, /*immortal*/ false,
|
|
|
|
/*force_direct_prefetch*/ false, /*level*/ -1,
|
|
|
|
/*block_cache_tracer*/ nullptr,
|
|
|
|
/*max_file_size_for_l0_meta_pin*/ 0, versions_->DbSessionId(),
|
|
|
|
/*cur_file_num*/ new_file_number),
|
2016-10-21 00:05:32 +00:00
|
|
|
std::move(sst_file_reader), file_to_ingest->file_size, &table_reader);
|
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
2019-01-30 00:16:53 +00:00
|
|
|
if (ingestion_options_.verify_checksums_before_ingest) {
|
2019-08-16 23:40:09 +00:00
|
|
|
// If customized readahead size is needed, we can pass a user option
|
|
|
|
// all the way to here. Right now we just rely on the default readahead
|
|
|
|
// to keep things simple.
|
2023-04-21 16:07:18 +00:00
|
|
|
// TODO: plumb Env::IOActivity
|
2019-08-20 17:40:59 +00:00
|
|
|
ReadOptions ro;
|
|
|
|
ro.readahead_size = ingestion_options_.verify_checksums_readahead_size;
|
2019-08-16 23:40:09 +00:00
|
|
|
status = table_reader->VerifyChecksum(
|
2019-08-20 17:40:59 +00:00
|
|
|
ro, TableReaderCaller::kExternalSSTIngestion);
|
2023-06-15 23:55:38 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
2019-01-30 00:16:53 +00:00
|
|
|
}
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
// Get the external file properties
|
|
|
|
auto props = table_reader->GetTableProperties();
|
|
|
|
const auto& uprops = props->user_collected_properties;
|
|
|
|
|
|
|
|
// Get table version
|
|
|
|
auto version_iter = uprops.find(ExternalSstFilePropertyNames::kVersion);
|
|
|
|
if (version_iter == uprops.end()) {
|
|
|
|
return Status::Corruption("External file version not found");
|
|
|
|
}
|
|
|
|
file_to_ingest->version = DecodeFixed32(version_iter->second.c_str());
|
|
|
|
|
|
|
|
auto seqno_iter = uprops.find(ExternalSstFilePropertyNames::kGlobalSeqno);
|
|
|
|
if (file_to_ingest->version == 2) {
|
|
|
|
// version 2 imply that we have global sequence number
|
|
|
|
if (seqno_iter == uprops.end()) {
|
|
|
|
return Status::Corruption(
|
|
|
|
"External file global sequence number not found");
|
|
|
|
}
|
|
|
|
|
|
|
|
// Set the global sequence number
|
|
|
|
file_to_ingest->original_seqno = DecodeFixed64(seqno_iter->second.c_str());
|
2021-12-02 16:29:12 +00:00
|
|
|
if (props->external_sst_file_global_seqno_offset == 0) {
|
2018-07-06 23:07:29 +00:00
|
|
|
file_to_ingest->global_seqno_offset = 0;
|
2016-10-21 00:05:32 +00:00
|
|
|
return Status::Corruption("Was not able to find file global seqno field");
|
|
|
|
}
|
2021-12-02 16:29:12 +00:00
|
|
|
file_to_ingest->global_seqno_offset =
|
|
|
|
static_cast<size_t>(props->external_sst_file_global_seqno_offset);
|
2017-01-20 19:53:33 +00:00
|
|
|
} else if (file_to_ingest->version == 1) {
|
|
|
|
// SST file V1 should not have global seqno field
|
|
|
|
assert(seqno_iter == uprops.end());
|
2017-02-17 01:02:51 +00:00
|
|
|
file_to_ingest->original_seqno = 0;
|
2017-01-20 19:53:33 +00:00
|
|
|
if (ingestion_options_.allow_blocking_flush ||
|
2022-11-02 21:34:24 +00:00
|
|
|
ingestion_options_.allow_global_seqno) {
|
2017-01-20 19:53:33 +00:00
|
|
|
return Status::InvalidArgument(
|
2022-11-02 21:34:24 +00:00
|
|
|
"External SST file V1 does not support global seqno");
|
2017-01-20 19:53:33 +00:00
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
} else {
|
2017-01-20 19:53:33 +00:00
|
|
|
return Status::InvalidArgument("External file version is not supported");
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
// Get number of entries in table
|
|
|
|
file_to_ingest->num_entries = props->num_entries;
|
2018-07-14 05:40:23 +00:00
|
|
|
file_to_ingest->num_range_deletions = props->num_range_deletions;
|
2016-10-21 00:05:32 +00:00
|
|
|
|
|
|
|
ParsedInternalKey key;
|
2023-04-21 16:07:18 +00:00
|
|
|
// TODO: plumb Env::IOActivity
|
2016-12-08 21:30:09 +00:00
|
|
|
ReadOptions ro;
|
2018-05-21 21:33:55 +00:00
|
|
|
std::unique_ptr<InternalIterator> iter(table_reader->NewIterator(
|
2019-06-20 21:28:22 +00:00
|
|
|
ro, sv->mutable_cf_options.prefix_extractor.get(), /*arena=*/nullptr,
|
|
|
|
/*skip_filters=*/false, TableReaderCaller::kExternalSSTIngestion));
|
2018-07-14 05:40:23 +00:00
|
|
|
std::unique_ptr<InternalIterator> range_del_iter(
|
|
|
|
table_reader->NewRangeTombstoneIterator(ro));
|
2016-10-21 00:05:32 +00:00
|
|
|
|
2018-07-14 05:40:23 +00:00
|
|
|
// Get first (smallest) and last (largest) key from file.
|
2019-09-20 19:00:55 +00:00
|
|
|
file_to_ingest->smallest_internal_key =
|
|
|
|
InternalKey("", 0, ValueType::kTypeValue);
|
|
|
|
file_to_ingest->largest_internal_key =
|
|
|
|
InternalKey("", 0, ValueType::kTypeValue);
|
2018-07-14 05:40:23 +00:00
|
|
|
bool bounds_set = false;
|
2020-10-28 17:11:13 +00:00
|
|
|
bool allow_data_in_errors = db_options_.allow_data_in_errors;
|
2016-10-21 00:05:32 +00:00
|
|
|
iter->SeekToFirst();
|
2018-07-14 05:40:23 +00:00
|
|
|
if (iter->Valid()) {
|
2020-10-28 17:11:13 +00:00
|
|
|
Status pik_status =
|
|
|
|
ParseInternalKey(iter->key(), &key, allow_data_in_errors);
|
|
|
|
if (!pik_status.ok()) {
|
|
|
|
return Status::Corruption("Corrupted key in external file. ",
|
|
|
|
pik_status.getState());
|
2018-07-14 05:40:23 +00:00
|
|
|
}
|
|
|
|
if (key.sequence != 0) {
|
2020-10-28 17:11:13 +00:00
|
|
|
return Status::Corruption("External file has non zero sequence number");
|
2018-07-14 05:40:23 +00:00
|
|
|
}
|
2019-08-15 03:58:59 +00:00
|
|
|
file_to_ingest->smallest_internal_key.SetFrom(key);
|
2018-07-14 05:40:23 +00:00
|
|
|
|
|
|
|
iter->SeekToLast();
|
2020-10-28 17:11:13 +00:00
|
|
|
pik_status = ParseInternalKey(iter->key(), &key, allow_data_in_errors);
|
|
|
|
if (!pik_status.ok()) {
|
|
|
|
return Status::Corruption("Corrupted key in external file. ",
|
|
|
|
pik_status.getState());
|
2018-07-14 05:40:23 +00:00
|
|
|
}
|
|
|
|
if (key.sequence != 0) {
|
2020-10-28 17:11:13 +00:00
|
|
|
return Status::Corruption("External file has non zero sequence number");
|
2018-07-14 05:40:23 +00:00
|
|
|
}
|
2019-08-15 03:58:59 +00:00
|
|
|
file_to_ingest->largest_internal_key.SetFrom(key);
|
2016-10-21 00:05:32 +00:00
|
|
|
|
2018-07-14 05:40:23 +00:00
|
|
|
bounds_set = true;
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
2018-07-14 05:40:23 +00:00
|
|
|
|
|
|
|
// We may need to adjust these key bounds, depending on whether any range
|
|
|
|
// deletion tombstones extend past them.
|
|
|
|
const Comparator* ucmp = cfd_->internal_comparator().user_comparator();
|
|
|
|
if (range_del_iter != nullptr) {
|
|
|
|
for (range_del_iter->SeekToFirst(); range_del_iter->Valid();
|
|
|
|
range_del_iter->Next()) {
|
2020-10-28 17:11:13 +00:00
|
|
|
Status pik_status =
|
|
|
|
ParseInternalKey(range_del_iter->key(), &key, allow_data_in_errors);
|
|
|
|
if (!pik_status.ok()) {
|
|
|
|
return Status::Corruption("Corrupted key in external file. ",
|
|
|
|
pik_status.getState());
|
2018-07-14 05:40:23 +00:00
|
|
|
}
|
|
|
|
RangeTombstone tombstone(key, range_del_iter->value());
|
|
|
|
|
2019-08-15 03:58:59 +00:00
|
|
|
InternalKey start_key = tombstone.SerializeKey();
|
2019-09-20 19:00:55 +00:00
|
|
|
if (!bounds_set ||
|
|
|
|
sstableKeyCompare(ucmp, start_key,
|
|
|
|
file_to_ingest->smallest_internal_key) < 0) {
|
2019-08-15 03:58:59 +00:00
|
|
|
file_to_ingest->smallest_internal_key = start_key;
|
2018-07-14 05:40:23 +00:00
|
|
|
}
|
2019-08-15 03:58:59 +00:00
|
|
|
InternalKey end_key = tombstone.SerializeEndKey();
|
2019-09-20 19:00:55 +00:00
|
|
|
if (!bounds_set ||
|
|
|
|
sstableKeyCompare(ucmp, end_key,
|
|
|
|
file_to_ingest->largest_internal_key) > 0) {
|
2019-08-15 03:58:59 +00:00
|
|
|
file_to_ingest->largest_internal_key = end_key;
|
2018-07-14 05:40:23 +00:00
|
|
|
}
|
|
|
|
bounds_set = true;
|
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
|
2016-12-05 22:16:23 +00:00
|
|
|
file_to_ingest->cf_id = static_cast<uint32_t>(props->column_family_id);
|
|
|
|
|
2016-12-06 21:56:17 +00:00
|
|
|
file_to_ingest->table_properties = *props;
|
|
|
|
|
2022-05-19 18:04:21 +00:00
|
|
|
auto s = GetSstInternalUniqueId(props->db_id, props->db_session_id,
|
|
|
|
props->orig_file_number,
|
|
|
|
&(file_to_ingest->unique_id));
|
|
|
|
if (!s.ok()) {
|
|
|
|
ROCKS_LOG_WARN(db_options_.info_log,
|
|
|
|
"Failed to get SST unique id for file %s",
|
|
|
|
file_to_ingest->internal_file_path.c_str());
|
|
|
|
file_to_ingest->unique_id = kNullUniqueId64x2;
|
|
|
|
}
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
2017-04-26 20:28:39 +00:00
|
|
|
Status ExternalSstFileIngestionJob::AssignLevelAndSeqnoForIngestedFile(
|
|
|
|
SuperVersion* sv, bool force_global_seqno, CompactionStyle compaction_style,
|
2019-09-13 21:48:18 +00:00
|
|
|
SequenceNumber last_seqno, IngestedFileInfo* file_to_ingest,
|
|
|
|
SequenceNumber* assigned_seqno) {
|
2017-04-26 20:28:39 +00:00
|
|
|
Status status;
|
|
|
|
*assigned_seqno = 0;
|
|
|
|
if (force_global_seqno) {
|
|
|
|
*assigned_seqno = last_seqno + 1;
|
2019-09-13 21:48:18 +00:00
|
|
|
if (compaction_style == kCompactionStyleUniversal || files_overlap_) {
|
2022-04-16 01:12:06 +00:00
|
|
|
if (ingestion_options_.fail_if_not_bottommost_level) {
|
|
|
|
status = Status::TryAgain(
|
|
|
|
"Files cannot be ingested to Lmax. Please make sure key range of "
|
|
|
|
"Lmax does not overlap with files to ingest.");
|
|
|
|
return status;
|
|
|
|
}
|
2017-04-26 20:28:39 +00:00
|
|
|
file_to_ingest->picked_level = 0;
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
|
2017-04-26 20:28:39 +00:00
|
|
|
bool overlap_with_db = false;
|
2016-10-21 00:05:32 +00:00
|
|
|
Arena arena;
|
2023-04-21 16:07:18 +00:00
|
|
|
// TODO: plumb Env::IOActivity
|
2016-10-21 00:05:32 +00:00
|
|
|
ReadOptions ro;
|
|
|
|
ro.total_order_seek = true;
|
|
|
|
int target_level = 0;
|
|
|
|
auto* vstorage = cfd_->current()->storage_info();
|
2017-05-17 18:32:26 +00:00
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
for (int lvl = 0; lvl < cfd_->NumberLevels(); lvl++) {
|
|
|
|
if (lvl > 0 && lvl < vstorage->base_level()) {
|
|
|
|
continue;
|
|
|
|
}
|
Add missing range conflict check between file ingestion and RefitLevel() (#10988)
Summary:
**Context:**
File ingestion never checks whether the key range it acts on overlaps with an ongoing RefitLevel() (used in `CompactRange()` with `change_level=true`). That's because RefitLevel() doesn't register and make its key range known to file ingestion. Though it checks overlapping with other compactions by https://github.com/facebook/rocksdb/blob/7.8.fb/db/external_sst_file_ingestion_job.cc#L998.
RefitLevel() (used in `CompactRange()` with `change_level=true`) doesn't check whether the key range it acts on overlaps with an ongoing file ingestion. That's because file ingestion does not register and make its key range known to other compactions.
- Note that non-refitlevel-compaction (e.g, manual compaction w/o RefitLevel() or general compaction) also does not check key range overlap with ongoing file ingestion for the same reason.
- But it's fine. Credited to cbi42's discovery, `WaitForIngestFile` was called by background and foreground compactions. They were introduced in https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.
- Regardless, this PR registers file ingestion like a compaction is a general approach that will also add range conflict check between file ingestion and non-refitlevel-compaction, though it has not been the issue motivated this PR.
Above are bugs resulting in two bad consequences:
- If file ingestion and RefitLevel() creates files in the same level, then range-overlapped files will be created at that level and caught as corruption by `force_consistency_checks=true`
- If file ingestion and RefitLevel() creates file in different levels, then with one further compaction on the ingested file, it can result in two same keys both with seqno 0 in two different levels. Then with iterator's [optimization](https://github.com/facebook/rocksdb/blame/c62f3221698fd273b673d4f7e54eabb8329a4369/db/db_iter.cc#L342-L343) that assumes no two same keys both with seqno 0, it will either break this assertion in debug build or, even worst, return value of this same key for the key after it, which is the wrong value to return, in release build.
Therefore we decide to introduce range conflict check for file ingestion and RefitLevel() inspired from the existing range conflict check among compactions.
**Summary:**
- Treat file ingestion job and RefitLevel() as `Compaction` of new compaction reasons: `CompactionReason::kExternalSstIngestion` and `CompactionReason::kRefitLevel` and register/unregister them. File ingestion is treated as compaction from L0 to different levels and RefitLevel() as compaction from source level to target level.
- Check for `RangeOverlapWithCompaction` with other ongoing compactions, `RegisterCompaction()` on this "compaction" before changing the LSM state in `VersionStorageInfo`, and `UnregisterCompaction()` after changing.
- Replace scattered fixes (https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.) that prevents overlapping between file ingestion and non-refit-level compaction with this fix cuz those practices are easy to overlook.
- Misc: logic cleanup, see PR comments
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10988
Test Plan:
- New unit test `DBCompactionTestWithOngoingFileIngestionParam*` that failed pre-fix and passed afterwards.
- Made compatible with existing tests, see PR comments
- make check
- [Ongoing] Stress test rehearsal with normal value and aggressive CI value https://github.com/facebook/rocksdb/pull/10761
Reviewed By: cbi42
Differential Revision: D41535685
Pulled By: hx235
fbshipit-source-id: 549833a577ba1496d20a870583d4caa737da1258
2022-12-29 23:05:36 +00:00
|
|
|
if (cfd_->RangeOverlapWithCompaction(
|
|
|
|
file_to_ingest->smallest_internal_key.user_key(),
|
|
|
|
file_to_ingest->largest_internal_key.user_key(), lvl)) {
|
|
|
|
// We must use L0 or any level higher than `lvl` to be able to overwrite
|
|
|
|
// the compaction output keys that we overlap with in this level, We also
|
|
|
|
// need to assign this file a seqno to overwrite the compaction output
|
|
|
|
// keys in level `lvl`
|
|
|
|
overlap_with_db = true;
|
|
|
|
break;
|
|
|
|
} else if (vstorage->NumLevelFiles(lvl) > 0) {
|
2016-10-21 00:05:32 +00:00
|
|
|
bool overlap_with_level = false;
|
2019-09-20 19:00:55 +00:00
|
|
|
status = sv->current->OverlapWithLevelIterator(
|
|
|
|
ro, env_options_, file_to_ingest->smallest_internal_key.user_key(),
|
|
|
|
file_to_ingest->largest_internal_key.user_key(), lvl,
|
|
|
|
&overlap_with_level);
|
2016-10-21 00:05:32 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
if (overlap_with_level) {
|
|
|
|
// We must use L0 or any level higher than `lvl` to be able to overwrite
|
|
|
|
// the keys that we overlap with in this level, We also need to assign
|
|
|
|
// this file a seqno to overwrite the existing keys in level `lvl`
|
2017-04-26 20:28:39 +00:00
|
|
|
overlap_with_db = true;
|
2016-10-21 00:05:32 +00:00
|
|
|
break;
|
|
|
|
}
|
2017-04-26 20:28:39 +00:00
|
|
|
|
|
|
|
if (compaction_style == kCompactionStyleUniversal && lvl != 0) {
|
|
|
|
const std::vector<FileMetaData*>& level_files =
|
|
|
|
vstorage->LevelFiles(lvl);
|
|
|
|
const SequenceNumber level_largest_seqno =
|
2022-03-03 01:41:02 +00:00
|
|
|
(*std::max_element(level_files.begin(), level_files.end(),
|
|
|
|
[](FileMetaData* f1, FileMetaData* f2) {
|
|
|
|
return f1->fd.largest_seqno <
|
|
|
|
f2->fd.largest_seqno;
|
|
|
|
}))
|
2018-07-27 23:00:26 +00:00
|
|
|
->fd.largest_seqno;
|
2018-02-15 22:11:08 +00:00
|
|
|
// should only assign seqno to current level's largest seqno when
|
|
|
|
// the file fits
|
|
|
|
if (level_largest_seqno != 0 &&
|
|
|
|
IngestedFileFitInLevel(file_to_ingest, lvl)) {
|
2017-04-26 20:28:39 +00:00
|
|
|
*assigned_seqno = level_largest_seqno;
|
|
|
|
} else {
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
} else if (compaction_style == kCompactionStyleUniversal) {
|
|
|
|
continue;
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
|
2020-02-28 22:10:51 +00:00
|
|
|
// We don't overlap with any keys in this level, but we still need to check
|
2016-10-21 00:05:32 +00:00
|
|
|
// if our file can fit in it
|
|
|
|
if (IngestedFileFitInLevel(file_to_ingest, lvl)) {
|
|
|
|
target_level = lvl;
|
|
|
|
}
|
|
|
|
}
|
2019-09-13 21:48:18 +00:00
|
|
|
// If files overlap, we have to ingest them at level 0 and assign the newest
|
|
|
|
// sequence number
|
|
|
|
if (files_overlap_) {
|
|
|
|
target_level = 0;
|
|
|
|
*assigned_seqno = last_seqno + 1;
|
|
|
|
}
|
2022-04-16 01:12:06 +00:00
|
|
|
|
|
|
|
if (ingestion_options_.fail_if_not_bottommost_level &&
|
|
|
|
target_level < cfd_->NumberLevels() - 1) {
|
|
|
|
status = Status::TryAgain(
|
|
|
|
"Files cannot be ingested to Lmax. Please make sure key range of Lmax "
|
Add missing range conflict check between file ingestion and RefitLevel() (#10988)
Summary:
**Context:**
File ingestion never checks whether the key range it acts on overlaps with an ongoing RefitLevel() (used in `CompactRange()` with `change_level=true`). That's because RefitLevel() doesn't register and make its key range known to file ingestion. Though it checks overlapping with other compactions by https://github.com/facebook/rocksdb/blob/7.8.fb/db/external_sst_file_ingestion_job.cc#L998.
RefitLevel() (used in `CompactRange()` with `change_level=true`) doesn't check whether the key range it acts on overlaps with an ongoing file ingestion. That's because file ingestion does not register and make its key range known to other compactions.
- Note that non-refitlevel-compaction (e.g, manual compaction w/o RefitLevel() or general compaction) also does not check key range overlap with ongoing file ingestion for the same reason.
- But it's fine. Credited to cbi42's discovery, `WaitForIngestFile` was called by background and foreground compactions. They were introduced in https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.
- Regardless, this PR registers file ingestion like a compaction is a general approach that will also add range conflict check between file ingestion and non-refitlevel-compaction, though it has not been the issue motivated this PR.
Above are bugs resulting in two bad consequences:
- If file ingestion and RefitLevel() creates files in the same level, then range-overlapped files will be created at that level and caught as corruption by `force_consistency_checks=true`
- If file ingestion and RefitLevel() creates file in different levels, then with one further compaction on the ingested file, it can result in two same keys both with seqno 0 in two different levels. Then with iterator's [optimization](https://github.com/facebook/rocksdb/blame/c62f3221698fd273b673d4f7e54eabb8329a4369/db/db_iter.cc#L342-L343) that assumes no two same keys both with seqno 0, it will either break this assertion in debug build or, even worst, return value of this same key for the key after it, which is the wrong value to return, in release build.
Therefore we decide to introduce range conflict check for file ingestion and RefitLevel() inspired from the existing range conflict check among compactions.
**Summary:**
- Treat file ingestion job and RefitLevel() as `Compaction` of new compaction reasons: `CompactionReason::kExternalSstIngestion` and `CompactionReason::kRefitLevel` and register/unregister them. File ingestion is treated as compaction from L0 to different levels and RefitLevel() as compaction from source level to target level.
- Check for `RangeOverlapWithCompaction` with other ongoing compactions, `RegisterCompaction()` on this "compaction" before changing the LSM state in `VersionStorageInfo`, and `UnregisterCompaction()` after changing.
- Replace scattered fixes (https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.) that prevents overlapping between file ingestion and non-refit-level compaction with this fix cuz those practices are easy to overlook.
- Misc: logic cleanup, see PR comments
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10988
Test Plan:
- New unit test `DBCompactionTestWithOngoingFileIngestionParam*` that failed pre-fix and passed afterwards.
- Made compatible with existing tests, see PR comments
- make check
- [Ongoing] Stress test rehearsal with normal value and aggressive CI value https://github.com/facebook/rocksdb/pull/10761
Reviewed By: cbi42
Differential Revision: D41535685
Pulled By: hx235
fbshipit-source-id: 549833a577ba1496d20a870583d4caa737da1258
2022-12-29 23:05:36 +00:00
|
|
|
"and ongoing compaction's output to Lmax"
|
2022-04-16 01:12:06 +00:00
|
|
|
"does not overlap with files to ingest.");
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
2022-11-02 21:34:24 +00:00
|
|
|
TEST_SYNC_POINT_CALLBACK(
|
2017-04-26 20:28:39 +00:00
|
|
|
"ExternalSstFileIngestionJob::AssignLevelAndSeqnoForIngestedFile",
|
|
|
|
&overlap_with_db);
|
2016-10-21 00:05:32 +00:00
|
|
|
file_to_ingest->picked_level = target_level;
|
2017-04-26 20:28:39 +00:00
|
|
|
if (overlap_with_db && *assigned_seqno == 0) {
|
|
|
|
*assigned_seqno = last_seqno + 1;
|
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
return status;
|
|
|
|
}
|
|
|
|
|
2017-05-17 18:32:26 +00:00
|
|
|
Status ExternalSstFileIngestionJob::CheckLevelForIngestedBehindFile(
|
|
|
|
IngestedFileInfo* file_to_ingest) {
|
|
|
|
auto* vstorage = cfd_->current()->storage_info();
|
Add missing range conflict check between file ingestion and RefitLevel() (#10988)
Summary:
**Context:**
File ingestion never checks whether the key range it acts on overlaps with an ongoing RefitLevel() (used in `CompactRange()` with `change_level=true`). That's because RefitLevel() doesn't register and make its key range known to file ingestion. Though it checks overlapping with other compactions by https://github.com/facebook/rocksdb/blob/7.8.fb/db/external_sst_file_ingestion_job.cc#L998.
RefitLevel() (used in `CompactRange()` with `change_level=true`) doesn't check whether the key range it acts on overlaps with an ongoing file ingestion. That's because file ingestion does not register and make its key range known to other compactions.
- Note that non-refitlevel-compaction (e.g, manual compaction w/o RefitLevel() or general compaction) also does not check key range overlap with ongoing file ingestion for the same reason.
- But it's fine. Credited to cbi42's discovery, `WaitForIngestFile` was called by background and foreground compactions. They were introduced in https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.
- Regardless, this PR registers file ingestion like a compaction is a general approach that will also add range conflict check between file ingestion and non-refitlevel-compaction, though it has not been the issue motivated this PR.
Above are bugs resulting in two bad consequences:
- If file ingestion and RefitLevel() creates files in the same level, then range-overlapped files will be created at that level and caught as corruption by `force_consistency_checks=true`
- If file ingestion and RefitLevel() creates file in different levels, then with one further compaction on the ingested file, it can result in two same keys both with seqno 0 in two different levels. Then with iterator's [optimization](https://github.com/facebook/rocksdb/blame/c62f3221698fd273b673d4f7e54eabb8329a4369/db/db_iter.cc#L342-L343) that assumes no two same keys both with seqno 0, it will either break this assertion in debug build or, even worst, return value of this same key for the key after it, which is the wrong value to return, in release build.
Therefore we decide to introduce range conflict check for file ingestion and RefitLevel() inspired from the existing range conflict check among compactions.
**Summary:**
- Treat file ingestion job and RefitLevel() as `Compaction` of new compaction reasons: `CompactionReason::kExternalSstIngestion` and `CompactionReason::kRefitLevel` and register/unregister them. File ingestion is treated as compaction from L0 to different levels and RefitLevel() as compaction from source level to target level.
- Check for `RangeOverlapWithCompaction` with other ongoing compactions, `RegisterCompaction()` on this "compaction" before changing the LSM state in `VersionStorageInfo`, and `UnregisterCompaction()` after changing.
- Replace scattered fixes (https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.) that prevents overlapping between file ingestion and non-refit-level compaction with this fix cuz those practices are easy to overlook.
- Misc: logic cleanup, see PR comments
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10988
Test Plan:
- New unit test `DBCompactionTestWithOngoingFileIngestionParam*` that failed pre-fix and passed afterwards.
- Made compatible with existing tests, see PR comments
- make check
- [Ongoing] Stress test rehearsal with normal value and aggressive CI value https://github.com/facebook/rocksdb/pull/10761
Reviewed By: cbi42
Differential Revision: D41535685
Pulled By: hx235
fbshipit-source-id: 549833a577ba1496d20a870583d4caa737da1258
2022-12-29 23:05:36 +00:00
|
|
|
// First, check if new files fit in the bottommost level
|
2017-05-17 18:32:26 +00:00
|
|
|
int bottom_lvl = cfd_->NumberLevels() - 1;
|
2022-11-02 21:34:24 +00:00
|
|
|
if (!IngestedFileFitInLevel(file_to_ingest, bottom_lvl)) {
|
2017-05-17 18:32:26 +00:00
|
|
|
return Status::InvalidArgument(
|
2022-11-02 21:34:24 +00:00
|
|
|
"Can't ingest_behind file as it doesn't fit "
|
|
|
|
"at the bottommost level!");
|
2017-05-17 18:32:26 +00:00
|
|
|
}
|
|
|
|
|
Add missing range conflict check between file ingestion and RefitLevel() (#10988)
Summary:
**Context:**
File ingestion never checks whether the key range it acts on overlaps with an ongoing RefitLevel() (used in `CompactRange()` with `change_level=true`). That's because RefitLevel() doesn't register and make its key range known to file ingestion. Though it checks overlapping with other compactions by https://github.com/facebook/rocksdb/blob/7.8.fb/db/external_sst_file_ingestion_job.cc#L998.
RefitLevel() (used in `CompactRange()` with `change_level=true`) doesn't check whether the key range it acts on overlaps with an ongoing file ingestion. That's because file ingestion does not register and make its key range known to other compactions.
- Note that non-refitlevel-compaction (e.g, manual compaction w/o RefitLevel() or general compaction) also does not check key range overlap with ongoing file ingestion for the same reason.
- But it's fine. Credited to cbi42's discovery, `WaitForIngestFile` was called by background and foreground compactions. They were introduced in https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.
- Regardless, this PR registers file ingestion like a compaction is a general approach that will also add range conflict check between file ingestion and non-refitlevel-compaction, though it has not been the issue motivated this PR.
Above are bugs resulting in two bad consequences:
- If file ingestion and RefitLevel() creates files in the same level, then range-overlapped files will be created at that level and caught as corruption by `force_consistency_checks=true`
- If file ingestion and RefitLevel() creates file in different levels, then with one further compaction on the ingested file, it can result in two same keys both with seqno 0 in two different levels. Then with iterator's [optimization](https://github.com/facebook/rocksdb/blame/c62f3221698fd273b673d4f7e54eabb8329a4369/db/db_iter.cc#L342-L343) that assumes no two same keys both with seqno 0, it will either break this assertion in debug build or, even worst, return value of this same key for the key after it, which is the wrong value to return, in release build.
Therefore we decide to introduce range conflict check for file ingestion and RefitLevel() inspired from the existing range conflict check among compactions.
**Summary:**
- Treat file ingestion job and RefitLevel() as `Compaction` of new compaction reasons: `CompactionReason::kExternalSstIngestion` and `CompactionReason::kRefitLevel` and register/unregister them. File ingestion is treated as compaction from L0 to different levels and RefitLevel() as compaction from source level to target level.
- Check for `RangeOverlapWithCompaction` with other ongoing compactions, `RegisterCompaction()` on this "compaction" before changing the LSM state in `VersionStorageInfo`, and `UnregisterCompaction()` after changing.
- Replace scattered fixes (https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.) that prevents overlapping between file ingestion and non-refit-level compaction with this fix cuz those practices are easy to overlook.
- Misc: logic cleanup, see PR comments
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10988
Test Plan:
- New unit test `DBCompactionTestWithOngoingFileIngestionParam*` that failed pre-fix and passed afterwards.
- Made compatible with existing tests, see PR comments
- make check
- [Ongoing] Stress test rehearsal with normal value and aggressive CI value https://github.com/facebook/rocksdb/pull/10761
Reviewed By: cbi42
Differential Revision: D41535685
Pulled By: hx235
fbshipit-source-id: 549833a577ba1496d20a870583d4caa737da1258
2022-12-29 23:05:36 +00:00
|
|
|
// Second, check if despite allow_ingest_behind=true we still have 0 seqnums
|
2017-05-17 18:32:26 +00:00
|
|
|
// at some upper level
|
|
|
|
for (int lvl = 0; lvl < cfd_->NumberLevels() - 1; lvl++) {
|
|
|
|
for (auto file : vstorage->LevelFiles(lvl)) {
|
2018-07-27 23:00:26 +00:00
|
|
|
if (file->fd.smallest_seqno == 0) {
|
2017-05-17 18:32:26 +00:00
|
|
|
return Status::InvalidArgument(
|
2022-11-02 21:34:24 +00:00
|
|
|
"Can't ingest_behind file as despite allow_ingest_behind=true "
|
|
|
|
"there are files with 0 seqno in database at upper levels!");
|
2017-05-17 18:32:26 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
file_to_ingest->picked_level = bottom_lvl;
|
|
|
|
return Status::OK();
|
|
|
|
}
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
Status ExternalSstFileIngestionJob::AssignGlobalSeqnoForIngestedFile(
|
|
|
|
IngestedFileInfo* file_to_ingest, SequenceNumber seqno) {
|
|
|
|
if (file_to_ingest->original_seqno == seqno) {
|
|
|
|
// This file already have the correct global seqno
|
|
|
|
return Status::OK();
|
|
|
|
} else if (!ingestion_options_.allow_global_seqno) {
|
|
|
|
return Status::InvalidArgument("Global seqno is required, but disabled");
|
|
|
|
} else if (file_to_ingest->global_seqno_offset == 0) {
|
|
|
|
return Status::InvalidArgument(
|
2020-02-28 22:10:51 +00:00
|
|
|
"Trying to set global seqno for a file that don't have a global seqno "
|
2016-10-21 00:05:32 +00:00
|
|
|
"field");
|
|
|
|
}
|
|
|
|
|
2018-07-27 23:00:26 +00:00
|
|
|
if (ingestion_options_.write_global_seqno) {
|
|
|
|
// Determine if we can write global_seqno to a given offset of file.
|
|
|
|
// If the file system does not support random write, then we should not.
|
|
|
|
// Otherwise we should.
|
Introduce a new storage specific Env API (#5761)
Summary:
The current Env API encompasses both storage/file operations, as well as OS related operations. Most of the APIs return a Status, which does not have enough metadata about an error, such as whether its retry-able or not, scope (i.e fault domain) of the error etc., that may be required in order to properly handle a storage error. The file APIs also do not provide enough control over the IO SLA, such as timeout, prioritization, hinting about placement and redundancy etc.
This PR separates out the file/storage APIs from Env into a new FileSystem class. The APIs are updated to return an IOStatus with metadata about the error, as well as to take an IOOptions structure as input in order to allow more control over the IO.
The user can set both ```options.env``` and ```options.file_system``` to specify that RocksDB should use the former for OS related operations and the latter for storage operations. Internally, a ```CompositeEnvWrapper``` has been introduced that inherits from ```Env``` and redirects individual methods to either an ```Env``` implementation or the ```FileSystem``` as appropriate. When options are sanitized during ```DB::Open```, ```options.env``` is replaced with a newly allocated ```CompositeEnvWrapper``` instance if both env and file_system have been specified. This way, the rest of the RocksDB code can continue to function as before.
This PR also ports PosixEnv to the new API by splitting it into two - PosixEnv and PosixFileSystem. PosixEnv is defined as a sub-class of CompositeEnvWrapper, and threading/time functions are overridden with Posix specific implementations in order to avoid an extra level of indirection.
The ```CompositeEnvWrapper``` translates ```IOStatus``` return code to ```Status```, and sets the severity to ```kSoftError``` if the io_status is retryable. The error handling code in RocksDB can then recover the DB automatically.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5761
Differential Revision: D18868376
Pulled By: anand1976
fbshipit-source-id: 39efe18a162ea746fabac6360ff529baba48486f
2019-12-13 22:47:08 +00:00
|
|
|
std::unique_ptr<FSRandomRWFile> rwfile;
|
2022-11-02 21:34:24 +00:00
|
|
|
Status status = fs_->NewRandomRWFile(file_to_ingest->internal_file_path,
|
|
|
|
env_options_, &rwfile, nullptr);
|
2021-09-09 04:20:38 +00:00
|
|
|
TEST_SYNC_POINT_CALLBACK("ExternalSstFileIngestionJob::NewRandomRWFile",
|
|
|
|
&status);
|
2018-07-27 23:00:26 +00:00
|
|
|
if (status.ok()) {
|
2021-01-25 22:35:45 +00:00
|
|
|
FSRandomRWFilePtr fsptr(std::move(rwfile), io_tracer_,
|
|
|
|
file_to_ingest->internal_file_path);
|
2018-07-27 23:00:26 +00:00
|
|
|
std::string seqno_val;
|
|
|
|
PutFixed64(&seqno_val, seqno);
|
2020-09-08 19:20:20 +00:00
|
|
|
status = fsptr->Write(file_to_ingest->global_seqno_offset, seqno_val,
|
|
|
|
IOOptions(), nullptr);
|
2019-06-21 17:12:29 +00:00
|
|
|
if (status.ok()) {
|
|
|
|
TEST_SYNC_POINT("ExternalSstFileIngestionJob::BeforeSyncGlobalSeqno");
|
2020-09-08 19:20:20 +00:00
|
|
|
status = SyncIngestedFile(fsptr.get());
|
2019-06-21 17:12:29 +00:00
|
|
|
TEST_SYNC_POINT("ExternalSstFileIngestionJob::AfterSyncGlobalSeqno");
|
|
|
|
if (!status.ok()) {
|
|
|
|
ROCKS_LOG_WARN(db_options_.info_log,
|
|
|
|
"Failed to sync ingested file %s after writing global "
|
|
|
|
"sequence number: %s",
|
|
|
|
file_to_ingest->internal_file_path.c_str(),
|
|
|
|
status.ToString().c_str());
|
|
|
|
}
|
|
|
|
}
|
2018-07-27 23:00:26 +00:00
|
|
|
if (!status.ok()) {
|
|
|
|
return status;
|
|
|
|
}
|
|
|
|
} else if (!status.IsNotSupported()) {
|
|
|
|
return status;
|
|
|
|
}
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
|
2018-07-27 23:00:26 +00:00
|
|
|
file_to_ingest->assigned_seqno = seqno;
|
|
|
|
return Status::OK();
|
2016-10-21 00:05:32 +00:00
|
|
|
}
|
|
|
|
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
IOStatus ExternalSstFileIngestionJob::GenerateChecksumForIngestedFile(
|
|
|
|
IngestedFileInfo* file_to_ingest) {
|
|
|
|
if (db_options_.file_checksum_gen_factory == nullptr ||
|
|
|
|
need_generate_file_checksum_ == false ||
|
|
|
|
ingestion_options_.write_global_seqno == false) {
|
|
|
|
// If file_checksum_gen_factory is not set, we are not able to generate
|
|
|
|
// the checksum. if write_global_seqno is false, it means we will use
|
|
|
|
// file checksum generated during Prepare(). This step will be skipped.
|
|
|
|
return IOStatus::OK();
|
|
|
|
}
|
2020-10-28 23:46:04 +00:00
|
|
|
std::string file_checksum;
|
|
|
|
std::string file_checksum_func_name;
|
|
|
|
std::string requested_checksum_func_name;
|
2022-02-17 07:17:03 +00:00
|
|
|
// TODO: rate limit file reads for checksum calculation during file ingestion.
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
IOStatus io_s = GenerateOneFileChecksum(
|
2020-08-13 00:28:10 +00:00
|
|
|
fs_.get(), file_to_ingest->internal_file_path,
|
2020-10-28 23:46:04 +00:00
|
|
|
db_options_.file_checksum_gen_factory.get(), requested_checksum_func_name,
|
|
|
|
&file_checksum, &file_checksum_func_name,
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
ingestion_options_.verify_checksums_readahead_size,
|
2022-02-17 07:17:03 +00:00
|
|
|
db_options_.allow_mmap_reads, io_tracer_, db_options_.rate_limiter.get(),
|
|
|
|
Env::IO_TOTAL /* rate_limiter_priority */);
|
Ingest SST files with checksum information (#6891)
Summary:
Application can ingest SST files with file checksum information, such that during ingestion, DB is able to check data integrity and identify of the SST file. The PR introduces generate_and_verify_file_checksum to IngestExternalFileOption to control if the ingested checksum information should be verified with the generated checksum.
1. If generate_and_verify_file_checksum options is *FALSE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enables the SST file checksum and the checksum function name matches the checksum function name in DB, we trust the ingested checksum, store it in Manifest. If the checksum function name does not match, we treat that as an error and fail the IngestExternalFile() call.
2. If generate_and_verify_file_checksum options is *TRUE*: *1)* if DB does not enable SST file checksum, the checksum information ingested will be ignored; *2)* if DB enable the SST file checksum, we will use the checksum generator from DB to calculate the checksum for each ingested SST files after they are copied or moved. Then, compare the checksum results with the ingested checksum information: _A)_ if the checksum function name does not match, _verification always report true_ and we store the DB generated checksum information in Manifest. _B)_ if the checksum function name mach, and checksum match, ingestion continues and stores the checksum information in the Manifest. Otherwise, terminate file ingestion and report file corruption.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/6891
Test Plan: added unit test, pass make asan_check
Reviewed By: pdillinger
Differential Revision: D21935988
Pulled By: zhichao-cao
fbshipit-source-id: 7b55f486632db467e76d72602218d0658aa7f6ed
2020-06-11 21:25:01 +00:00
|
|
|
if (!io_s.ok()) {
|
|
|
|
return io_s;
|
|
|
|
}
|
|
|
|
file_to_ingest->file_checksum = file_checksum;
|
|
|
|
file_to_ingest->file_checksum_func_name = file_checksum_func_name;
|
|
|
|
return IOStatus::OK();
|
|
|
|
}
|
|
|
|
|
2016-10-21 00:05:32 +00:00
|
|
|
bool ExternalSstFileIngestionJob::IngestedFileFitInLevel(
|
|
|
|
const IngestedFileInfo* file_to_ingest, int level) {
|
|
|
|
if (level == 0) {
|
|
|
|
// Files can always fit in L0
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
auto* vstorage = cfd_->current()->storage_info();
|
2019-09-20 19:00:55 +00:00
|
|
|
Slice file_smallest_user_key(
|
|
|
|
file_to_ingest->smallest_internal_key.user_key());
|
2019-08-15 03:58:59 +00:00
|
|
|
Slice file_largest_user_key(file_to_ingest->largest_internal_key.user_key());
|
2016-10-21 00:05:32 +00:00
|
|
|
|
|
|
|
if (vstorage->OverlapInLevel(level, &file_smallest_user_key,
|
|
|
|
&file_largest_user_key)) {
|
|
|
|
// File overlap with another files in this level, we cannot
|
|
|
|
// add it to this level
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
Add missing range conflict check between file ingestion and RefitLevel() (#10988)
Summary:
**Context:**
File ingestion never checks whether the key range it acts on overlaps with an ongoing RefitLevel() (used in `CompactRange()` with `change_level=true`). That's because RefitLevel() doesn't register and make its key range known to file ingestion. Though it checks overlapping with other compactions by https://github.com/facebook/rocksdb/blob/7.8.fb/db/external_sst_file_ingestion_job.cc#L998.
RefitLevel() (used in `CompactRange()` with `change_level=true`) doesn't check whether the key range it acts on overlaps with an ongoing file ingestion. That's because file ingestion does not register and make its key range known to other compactions.
- Note that non-refitlevel-compaction (e.g, manual compaction w/o RefitLevel() or general compaction) also does not check key range overlap with ongoing file ingestion for the same reason.
- But it's fine. Credited to cbi42's discovery, `WaitForIngestFile` was called by background and foreground compactions. They were introduced in https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.
- Regardless, this PR registers file ingestion like a compaction is a general approach that will also add range conflict check between file ingestion and non-refitlevel-compaction, though it has not been the issue motivated this PR.
Above are bugs resulting in two bad consequences:
- If file ingestion and RefitLevel() creates files in the same level, then range-overlapped files will be created at that level and caught as corruption by `force_consistency_checks=true`
- If file ingestion and RefitLevel() creates file in different levels, then with one further compaction on the ingested file, it can result in two same keys both with seqno 0 in two different levels. Then with iterator's [optimization](https://github.com/facebook/rocksdb/blame/c62f3221698fd273b673d4f7e54eabb8329a4369/db/db_iter.cc#L342-L343) that assumes no two same keys both with seqno 0, it will either break this assertion in debug build or, even worst, return value of this same key for the key after it, which is the wrong value to return, in release build.
Therefore we decide to introduce range conflict check for file ingestion and RefitLevel() inspired from the existing range conflict check among compactions.
**Summary:**
- Treat file ingestion job and RefitLevel() as `Compaction` of new compaction reasons: `CompactionReason::kExternalSstIngestion` and `CompactionReason::kRefitLevel` and register/unregister them. File ingestion is treated as compaction from L0 to different levels and RefitLevel() as compaction from source level to target level.
- Check for `RangeOverlapWithCompaction` with other ongoing compactions, `RegisterCompaction()` on this "compaction" before changing the LSM state in `VersionStorageInfo`, and `UnregisterCompaction()` after changing.
- Replace scattered fixes (https://github.com/facebook/rocksdb/commit/0f88160f67d36ea30e3aca3a3cef924c3a009be6, https://github.com/facebook/rocksdb/commit/5c64fb67d2fc198f1a73ff3ae543749a6a41f513 and https://github.com/facebook/rocksdb/commit/87dfc1d23e0e16ff73e15f63c6fa0fb3b3fc8c8c.) that prevents overlapping between file ingestion and non-refit-level compaction with this fix cuz those practices are easy to overlook.
- Misc: logic cleanup, see PR comments
Pull Request resolved: https://github.com/facebook/rocksdb/pull/10988
Test Plan:
- New unit test `DBCompactionTestWithOngoingFileIngestionParam*` that failed pre-fix and passed afterwards.
- Made compatible with existing tests, see PR comments
- make check
- [Ongoing] Stress test rehearsal with normal value and aggressive CI value https://github.com/facebook/rocksdb/pull/10761
Reviewed By: cbi42
Differential Revision: D41535685
Pulled By: hx235
fbshipit-source-id: 549833a577ba1496d20a870583d4caa737da1258
2022-12-29 23:05:36 +00:00
|
|
|
// File did not overlap with level files, nor compaction output
|
2016-10-21 00:05:32 +00:00
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
2019-06-21 17:12:29 +00:00
|
|
|
template <typename TWritableFile>
|
|
|
|
Status ExternalSstFileIngestionJob::SyncIngestedFile(TWritableFile* file) {
|
|
|
|
assert(file != nullptr);
|
|
|
|
if (db_options_.use_fsync) {
|
Introduce a new storage specific Env API (#5761)
Summary:
The current Env API encompasses both storage/file operations, as well as OS related operations. Most of the APIs return a Status, which does not have enough metadata about an error, such as whether its retry-able or not, scope (i.e fault domain) of the error etc., that may be required in order to properly handle a storage error. The file APIs also do not provide enough control over the IO SLA, such as timeout, prioritization, hinting about placement and redundancy etc.
This PR separates out the file/storage APIs from Env into a new FileSystem class. The APIs are updated to return an IOStatus with metadata about the error, as well as to take an IOOptions structure as input in order to allow more control over the IO.
The user can set both ```options.env``` and ```options.file_system``` to specify that RocksDB should use the former for OS related operations and the latter for storage operations. Internally, a ```CompositeEnvWrapper``` has been introduced that inherits from ```Env``` and redirects individual methods to either an ```Env``` implementation or the ```FileSystem``` as appropriate. When options are sanitized during ```DB::Open```, ```options.env``` is replaced with a newly allocated ```CompositeEnvWrapper``` instance if both env and file_system have been specified. This way, the rest of the RocksDB code can continue to function as before.
This PR also ports PosixEnv to the new API by splitting it into two - PosixEnv and PosixFileSystem. PosixEnv is defined as a sub-class of CompositeEnvWrapper, and threading/time functions are overridden with Posix specific implementations in order to avoid an extra level of indirection.
The ```CompositeEnvWrapper``` translates ```IOStatus``` return code to ```Status```, and sets the severity to ```kSoftError``` if the io_status is retryable. The error handling code in RocksDB can then recover the DB automatically.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5761
Differential Revision: D18868376
Pulled By: anand1976
fbshipit-source-id: 39efe18a162ea746fabac6360ff529baba48486f
2019-12-13 22:47:08 +00:00
|
|
|
return file->Fsync(IOOptions(), nullptr);
|
2019-06-21 17:12:29 +00:00
|
|
|
} else {
|
Introduce a new storage specific Env API (#5761)
Summary:
The current Env API encompasses both storage/file operations, as well as OS related operations. Most of the APIs return a Status, which does not have enough metadata about an error, such as whether its retry-able or not, scope (i.e fault domain) of the error etc., that may be required in order to properly handle a storage error. The file APIs also do not provide enough control over the IO SLA, such as timeout, prioritization, hinting about placement and redundancy etc.
This PR separates out the file/storage APIs from Env into a new FileSystem class. The APIs are updated to return an IOStatus with metadata about the error, as well as to take an IOOptions structure as input in order to allow more control over the IO.
The user can set both ```options.env``` and ```options.file_system``` to specify that RocksDB should use the former for OS related operations and the latter for storage operations. Internally, a ```CompositeEnvWrapper``` has been introduced that inherits from ```Env``` and redirects individual methods to either an ```Env``` implementation or the ```FileSystem``` as appropriate. When options are sanitized during ```DB::Open```, ```options.env``` is replaced with a newly allocated ```CompositeEnvWrapper``` instance if both env and file_system have been specified. This way, the rest of the RocksDB code can continue to function as before.
This PR also ports PosixEnv to the new API by splitting it into two - PosixEnv and PosixFileSystem. PosixEnv is defined as a sub-class of CompositeEnvWrapper, and threading/time functions are overridden with Posix specific implementations in order to avoid an extra level of indirection.
The ```CompositeEnvWrapper``` translates ```IOStatus``` return code to ```Status```, and sets the severity to ```kSoftError``` if the io_status is retryable. The error handling code in RocksDB can then recover the DB automatically.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5761
Differential Revision: D18868376
Pulled By: anand1976
fbshipit-source-id: 39efe18a162ea746fabac6360ff529baba48486f
2019-12-13 22:47:08 +00:00
|
|
|
return file->Sync(IOOptions(), nullptr);
|
2019-06-21 17:12:29 +00:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2020-02-20 20:07:53 +00:00
|
|
|
} // namespace ROCKSDB_NAMESPACE
|