dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db
History
mayue.fight 8d8eb0e77e Support Clip DB to KeyRange (#11379) 
Summary:
This PR is part of the request https://github.com/facebook/rocksdb/issues/11317.
(Another part is https://github.com/facebook/rocksdb/pull/11378)

ClipDB() will clip the entries in the CF according to the range [begin_key, end_key). All the entries outside this range will be completely deleted (including tombstones).
 This feature is mainly used to ensure that there is no overlapping Key when calling CreateColumnFamilyWithImports() to import multiple CFs.

When Calling ClipDB [begin, end), there are the following steps

1.  Quickly and directly delete files without overlap
 DeleteFilesInRanges(nullptr, begin) + DeleteFilesInRanges(end, nullptr)
2. Delete the Key outside the range
Delete[smallest_key, begin) + Delete[end, largest_key]
3. Delete the tombstone through Manul Compact
CompactRange(option, nullptr, nullptr)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11379

Reviewed By: ajkr

Differential Revision: D45840358

Pulled By: cbi42

fbshipit-source-id: 54152e8a45fd8ede137f99787eb252f0b51440a4
 		2023-05-18 13:25:01 -07:00
..
blob Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
compaction Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
db_impl Support Clip DB to KeyRange (#11379) 2023-05-18 13:25:01 -07:00
wide Add a new MultiGetEntity API (#11222) 2023-02-15 09:34:17 -08:00
arena_wrapped_db_iter.cc Ignore async_io ReadOption if FileSystem doesn't support it (#11296) 2023-03-17 14:57:09 -07:00
arena_wrapped_db_iter.h
…
builder.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
builder.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
c.cc Block fetch CPU time counters in perf context (#11342) 2023-04-15 11:09:44 -07:00
c_test.c Add in-transaction multi-get-for-update to the C interface (#11107) 2023-03-27 12:14:18 -07:00
column_family.cc Support compacting files to different temperatures in FIFO compaction (#11428) 2023-05-11 16:40:59 -07:00
column_family.h New stat rocksdb.{cf|db}-write-stall-stats exposed in a structural way (#11300) 2023-03-18 09:51:58 -07:00
column_family_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
compact_files_test.cc Drain unnecessary levels when `level_compaction_dynamic_level_bytes=true` (#11340) 2023-04-06 11:20:43 -07:00
comparator_db_test.cc
…
convenience.cc Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
corruption_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
cuckoo_table_db_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_basic_test.cc Ensure VerifyFileChecksums reads don't exceed readahead_size (#11328) 2023-04-05 16:22:08 -07:00
db_block_cache_test.cc Add hash_seed to Caches (#11391) 2023-05-09 22:24:26 -07:00
db_bloom_filter_test.cc Deflake DBBloomFilterTest.OptimizeFiltersForHits (#11383) 2023-04-17 10:36:22 -07:00
db_clip_test.cc Support Clip DB to KeyRange (#11379) 2023-05-18 13:25:01 -07:00
db_compaction_filter_test.cc Always allow L0->L1 trivial move during manual compaction (#11375) 2023-04-20 11:10:48 -07:00
db_compaction_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
db_dynamic_level_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_encryption_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_filesnapshot.cc Fix bug of prematurely excluded CF in atomic flush contains unflushed data that should've been included in the atomic flush (#11148) 2023-03-14 16:53:20 -07:00
db_flush_test.cc Fix bug of prematurely excluded CF in atomic flush contains unflushed data that should've been included in the atomic flush (#11148) 2023-03-14 16:53:20 -07:00
db_info_dumper.cc
…
db_info_dumper.h
…
db_inplace_update_test.cc
…
db_io_failure_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_iter.cc DBIter::FindValueForCurrentKey: remove unused Status s (#11394) 2023-05-03 08:52:03 -07:00
db_iter.h Fix backward iteration issue when user defined timestamp is enabled in BlobDB (#11258) 2023-03-01 13:28:54 -08:00
db_iter_stress_test.cc
…
db_iter_test.cc
…
db_iterator_test.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
db_kv_checksum_test.cc
…
db_log_iter_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_logical_block_size_cache_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_memtable_test.cc
…
db_merge_operand_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_merge_operator_test.cc Better support for merge operation with data block hash index (#11356) 2023-04-07 10:06:03 -07:00
db_options_test.cc Delete temp OPTIONS file on failure to write it (#11423) 2023-05-12 22:39:39 -07:00
db_properties_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
db_range_del_test.cc Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
db_rate_limiter_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
db_readonly_with_timestamp_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_secondary_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_sst_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
db_statistics_test.cc Add block checksum mismatch ticker stat (#11438) 2023-05-12 18:16:11 -07:00
db_table_properties_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_tailing_iter_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_test.cc Support Clip DB to KeyRange (#11379) 2023-05-18 13:25:01 -07:00
db_test2.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
db_test_util.cc Fix flaky test `DBTestUniversalManualCompactionOutputPathId.ManualCompactionOutputPathId` (#11412) 2023-05-03 11:12:20 -07:00
db_test_util.h Fix flaky test `DBTestUniversalManualCompactionOutputPathId.ManualCompactionOutputPathId` (#11412) 2023-05-03 11:12:20 -07:00
db_universal_compaction_test.cc Fix flaky test `DBTestUniversalManualCompactionOutputPathId.ManualCompactionOutputPathId` (#11412) 2023-05-03 11:12:20 -07:00
db_wal_test.cc Delete empty WAL files on reopen (#11409) 2023-05-02 15:54:29 -07:00
db_with_timestamp_basic_test.cc Filter table files by timestamp: Get operator (#11332) 2023-04-06 15:39:38 -07:00
db_with_timestamp_compaction_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_with_timestamp_test_util.cc
…
db_with_timestamp_test_util.h
…
db_write_buffer_manager_test.cc Add `SetAllowStall()` (#11335) 2023-03-30 09:43:33 -07:00
db_write_test.cc Deflake DBWriteTest.LockWALInEffect (#11382) 2023-04-17 11:00:08 -07:00
dbformat.cc
…
dbformat.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
dbformat_test.cc
…
deletefile_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
error_handler.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
error_handler.h
…
error_handler_fs_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
event_helpers.cc Remove FactoryFunc from LoadXXXObject (#11203) 2023-02-17 12:54:07 -08:00
event_helpers.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
experimental.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
external_sst_file_basic_test.cc Clarify `SstFileWriter::DeleteRange()` ordering requirements (#11390) 2023-04-20 13:02:16 -07:00
external_sst_file_ingestion_job.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
external_sst_file_ingestion_job.h
…
external_sst_file_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
fault_injection_test.cc
…
file_indexer.cc
…
file_indexer.h
…
file_indexer_test.cc
…
filename_test.cc
…
flush_job.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
flush_job.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
flush_job_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
flush_scheduler.cc
…
flush_scheduler.h
…
forward_iterator.cc Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
forward_iterator.h Ignore async_io ReadOption if FileSystem doesn't support it (#11296) 2023-03-17 14:57:09 -07:00
forward_iterator_bench.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
history_trimming_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
import_column_family_job.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
import_column_family_job.h Support range deletion tombstones in `CreateColumnFamilyWithImport` (#11252) 2023-03-13 11:06:59 -07:00
import_column_family_test.cc Fix the wrong calculation of largest_key in import_column_family_job (#11381) 2023-04-15 10:33:23 -07:00
internal_stats.cc Fix write stall stats dump format (#11445) 2023-05-15 11:47:17 -07:00
internal_stats.h Add hash_seed to Caches (#11391) 2023-05-09 22:24:26 -07:00
job_context.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
kv_checksum.h Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
listener_test.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
log_format.h Add support in log writer and reader for a user-defined timestamp size record (#11433) 2023-05-11 17:26:19 -07:00
log_reader.cc Add support in log writer and reader for a user-defined timestamp size record (#11433) 2023-05-11 17:26:19 -07:00
log_reader.h Add support in log writer and reader for a user-defined timestamp size record (#11433) 2023-05-11 17:26:19 -07:00
log_test.cc Add support in log writer and reader for a user-defined timestamp size record (#11433) 2023-05-11 17:26:19 -07:00
log_writer.cc Add support in log writer and reader for a user-defined timestamp size record (#11433) 2023-05-11 17:26:19 -07:00
log_writer.h Add support in log writer and reader for a user-defined timestamp size record (#11433) 2023-05-11 17:26:19 -07:00
logs_with_prep_tracker.cc
…
logs_with_prep_tracker.h
…
lookup_key.h
…
malloc_stats.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
malloc_stats.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
manual_compaction_test.cc
…
memtable.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
memtable.h Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
memtable_list.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
memtable_list.h
…
memtable_list_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
merge_context.h
…
merge_helper.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
merge_helper.h
…
merge_helper_test.cc
…
merge_operator.cc
…
merge_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
obsolete_files_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
options_file_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
output_validator.cc
…
output_validator.h
…
perf_context_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
periodic_task_scheduler.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
periodic_task_scheduler.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
periodic_task_scheduler_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
pinned_iterators_manager.h
…
plain_table_db_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
post_memtable_callback.h
…
pre_release_callback.h
…
prefix_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
range_del_aggregator.cc Improve documentation for MergingIterator (#11161) 2023-03-03 12:17:30 -08:00
range_del_aggregator.h Improve documentation for MergingIterator (#11161) 2023-03-03 12:17:30 -08:00
range_del_aggregator_bench.cc
…
range_del_aggregator_test.cc Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
range_tombstone_fragmenter.cc
…
range_tombstone_fragmenter.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
range_tombstone_fragmenter_test.cc
…
read_callback.h
…
repair.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
repair_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
seqno_time_test.cc Remove wait_unscheduled from waitForCompact internal API (#11443) 2023-05-17 18:13:50 -07:00
seqno_to_time_mapping.cc
…
seqno_to_time_mapping.h
…
snapshot_checker.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
snapshot_impl.cc
…
snapshot_impl.h
…
table_cache.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
table_cache.h Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
table_cache_sync_and_async.h Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
table_properties_collector.cc
…
table_properties_collector.h
…
table_properties_collector_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
transaction_log_impl.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
transaction_log_impl.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
trim_history_scheduler.cc
…
trim_history_scheduler.h
…
version_builder.cc Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
version_builder.h Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
version_builder_test.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
version_edit.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
version_edit.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
version_edit_handler.cc Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
version_edit_handler.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
version_edit_test.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
version_set.cc Support Clip DB to KeyRange (#11379) 2023-05-18 13:25:01 -07:00
version_set.h Support Clip DB to KeyRange (#11379) 2023-05-18 13:25:01 -07:00
version_set_sync_and_async.h Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
version_set_test.cc Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
version_util.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
wal_edit.cc
…
wal_edit.h
…
wal_edit_test.cc
…
wal_manager.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
wal_manager.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
wal_manager_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
write_batch.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
write_batch_base.cc
…
write_batch_internal.h
…
write_batch_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
write_callback.h
…
write_callback_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
write_controller.cc
…
write_controller.h
…
write_controller_test.cc
…
write_stall_stats.cc Fix initialization-order-fiasco in write_stall_stats.cc (#11355) 2023-04-05 14:42:31 -07:00
write_stall_stats.h Fix initialization-order-fiasco in write_stall_stats.cc (#11355) 2023-04-05 14:42:31 -07:00
write_thread.cc Ensure LockWAL() stall cleared for UnlockWAL() return (#11172) 2023-02-03 12:08:37 -08:00
write_thread.h Ensure LockWAL() stall cleared for UnlockWAL() return (#11172) 2023-02-03 12:08:37 -08:00