dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db
History
Yu Zhang 13e1c32a18 Follow ups for TimedPut and write time property (#12455) 
Summary:
This PR contains a few follow ups from https://github.com/facebook/rocksdb/issues/12419 and https://github.com/facebook/rocksdb/issues/12428 including:

1) Handle a special case for `WriteBatch::TimedPut`. When the user specified write time is `std::numeric_limits<uint64_t>::max()`, it's not treated as an error, but it instead creates and writes a regular `Put` entry.

2) Update the `InternalIterator::write_unix_time` APIs to handle `kTypeValuePreferredSeqno` entries.

3) FlushJob is updated to use the seqno to time mapping copy in `SuperVersion`. FlushJob currently copy the DB's seqno to time mapping while holding db mutex and only copies the part of interest, a.k.a, the part that only goes back to the earliest sequence number of the to-be-flushed memtables. While updating FlushJob to use the mapping copy in `SuperVersion`, it's given access to the full mapping to help cover the need to convert `kTypeValuePreferredSeqno`'s write time to preferred seqno as much as possible.

Test plans:
Added unit tests

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12455

Reviewed By: pdillinger

Differential Revision: D55165422

Pulled By: jowlyzhang

fbshipit-source-id: dc022653077f678c24661de5743146a74cce4b47
 		2024-03-21 10:00:15 -07:00
..
blob Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
compaction Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
db_impl Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
wide Introduce MultiCfIterator (#12153) 2024-03-05 10:22:43 -08:00
arena_wrapped_db_iter.cc Access DBImpl* and CFD* by CFHImpl* in Iterators (#12395) 2024-03-01 10:28:20 -08:00
arena_wrapped_db_iter.h Access DBImpl* and CFD* by CFHImpl* in Iterators (#12395) 2024-03-01 10:28:20 -08:00
builder.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
builder.h Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
c.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
c_test.c Fix windows build and CI (#12426) 2024-03-14 12:04:41 -07:00
column_family.cc Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
column_family.h Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
column_family_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
compact_files_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
comparator_db_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
convenience.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
convenience_impl.h Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444) 2023-08-08 17:26:50 -07:00
corruption_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
cuckoo_table_db_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_basic_test.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
db_block_cache_test.cc don't run ZSTD_TrainDictionary in BlockBasedTableBuilder if there isn't compression needed (#12453) 2024-03-20 11:07:32 -07:00
db_bloom_filter_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_clip_test.cc Support Clip DB to KeyRange (#11379) 2023-05-18 13:25:01 -07:00
db_compaction_filter_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_compaction_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_dynamic_level_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_encryption_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_filesnapshot.cc Remove the force mode for EnableFileDeletions API (#12337) 2024-02-13 18:36:25 -08:00
db_flush_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_info_dumper.cc Don't log an error when an auxiliary dir is missing (#12326) 2024-02-05 10:26:41 -08:00
db_info_dumper.h
…
db_inplace_update_test.cc
…
db_io_failure_test.cc Retry block reads on checksum mismatch (#12427) 2024-03-18 16:16:05 -07:00
db_iter.cc Fix a bug in iterator with UDT + `ReadOptions::pin_data` (#12451) 2024-03-18 09:05:11 -07:00
db_iter.h Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
db_iter_stress_test.cc Automated modernization (#12210) 2024-01-05 11:53:57 -08:00
db_iter_test.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
db_iterator_test.cc Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
db_kv_checksum_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_log_iter_test.cc Disable flaky part of TransactionLogIteratorCheckWhenArchive (#12423) 2024-03-12 12:54:53 -07:00
db_logical_block_size_cache_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_memtable_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_merge_operand_test.cc Add `ContinueCallback` to `GetMergeOperands()` (#12438) 2024-03-15 12:25:49 -07:00
db_merge_operator_test.cc Fix the handling of wide-column base values in the max_successive_merges logic (#11913) 2023-10-02 16:25:25 -07:00
db_options_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_properties_test.cc Remove the force mode for EnableFileDeletions API (#12337) 2024-02-13 18:36:25 -08:00
db_range_del_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_rate_limiter_test.cc Enforce status checking after Valid() returns false for IteratorWrapper (#11975) 2023-10-18 09:38:38 -07:00
db_readonly_with_timestamp_test.cc Enforce status checking after Valid() returns false for IteratorWrapper (#11975) 2023-10-18 09:38:38 -07:00
db_secondary_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_sst_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_statistics_test.cc Fix double counting of BYTES_WRITTEN ticker (#12111) 2023-12-08 17:12:11 -08:00
db_table_properties_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_tailing_iter_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_test.cc Introduce MultiCfIterator (#12153) 2024-03-05 10:22:43 -08:00
db_test2.cc Fix kBlockCacheTier read with table cache miss (#12443) 2024-03-15 14:41:58 -07:00
db_test_util.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
db_test_util.h Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
db_universal_compaction_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_wal_test.cc Remove the force mode for EnableFileDeletions API (#12337) 2024-02-13 18:36:25 -08:00
db_with_timestamp_basic_test.cc Deprecate some variants of Get and MultiGet (#12327) 2024-02-16 09:21:06 -08:00
db_with_timestamp_compaction_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_with_timestamp_test_util.cc
…
db_with_timestamp_test_util.h
…
db_write_buffer_manager_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_write_test.cc Fix windows build and CI (#12426) 2024-03-14 12:04:41 -07:00
dbformat.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
dbformat.h Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
dbformat_test.cc Logically strip timestamp during flush (#11557) 2023-06-29 15:50:50 -07:00
deletefile_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
error_handler.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
error_handler.h Consolidate stats recording in error handler (#11992) 2024-01-22 14:57:30 -08:00
error_handler_fs_test.cc Quarantine files in a limbo state after a manifest error (#12030) 2023-11-11 08:11:11 -08:00
event_helpers.cc Fix/cleanup SeqnoToTimeMapping (#12253) 2024-01-19 21:50:38 -08:00
event_helpers.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
experimental.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
external_sst_file_basic_test.cc Fix windows build and CI (#12426) 2024-03-14 12:04:41 -07:00
external_sst_file_ingestion_job.cc Fix/improve temperature handling for file ingestion (#12402) 2024-03-05 16:56:08 -08:00
external_sst_file_ingestion_job.h Add support to bulk load external files for UDT in memtable only feature (#12356) 2024-02-21 15:41:53 -08:00
external_sst_file_test.cc Add support to bulk load external files for UDT in memtable only feature (#12356) 2024-02-21 15:41:53 -08:00
fault_injection_test.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
file_indexer.cc Simplify conditional judgment (#11580) 2023-07-03 09:41:48 -07:00
file_indexer.h Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
file_indexer_test.cc Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
filename_test.cc Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
flush_job.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
flush_job.h Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
flush_job_test.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
flush_scheduler.cc
…
flush_scheduler.h
…
forward_iterator.cc Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
forward_iterator.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
forward_iterator_bench.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
history_trimming_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
import_column_family_job.cc Fix/improve temperature handling for file ingestion (#12402) 2024-03-05 16:56:08 -08:00
import_column_family_job.h Support to create a CF by importing multiple non-overlapping CFs (#11378) 2023-06-15 12:25:04 -07:00
import_column_family_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
internal_stats.cc Log pending compaction bytes in a couple places (#12267) 2024-01-23 09:14:59 -08:00
internal_stats.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
job_context.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
kv_checksum.h Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
listener_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
log_format.h Add support in log writer and reader for a user-defined timestamp size record (#11433) 2023-05-11 17:26:19 -07:00
log_reader.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
log_reader.h switch to use RocksDB UnorderedMap (#11507) 2023-06-05 13:36:26 -07:00
log_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
log_writer.cc Don't write to WAL after previous WAL write error (#12448) 2024-03-18 12:27:49 -07:00
log_writer.h Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
logs_with_prep_tracker.cc
…
logs_with_prep_tracker.h Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
lookup_key.h Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
malloc_stats.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
malloc_stats.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
manual_compaction_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
memtable.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
memtable.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
memtable_list.cc Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
memtable_list.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
memtable_list_test.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
merge_context.h Add `ContinueCallback` to `GetMergeOperands()` (#12438) 2024-03-15 12:25:49 -07:00
merge_helper.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
merge_helper.h Eliminate some code duplication in MergeHelper (#12121) 2023-12-05 14:07:42 -08:00
merge_helper_test.cc Basic Support for Merge with user-defined timestamp (#10819) 2022-10-31 22:28:58 -07:00
merge_operator.cc Add helper methods WideColumnsHelper::{Has,Get}DefaultColumn (#11813) 2023-09-11 16:32:32 -07:00
merge_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
multi_cf_iterator.cc MultiCfIterator Implementations (#12422) 2024-03-18 09:05:30 -07:00
multi_cf_iterator.h MultiCfIterator Implementations (#12422) 2024-03-18 09:05:30 -07:00
multi_cf_iterator_test.cc MultiCfIterator Implementations (#12422) 2024-03-18 09:05:30 -07:00
obsolete_files_test.cc Remove the force mode for EnableFileDeletions API (#12337) 2024-02-13 18:36:25 -08:00
options_file_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
output_validator.cc Removed `check_flush_compaction_key_order` (#12311) 2024-01-31 16:30:26 -08:00
output_validator.h Removed `check_flush_compaction_key_order` (#12311) 2024-01-31 16:30:26 -08:00
perf_context_test.cc Add `write_memtable_time` to perf level `kEnableWait` (#12394) 2024-02-29 15:08:26 -08:00
periodic_task_scheduler.cc Remove extra semi colon from instagram/ranking/mezql/shots/parser/fast/Token.cpp 2024-03-04 06:32:50 -08:00
periodic_task_scheduler.h Improve efficiency of create_missing_column_families, light refactor (#11920) 2023-10-04 14:14:22 -07:00
periodic_task_scheduler_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
pinned_iterators_manager.h Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
plain_table_db_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
post_memtable_callback.h
…
pre_release_callback.h
…
prefix_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
range_del_aggregator.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
range_del_aggregator.h Add new Iterator API Refresh(const snapshot*) (#10594) 2023-09-15 10:44:43 -07:00
range_del_aggregator_bench.cc Improve FragmentTombstones() speed by lazily initializing `seq_set_` (#10848) 2022-10-25 11:33:04 -07:00
range_del_aggregator_test.cc Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
range_tombstone_fragmenter.cc Add support for range deletion when user timestamps are not persisted (#12254) 2024-01-29 11:37:34 -08:00
range_tombstone_fragmenter.h Add support for range deletion when user timestamps are not persisted (#12254) 2024-01-29 11:37:34 -08:00
range_tombstone_fragmenter_test.cc snapshots of FragmentedRangeTombstoneList must in ascending order (#11046) 2022-12-19 15:06:22 -08:00
read_callback.h
…
repair.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
repair_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
seqno_time_test.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
seqno_to_time_mapping.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
seqno_to_time_mapping.h Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
snapshot_checker.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
snapshot_impl.cc Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
snapshot_impl.h Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
table_cache.cc Fix kBlockCacheTier read with table cache miss (#12443) 2024-03-15 14:41:58 -07:00
table_cache.h Fix row cache falsely return kNotFound when timestamp enabled (#11816) 2023-09-20 11:34:38 -07:00
table_cache_sync_and_async.h Remove extra semi colon from internal_repo_rocksdb/repo/db/table_cache_sync_and_async.h 2024-03-04 06:34:44 -08:00
table_properties_collector.cc Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
table_properties_collector.h Rename IntTblPropCollector -> InternalTblPropColl (#12320) 2024-02-02 14:14:43 -08:00
table_properties_collector_test.cc Rename IntTblPropCollector -> InternalTblPropColl (#12320) 2024-02-02 14:14:43 -08:00
transaction_log_impl.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
transaction_log_impl.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
trim_history_scheduler.cc
…
trim_history_scheduler.h Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
version_builder.cc Fix blob files not reclaimed after deleting all SSTs (#12235) 2024-01-16 11:15:23 -08:00
version_builder.h Block per key-value checksum (#11287) 2023-04-25 12:08:23 -07:00
version_builder_test.cc Fix blob files not reclaimed after deleting all SSTs (#12235) 2024-01-16 11:15:23 -08:00
version_edit.cc Remove extra semi colon from hbt/src/tagstack/tests/SlicerTest.cpp (#12461) 2024-03-20 12:44:50 -07:00
version_edit.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
version_edit_handler.cc Best-effort recovery support for atomic flush (#12406) 2024-03-06 14:40:40 -08:00
version_edit_handler.h Best-effort recovery support for atomic flush (#12406) 2024-03-06 14:40:40 -08:00
version_edit_test.cc internal_repo_rocksdb (4372117296613874540) (#12117) 2023-12-04 11:17:32 -08:00
version_set.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
version_set.h Add a separate range classes for internal usage (#12071) 2024-02-06 18:35:36 -08:00
version_set_sync_and_async.h Add helper methods WideColumnsHelper::{Has,Get}DefaultColumn (#11813) 2023-09-11 16:32:32 -07:00
version_set_test.cc Best-effort recovery support for atomic flush (#12406) 2024-03-06 14:40:40 -08:00
version_util.h Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
wal_edit.cc Do not hold mutex when write keys if not necessary (#7516) 2022-07-21 13:35:36 -07:00
wal_edit.h Do not hold mutex when write keys if not necessary (#7516) 2022-07-21 13:35:36 -07:00
wal_edit_test.cc Do not hold mutex when write keys if not necessary (#7516) 2022-07-21 13:35:36 -07:00
wal_manager.cc Fix data race in WalManager (#12439) 2024-03-14 21:24:06 -07:00
wal_manager.h Fix data race in WalManager (#12439) 2024-03-14 21:24:06 -07:00
wal_manager_test.cc Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00
write_batch.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
write_batch_base.cc
…
write_batch_internal.h Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
write_batch_test.cc Follow ups for TimedPut and write time property (#12455) 2024-03-21 10:00:15 -07:00
write_callback.h
…
write_callback_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
write_controller.cc
…
write_controller.h Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
write_controller_test.cc Ran clang-format on db/ directory (#10910) 2022-11-02 14:34:24 -07:00
write_stall_stats.cc Fix initialization-order-fiasco in write_stall_stats.cc (#11355) 2023-04-05 14:42:31 -07:00
write_stall_stats.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
write_thread.cc WriteThread::EnterAsBatchGroupLeader reorder writers (#12138) 2024-02-27 15:23:54 -08:00
write_thread.h Group SST write in flush, compaction and db open with new stats (#11910) 2023-12-29 15:29:23 -08:00