dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-12-04 02:02:41 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/table/block_based
History
Yu Zhang 1104eaa35e Add initial support for TimedPut API (#12419) 
Summary:
This PR adds support for `TimedPut` API. We introduced a new type `kTypeValuePreferredSeqno` for entries added to the DB via the `TimedPut` API.

The life cycle of such an entry on the write/flush/compaction paths are:

1) It is initially added to memtable as:
`<user_key, seq, kTypeValuePreferredSeqno>: {value, write_unix_time}`

2) When it's flushed to L0 sst files, it's converted to:
`<user_key, seq, kTypeValuePreferredSeqno>: {value, preferred_seqno}`
 when we have easy access to the seqno to time mapping.

3) During compaction, if certain conditions are met, we swap in the `preferred_seqno` and the entry will become:
`<user_key, preferred_seqno, kTypeValue>: value`. This step helps fast track these entries to the cold tier if they are eligible after the sequence number swap.

On the read path:
A `kTypeValuePreferredSeqno` entry acts the same as a `kTypeValue` entry, the unix_write_time/preferred seqno part packed in value is completely ignored.

Needed follow ups:
1) The seqno to time mapping accessible in flush needs to be extended to cover the `write_unix_time` for possible `kTypeValuePreferredSeqno` entries. This also means we need to track these `write_unix_time` in memtable.

2) Compaction filter support for the new `kTypeValuePreferredSeqno` type for feature parity with other `kTypeValue` and equivalent types.

3) Stress test coverage for the feature

Pull Request resolved: https://github.com/facebook/rocksdb/pull/12419

Test Plan: Added unit tests

Reviewed By: pdillinger

Differential Revision: D54920296

Pulled By: jowlyzhang

fbshipit-source-id: c8b43f7a7c465e569141770e93c748371ff1da9e
2024-03-14 15:44:55 -07:00
..
binary_search_index_reader.cc Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
binary_search_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
block.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
block.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
block_based_table_builder.cc Remove extra semi colon from internal_repo_rocksdb/repo/db/table_cache_sync_and_async.h 2024-03-04 06:34:44 -08:00
block_based_table_builder.h Fix/cleanup SeqnoToTimeMapping (#12253) 2024-01-19 21:50:38 -08:00
block_based_table_factory.cc Persist table options use_delta_encoding in options file (#11987) 2024-02-23 14:13:28 -08:00
block_based_table_factory.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
block_based_table_iterator.cc Remove extra semi colon from internal_repo_rocksdb/repo/db/table_cache_sync_and_async.h 2024-03-04 06:34:44 -08:00
block_based_table_iterator.h Add more detail to some statuses (#12307) 2024-01-29 16:31:09 -08:00
block_based_table_reader.cc Add support for range deletion when user timestamps are not persisted (#12254) 2024-01-29 11:37:34 -08:00
block_based_table_reader.h Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
block_based_table_reader_impl.h Fix various failures in auto_readahead_size (#11884) 2023-10-02 17:47:24 -07:00
block_based_table_reader_sync_and_async.h Deprecate some variants of Get and MultiGet (#12327) 2024-02-16 09:21:06 -08:00
block_based_table_reader_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
block_builder.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
block_builder.h Logically strip timestamp during flush (#11557) 2023-06-29 15:50:50 -07:00
block_cache.cc Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
block_cache.h Support compressed and local flash secondary cache stacking (#11812) 2023-09-21 20:30:53 -07:00
block_prefetcher.cc Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
block_prefetcher.h Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
block_prefix_index.cc Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_prefix_index.h Fix bug with kHashSearch and changing prefix_extractor with SetOptions (#10128) 2022-06-10 08:51:45 -07:00
block_test.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
block_type.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
cachable_entry.h HyperClockCache support for SecondaryCache, with refactoring (#11301) 2023-03-17 20:23:49 -07:00
data_block_footer.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_footer.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
data_block_hash_index.cc Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
data_block_hash_index.h Fix build with gcc 13 by including <cstdint> (#11118) 2023-01-25 14:30:32 -08:00
data_block_hash_index_test.cc Rename IntTblPropCollector -> InternalTblPropColl (#12320) 2024-02-02 14:14:43 -08:00
filter_block.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
filter_block_reader_common.cc Avoid double block cache lookup during Seek with async_io option (#11616) 2023-09-18 11:32:30 -07:00
filter_block_reader_common.h Use user-provided ReadOptions for metadata block reads more often (#11208) 2023-04-04 16:53:14 -07:00
filter_policy.cc Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
filter_policy_internal.h Make RibbonFilterPolicy::bloom_before_level mutable (SetOptions()) (#11838) 2023-09-15 15:46:10 -07:00
flush_block_policy.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
flush_block_policy_impl.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
full_filter_block.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
full_filter_block.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
full_filter_block_test.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
hash_index_reader.cc Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
hash_index_reader.h Extend Get/MultiGet deadline support to table open (#6982) 2020-06-29 14:53:17 -07:00
index_builder.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
index_builder.h Fix some bugs in index builder and reader for the UDT in memtable only feature (#12062) 2023-11-21 14:05:02 -08:00
index_reader_common.cc Avoid double block cache lookup during Seek with async_io option (#11616) 2023-09-18 11:32:30 -07:00
index_reader_common.h Add support to strip / pad timestamp when creating / reading a block based table (#11495) 2023-06-01 11:10:03 -07:00
mock_block_based_table.h Remove deprecated block-based filter (#10184) 2022-06-16 15:51:33 -07:00
parsed_full_filter_block.cc Hide FilterBits{Builder,Reader} from public API (#9592) 2022-02-17 16:34:46 -08:00
parsed_full_filter_block.h Major Cache refactoring, CPU efficiency improvement (#10975) 2023-01-11 14:20:40 -08:00
partitioned_filter_block.cc Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
partitioned_filter_block.h Remove 'virtual' when implied by 'override' (#12319) 2024-01-31 13:14:42 -08:00
partitioned_filter_block_test.cc internal_repo_rocksdb (-8794174668376270091) (#12114) 2023-12-01 11:10:30 -08:00
partitioned_index_iterator.cc Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
partitioned_index_iterator.h Format files under table/ by clang-format (#10852) 2022-10-25 11:50:38 -07:00
partitioned_index_reader.cc Refactor FilePrefetchBuffer code (#12097) 2024-01-05 09:29:01 -08:00
partitioned_index_reader.h Record and use the tail size to prefetch table tail (#11406) 2023-05-08 13:14:28 -07:00
reader_common.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
reader_common.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
uncompression_dict_reader.cc Avoid double block cache lookup during Seek with async_io option (#11616) 2023-09-18 11:32:30 -07:00
uncompression_dict_reader.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00