dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-28 15:33:54 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db
History
Peter Dillinger 3fd1f11d35 Fix race to make BlockBasedTableOptions effectively mutable (#13082) 
Summary:
Fix a longstanding race condition in SetOptions for `block_based_table_factory` options. The fix is mostly described in new, unified `TableFactoryParseFn()` in `cf_options.cc`. Also in this PR:
* Adds a virtual `Clone()` function to TableFactory
* To avoid behavioral hiccups with `SetOptions`, make the "hidden state" of `BlockBasedTableFactory` shared between an original and a clone. For example, `TailPrefetchStats`
* `Configurable` was allowed to be copied but was not safe to do so, because the copy would have and use pointers into object it was copied from (!!!). This has been fixed using relative instead of absolute pointers, though it's still technically relying on undefined behavior (consistent object layout for non-standard-layout types).

For future follow-up:
* Deny SetOptions on block cache options (dubious and not yet made safe with proper shared_ptr handling)

Fixes https://github.com/facebook/rocksdb/issues/10079

Pull Request resolved: https://github.com/facebook/rocksdb/pull/13082

Test Plan:
added to unit tests and crash test

Ran TSAN blackbox crashtest for hours with options to amplify potential race (see https://github.com/facebook/rocksdb/issues/10079)

Reviewed By: cbi42

Differential Revision: D64947243

Pulled By: pdillinger

fbshipit-source-id: 8390299149f50e2a2b39a5247680f2637edb23c8
2024-10-25 10:24:54 -07:00
..
blob Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
compaction Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
db_impl Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
wide Fix the output of ldb dump_wal for PutEntity records (#12677) 2024-05-20 17:04:14 -07:00
arena_wrapped_db_iter.cc Steps toward deprecating implicit prefix seek, related fixes (#13026) 2024-09-20 15:54:19 -07:00
arena_wrapped_db_iter.h Support the on-demand loading of blobs during iteration (#13069) 2024-10-16 12:34:57 -07:00
attribute_group_iterator_impl.cc MultiCfIterator - AttributeGroupIter Impl & CoalescingIter Optimization (#12534) 2024-04-16 08:45:38 -07:00
attribute_group_iterator_impl.h Support allow_unprepared_value for multi-CF iterators (#13079) 2024-10-20 20:53:08 -07:00
builder.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
builder.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
c.cc Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
c_test.c Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
coalescing_iterator.cc MultiCfIterator - AttributeGroupIter Impl & CoalescingIter Optimization (#12534) 2024-04-16 08:45:38 -07:00
coalescing_iterator.h Support allow_unprepared_value for multi-CF iterators (#13079) 2024-10-20 20:53:08 -07:00
column_family.cc Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
column_family.h Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
column_family_test.cc Remove stale entries from L0 files when UDT is not persisted (#13035) 2024-10-14 12:28:35 -07:00
compact_files_test.cc Prevent data block compression with BlockBasedTableOptions::block_align (#12592) 2024-04-26 20:05:30 -07:00
comparator_db_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
convenience.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
convenience_impl.h
corruption_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
cuckoo_table_db_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_basic_test.cc Steps toward making IDENTITY file obsolete (#13019) 2024-09-19 14:05:21 -07:00
db_block_cache_test.cc Fix race to make BlockBasedTableOptions effectively mutable (#13082) 2024-10-25 10:24:54 -07:00
db_bloom_filter_test.cc Fix race to make BlockBasedTableOptions effectively mutable (#13082) 2024-10-25 10:24:54 -07:00
db_clip_test.cc
db_compaction_filter_test.cc Replace ScopedArenaIterator with ScopedArenaPtr<InternalIterator> (#12470) 2024-03-22 13:40:42 -07:00
db_compaction_test.cc Compact one file at a time for FIFO temperature change compactions (#13018) 2024-09-19 15:50:41 -07:00
db_dynamic_level_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_encryption_test.cc
db_filesnapshot.cc Avoid unnecessary work in internal calls to GetSortedWalFiles (#12831) 2024-07-01 23:29:02 -07:00
db_flush_test.cc Refactor SyncWAL and SyncClosedLogs for code sharing (#12707) 2024-05-30 14:53:13 -07:00
db_follower_test.cc Move file tracking in VersionEditHandlerPointInTime to VersionBuilder (#12928) 2024-08-12 21:09:37 -07:00
db_info_dumper.cc
db_info_dumper.h
db_inplace_update_test.cc
db_io_failure_test.cc Fix a couple of missing cases of retry on corruption (#13007) 2024-09-13 13:56:49 -07:00
db_iter.cc Support the on-demand loading of blobs during iteration (#13069) 2024-10-16 12:34:57 -07:00
db_iter.h Support the on-demand loading of blobs during iteration (#13069) 2024-10-16 12:34:57 -07:00
db_iter_stress_test.cc
db_iter_test.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
db_iterator_test.cc Add Iterator property "rocksdb.iterator.is-value-pinned" (#12659) 2024-05-15 19:11:52 -07:00
db_kv_checksum_test.cc Fix MultiGet dropping memtable kv checksum corruption (#12842) 2024-08-08 13:34:11 -07:00
db_log_iter_test.cc Disable flaky part of TransactionLogIteratorCheckWhenArchive (#12423) 2024-03-12 12:54:53 -07:00
db_logical_block_size_cache_test.cc
db_memtable_test.cc Add an option to verify memtable key order during reads (#12889) 2024-08-19 13:53:25 -07:00
db_merge_operand_test.cc Add ContinueCallback to GetMergeOperands() (#12438) 2024-03-15 12:25:49 -07:00
db_merge_operator_test.cc
db_options_test.cc Fix race to make BlockBasedTableOptions effectively mutable (#13082) 2024-10-25 10:24:54 -07:00
db_properties_test.cc GetAggregatedIntProperty accumulates property once per block cache (#12755) 2024-06-18 10:46:55 -07:00
db_range_del_test.cc Fail DeleteRange() early when row_cache is configured (#12710) 2024-05-29 15:03:15 -07:00
db_rate_limiter_test.cc Fix db_rate_limiter_test for win (#12816) 2024-07-01 16:14:19 -07:00
db_readonly_with_timestamp_test.cc
db_secondary_test.cc Fix Compaction Stats (#13071) 2024-10-16 19:20:37 -07:00
db_sst_test.cc Fix orphaned files in SstFileManager (#13015) 2024-09-18 13:27:44 -07:00
db_statistics_test.cc
db_table_properties_test.cc Run internal cpp modernizer on RocksDB repo (#12398) 2024-03-04 10:08:32 -08:00
db_tailing_iter_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_test.cc Improve DBTest.DynamicLevelCompressionPerLevel (#13044) 2024-10-09 12:51:19 -07:00
db_test2.cc Steps toward deprecating implicit prefix seek, related fixes (#13026) 2024-09-20 15:54:19 -07:00
db_test_util.cc Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
db_test_util.h Options for file temperature for more files (#12957) 2024-08-23 19:49:25 -07:00
db_universal_compaction_test.cc Option to decouple index and filter partitions (#12939) 2024-08-16 15:34:31 -07:00
db_wal_test.cc Fix a bug with auto recovery on WAL write error (#12995) 2024-09-17 14:10:33 -07:00
db_with_timestamp_basic_test.cc Steps toward deprecating implicit prefix seek, related fixes (#13026) 2024-09-20 15:54:19 -07:00
db_with_timestamp_compaction_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_with_timestamp_test_util.cc
db_with_timestamp_test_util.h
db_write_buffer_manager_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
db_write_test.cc Disallow memtable flush and sst ingest while WAL is locked (#12652) 2024-05-21 10:17:34 -07:00
dbformat.cc Add some per key optimization for UDT in memtable only feature (#13031) 2024-10-03 17:57:50 -07:00
dbformat.h Remove stale entries from L0 files when UDT is not persisted (#13035) 2024-10-14 12:28:35 -07:00
dbformat_test.cc Refactor IndexBuilder::AddIndexEntry (#12867) 2024-07-22 14:27:31 -07:00
deletefile_test.cc Add an option to wait for purge in WaitForCompact (#12520) 2024-04-17 17:33:27 -07:00
error_handler.cc Fix a bug with auto recovery on WAL write error (#12995) 2024-09-17 14:10:33 -07:00
error_handler.h Fix a bug with auto recovery on WAL write error (#12995) 2024-09-17 14:10:33 -07:00
error_handler_fs_test.cc Remove the return value of SetBGError() (#12792) 2024-06-26 18:17:05 -07:00
event_helpers.cc Record largest seqno in table properties and verify in file ingestion (#12951) 2024-08-21 16:24:18 -07:00
event_helpers.h
experimental.cc Fix and generalize framework for filtering range queries, etc. (#13005) 2024-09-18 15:26:37 -07:00
external_sst_file_basic_test.cc Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
external_sst_file_ingestion_job.cc Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
external_sst_file_ingestion_job.h Ingest files in separate batches if they overlap (#13064) 2024-10-15 17:22:01 -07:00
external_sst_file_test.cc Add an ingestion option to not fill block cache (#13067) 2024-10-16 14:11:22 -07:00
fault_injection_test.cc FaultInjectionTestFS follow-up and clean-up (#12861) 2024-07-15 10:28:34 -07:00
file_indexer.cc
file_indexer.h
file_indexer_test.cc
filename_test.cc
flush_job.cc Remove stale entries from L0 files when UDT is not persisted (#13035) 2024-10-14 12:28:35 -07:00
flush_job.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
flush_job_test.cc Remove stale entries from L0 files when UDT is not persisted (#13035) 2024-10-14 12:28:35 -07:00
flush_scheduler.cc
flush_scheduler.h
forward_iterator.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
forward_iterator.h Support returning write unix time in iterator property (#12428) 2024-03-15 15:37:37 -07:00
forward_iterator_bench.cc
history_trimming_iterator.h
import_column_family_job.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
import_column_family_job.h
import_column_family_test.cc Introduce minimum compaction debt requirement for parallel compaction (#13054) 2024-10-04 15:01:54 -07:00
internal_stats.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
internal_stats.h GetAggregatedIntProperty accumulates property once per block cache (#12755) 2024-06-18 10:46:55 -07:00
job_context.h Preserve Options File (#13074) 2024-10-16 09:22:51 -07:00
kv_checksum.h
listener_test.cc Add some checks at property block creation side (#12898) 2024-07-31 13:28:17 -07:00
log_format.h
log_reader.cc Fix recycled WAL detection when wal_compression is enabled (#12643) 2024-05-22 15:34:37 -07:00
log_reader.h Enable recycle_log_file_num option for point in time recovery (#12403) 2024-03-21 12:29:35 -07:00
log_test.cc Fix recycled WAL detection when wal_compression is enabled (#12643) 2024-05-22 15:34:37 -07:00
log_writer.cc Handle injected write error after successful WAL write in crash test + misc (#12838) 2024-07-29 13:51:49 -07:00
log_writer.h Ensure Close() before LinkFile() for WALs in Checkpoint (#12734) 2024-06-12 11:48:45 -07:00
logs_with_prep_tracker.cc
logs_with_prep_tracker.h
lookup_key.h
malloc_stats.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
malloc_stats.h
manual_compaction_test.cc
memtable.cc Remove stale entries from L0 files when UDT is not persisted (#13035) 2024-10-14 12:28:35 -07:00
memtable.h Remove stale entries from L0 files when UDT is not persisted (#13035) 2024-10-14 12:28:35 -07:00
memtable_list.cc Fix needs_flush assertion in file ingestion (#13045) 2024-10-02 17:19:18 -07:00
memtable_list.h Fix needs_flush assertion in file ingestion (#13045) 2024-10-02 17:19:18 -07:00
memtable_list_test.cc Fix MultiGet dropping memtable kv checksum corruption (#12842) 2024-08-08 13:34:11 -07:00
merge_context.h Add ContinueCallback to GetMergeOperands() (#12438) 2024-03-15 12:25:49 -07:00
merge_helper.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
merge_helper.h
merge_helper_test.cc
merge_operator.cc
merge_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
multi_cf_iterator_impl.h Support allow_unprepared_value for multi-CF iterators (#13079) 2024-10-20 20:53:08 -07:00
multi_cf_iterator_test.cc Support allow_unprepared_value for multi-CF iterators (#13079) 2024-10-20 20:53:08 -07:00
obsolete_files_test.cc Remove the force mode for EnableFileDeletions API (#12337) 2024-02-13 18:36:25 -08:00
options_file_test.cc
output_validator.cc
output_validator.h
perf_context_test.cc Add write_memtable_time to perf level kEnableWait (#12394) 2024-02-29 15:08:26 -08:00
periodic_task_scheduler.cc Remove extra semi colon from instagram/ranking/mezql/shots/parser/fast/Token.cpp 2024-03-04 06:32:50 -08:00
periodic_task_scheduler.h
periodic_task_scheduler_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
pinned_iterators_manager.h Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
plain_table_db_test.cc Prefer static_cast in place of most reinterpret_cast (#12308) 2024-02-07 10:44:11 -08:00
post_memtable_callback.h
pre_release_callback.h
prefix_test.cc
range_del_aggregator.cc Replace ScopedArenaIterator with ScopedArenaPtr<InternalIterator> (#12470) 2024-03-22 13:40:42 -07:00
range_del_aggregator.h Replace ScopedArenaIterator with ScopedArenaPtr<InternalIterator> (#12470) 2024-03-22 13:40:42 -07:00
range_del_aggregator_bench.cc
range_del_aggregator_test.cc
range_tombstone_fragmenter.cc
range_tombstone_fragmenter.h Remove stale entries from L0 files when UDT is not persisted (#13035) 2024-10-14 12:28:35 -07:00
range_tombstone_fragmenter_test.cc
read_callback.h
repair.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
repair_test.cc
seqno_time_test.cc Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
seqno_to_time_mapping.cc Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
seqno_to_time_mapping.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
snapshot_checker.h Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
snapshot_impl.cc
snapshot_impl.h
table_cache.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
table_cache.h Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
table_cache_sync_and_async.h Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
table_properties_collector.cc
table_properties_collector.h Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
table_properties_collector_test.cc Add CompactForTieringCollector to support automatically trigger compaction for tiering use case (#12760) 2024-06-18 10:51:29 -07:00
transaction_log_impl.cc Rename, deprecate LogFile and VectorLogPtr (#12695) 2024-05-28 09:24:49 -07:00
transaction_log_impl.h Rename, deprecate LogFile and VectorLogPtr (#12695) 2024-05-28 09:24:49 -07:00
trim_history_scheduler.cc
trim_history_scheduler.h
version_builder.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
version_builder.h Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
version_builder_test.cc
version_edit.cc Support ingesting SST files generated by a live DB (#12750) 2024-07-19 16:14:54 -07:00
version_edit.h Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
version_edit_handler.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
version_edit_handler.h Add some documentation for version edit handlers (#12948) 2024-08-21 10:09:10 -07:00
version_edit_test.cc
version_set.cc Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
version_set.h Optimize compaction for standalone range deletion files (#13078) 2024-10-25 09:32:14 -07:00
version_set_sync_and_async.h Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
version_set_test.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
version_util.h
wal_edit.cc
wal_edit.h
wal_edit_test.cc
wal_manager.cc Avoid unnecessary work in internal calls to GetSortedWalFiles (#12831) 2024-07-01 23:29:02 -07:00
wal_manager.h Avoid unnecessary work in internal calls to GetSortedWalFiles (#12831) 2024-07-01 23:29:02 -07:00
wal_manager_test.cc
write_batch.cc Refactor table_factory into MutableCFOptions (#13077) 2024-10-17 14:13:20 -07:00
write_batch_base.cc
write_batch_internal.h Add initial support for TimedPut API (#12419) 2024-03-14 15:44:55 -07:00
write_batch_test.cc Steps toward deprecating implicit prefix seek, related fixes (#13026) 2024-09-20 15:54:19 -07:00
write_callback.h
write_callback_test.cc Add public API WriteWithCallback to support custom callbacks (#12603) 2024-05-31 19:30:19 -07:00
write_controller.cc
write_controller.h
write_controller_test.cc
write_stall_stats.cc
write_stall_stats.h
write_thread.cc Add public API WriteWithCallback to support custom callbacks (#12603) 2024-05-31 19:30:19 -07:00
write_thread.h Add public API WriteWithCallback to support custom callbacks (#12603) 2024-05-31 19:30:19 -07:00