mirror of
https://github.com/facebook/rocksdb.git
synced 2024-11-26 16:30:56 +00:00
bf98dcf9a8
Summary: The original goal is to propagate failures from `GetContext::SaveValue()` -> `GetContext::GetBlobValue()` -> `BlobFetcher::FetchBlob()` up to the user. This call sequence happens when a merge chain ends with a base value in a blob file. There's also fixes for bugs encountered along the way where non-ok statuses were ignored/overwritten, and a bit of plumbing work for functions that had no capability to return a status. Pull Request resolved: https://github.com/facebook/rocksdb/pull/12462 Test Plan: A repro command ``` db=/dev/shm/dbstress_db ; exp=/dev/shm/dbstress_exp ; rm -rf $db $exp ; mkdir -p $db $exp ./db_stress \ --clear_column_family_one_in=0 \ --test_batches_snapshots=0 \ --write_fault_one_in=0 \ --use_put_entity_one_in=0 \ --prefixpercent=0 \ --read_fault_one_in=0 \ --readpercent=0 \ --reopen=0 \ --set_options_one_in=10000 \ --delpercent=0 \ --delrangepercent=0 \ --open_metadata_write_fault_one_in=0 \ --open_read_fault_one_in=0 \ --open_write_fault_one_in=0 \ --destroy_db_initially=0 \ --ingest_external_file_one_in=0 \ --iterpercent=0 \ --nooverwritepercent=0 \ --db=$db \ --enable_blob_files=1 \ --expected_values_dir=$exp \ --max_background_compactions=20 \ --max_bytes_for_level_base=2097152 \ --max_key=100000 \ --min_blob_size=0 \ --open_files=-1 \ --ops_per_thread=100000000 \ --prefix_size=-1 \ --target_file_size_base=524288 \ --use_merge=1 \ --value_size_mult=32 \ --write_buffer_size=524288 \ --writepercent=100 ``` It used to fail like: ``` ... frame https://github.com/facebook/rocksdb/issues/9: 0x00007fc63903bc93 libc.so.6`__GI___assert_fail(assertion="HasDefaultColumn(columns)", file="fbcode/internal_repo_rocksdb/repo/db/wide/wide_columns_helper.h", line=33, function="static const rocksdb::Slice &rocksdb::WideColumnsHelper::GetDefaultColumn(const rocksdb::WideColumns &)") at assert.c:101:3 frame https://github.com/facebook/rocksdb/issues/10: 0x00000000006f7e92 db_stress`rocksdb::Version::Get(rocksdb::ReadOptions const&, rocksdb::LookupKey const&, rocksdb::PinnableSlice*, rocksdb::PinnableWideColumns*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>*, rocksdb::Status*, rocksdb::MergeContext*, unsigned long*, rocksdb::PinnedIteratorsManager*, bool*, bool*, unsigned long*, rocksdb::ReadCallback*, bool*, bool) [inlined] rocksdb::WideColumnsHelper::GetDefaultColumn(columns=size=0) at wide_columns_helper.h:33 frame https://github.com/facebook/rocksdb/issues/11: 0x00000000006f7e76 db_stress`rocksdb::Version::Get(this=0x00007fc5ec763000, read_options=<unavailable>, k=<unavailable>, value=0x0000000000000000, columns=0x00007fc6035fd1d8, timestamp=<unavailable>, status=0x00007fc6035fd250, merge_context=0x00007fc6035fce40, max_covering_tombstone_seq=0x00007fc6035fce90, pinned_iters_mgr=0x00007fc6035fcdf0, value_found=0x0000000000000000, key_exists=0x0000000000000000, seq=0x0000000000000000, callback=0x0000000000000000, is_blob=0x0000000000000000, do_merge=<unavailable>) at version_set.cc:2492 frame https://github.com/facebook/rocksdb/issues/12: 0x000000000051e245 db_stress`rocksdb::DBImpl::GetImpl(this=0x00007fc637a86000, read_options=0x00007fc6035fcf60, key=<unavailable>, get_impl_options=0x00007fc6035fd000) at db_impl.cc:2408 frame https://github.com/facebook/rocksdb/issues/13: 0x000000000050cec2 db_stress`rocksdb::DBImpl::GetEntity(this=0x00007fc637a86000, _read_options=<unavailable>, column_family=<unavailable>, key=0x00007fc6035fd3c8, columns=0x00007fc6035fd1d8) at db_impl.cc:2109 frame https://github.com/facebook/rocksdb/issues/14: 0x000000000074f688 db_stress`rocksdb::(anonymous namespace)::MemTableInserter::MergeCF(this=0x00007fc6035fd450, column_family_id=2, key=0x00007fc6035fd3c8, value=0x00007fc6035fd3a0) at write_batch.cc:2656 frame https://github.com/facebook/rocksdb/issues/15: 0x00000000007476fc db_stress`rocksdb::WriteBatchInternal::Iterate(wb=0x00007fc6035fe698, handler=0x00007fc6035fd450, begin=12, end=<unavailable>) at write_batch.cc:607 frame https://github.com/facebook/rocksdb/issues/16: 0x000000000074d7dd db_stress`rocksdb::WriteBatchInternal::InsertInto(rocksdb::WriteThread::WriteGroup&, unsigned long, rocksdb::ColumnFamilyMemTables*, rocksdb::FlushScheduler*, rocksdb::TrimHistoryScheduler*, bool, unsigned long, rocksdb::DB*, bool, bool, bool) [inlined] rocksdb::WriteBatch::Iterate(this=<unavailable>, handler=0x00007fc6035fd450) const at write_batch.cc:505 frame https://github.com/facebook/rocksdb/issues/17: 0x000000000074d77b db_stress`rocksdb::WriteBatchInternal::InsertInto(write_group=<unavailable>, sequence=<unavailable>, memtables=<unavailable>, flush_scheduler=<unavailable>, trim_history_scheduler=<unavailable>, ignore_missing_column_families=<unavailable>, recovery_log_number=0, db=0x00007fc637a86000, concurrent_memtable_writes=<unavailable>, seq_per_batch=false, batch_per_txn=<unavailable>) at write_batch.cc:3084 frame https://github.com/facebook/rocksdb/issues/18: 0x0000000000631d77 db_stress`rocksdb::DBImpl::PipelinedWriteImpl(this=0x00007fc637a86000, write_options=<unavailable>, my_batch=0x00007fc6035fe698, callback=0x0000000000000000, log_used=<unavailable>, log_ref=0, disable_memtable=<unavailable>, seq_used=0x0000000000000000) at db_impl_write.cc:807 frame https://github.com/facebook/rocksdb/issues/19: 0x000000000062ceeb db_stress`rocksdb::DBImpl::WriteImpl(this=<unavailable>, write_options=<unavailable>, my_batch=0x00007fc6035fe698, callback=0x0000000000000000, log_used=<unavailable>, log_ref=0, disable_memtable=<unavailable>, seq_used=0x0000000000000000, batch_cnt=0, pre_release_callback=0x0000000000000000, post_memtable_callback=0x0000000000000000) at db_impl_write.cc:312 frame https://github.com/facebook/rocksdb/issues/20: 0x000000000062c8ec db_stress`rocksdb::DBImpl::Write(this=0x00007fc637a86000, write_options=0x00007fc6035feca8, my_batch=0x00007fc6035fe698) at db_impl_write.cc:157 frame https://github.com/facebook/rocksdb/issues/21: 0x000000000062b847 db_stress`rocksdb::DB::Merge(this=0x00007fc637a86000, opt=0x00007fc6035feca8, column_family=0x00007fc6370bf140, key=0x00007fc6035fe8d8, value=0x00007fc6035fe830) at db_impl_write.cc:2544 frame https://github.com/facebook/rocksdb/issues/22: 0x000000000062b6ef db_stress`rocksdb::DBImpl::Merge(this=0x00007fc637a86000, o=<unavailable>, column_family=0x00007fc6370bf140, key=0x00007fc6035fe8d8, val=0x00007fc6035fe830) at db_impl_write.cc:72 frame https://github.com/facebook/rocksdb/issues/23: 0x00000000004d6397 db_stress`rocksdb::NonBatchedOpsStressTest::TestPut(this=0x00007fc637041000, thread=0x00007fc6370dbc00, write_opts=0x00007fc6035feca8, read_opts=0x00007fc6035fe9c8, rand_column_families=<unavailable>, rand_keys=size=1, value={P\xe9_\x03\xc6\x7f\0\0}) at no_batched_ops_stress.cc:1317 frame https://github.com/facebook/rocksdb/issues/24: 0x000000000049361d db_stress`rocksdb::StressTest::OperateDb(this=0x00007fc637041000, thread=0x00007fc6370dbc00) at db_stress_test_base.cc:1148 ... ``` Reviewed By: ltamasi Differential Revision: D55157795 Pulled By: ajkr fbshipit-source-id: 5f7c1380ead5794c29d41680028e34b839744764 |
||
|---|---|---|
| .. | ||
| blob | ||
| compaction | ||
| db_impl | ||
| wide | ||
| arena_wrapped_db_iter.cc | ||
| arena_wrapped_db_iter.h | ||
| builder.cc | ||
| builder.h | ||
| c.cc | ||
| c_test.c | ||
| column_family.cc | ||
| column_family.h | ||
| column_family_test.cc | ||
| compact_files_test.cc | ||
| comparator_db_test.cc | ||
| convenience.cc | ||
| convenience_impl.h | ||
| corruption_test.cc | ||
| cuckoo_table_db_test.cc | ||
| db_basic_test.cc | ||
| db_block_cache_test.cc | ||
| db_bloom_filter_test.cc | ||
| db_clip_test.cc | ||
| db_compaction_filter_test.cc | ||
| db_compaction_test.cc | ||
| db_dynamic_level_test.cc | ||
| db_encryption_test.cc | ||
| db_filesnapshot.cc | ||
| db_flush_test.cc | ||
| db_info_dumper.cc | ||
| db_info_dumper.h | ||
| db_inplace_update_test.cc | ||
| db_io_failure_test.cc | ||
| db_iter.cc | ||
| db_iter.h | ||
| db_iter_stress_test.cc | ||
| db_iter_test.cc | ||
| db_iterator_test.cc | ||
| db_kv_checksum_test.cc | ||
| db_log_iter_test.cc | ||
| db_logical_block_size_cache_test.cc | ||
| db_memtable_test.cc | ||
| db_merge_operand_test.cc | ||
| db_merge_operator_test.cc | ||
| db_options_test.cc | ||
| db_properties_test.cc | ||
| db_range_del_test.cc | ||
| db_rate_limiter_test.cc | ||
| db_readonly_with_timestamp_test.cc | ||
| db_secondary_test.cc | ||
| db_sst_test.cc | ||
| db_statistics_test.cc | ||
| db_table_properties_test.cc | ||
| db_tailing_iter_test.cc | ||
| db_test.cc | ||
| db_test2.cc | ||
| db_test_util.cc | ||
| db_test_util.h | ||
| db_universal_compaction_test.cc | ||
| db_wal_test.cc | ||
| db_with_timestamp_basic_test.cc | ||
| db_with_timestamp_compaction_test.cc | ||
| db_with_timestamp_test_util.cc | ||
| db_with_timestamp_test_util.h | ||
| db_write_buffer_manager_test.cc | ||
| db_write_test.cc | ||
| dbformat.cc | ||
| dbformat.h | ||
| dbformat_test.cc | ||
| deletefile_test.cc | ||
| error_handler.cc | ||
| error_handler.h | ||
| error_handler_fs_test.cc | ||
| event_helpers.cc | ||
| event_helpers.h | ||
| experimental.cc | ||
| external_sst_file_basic_test.cc | ||
| external_sst_file_ingestion_job.cc | ||
| external_sst_file_ingestion_job.h | ||
| external_sst_file_test.cc | ||
| fault_injection_test.cc | ||
| file_indexer.cc | ||
| file_indexer.h | ||
| file_indexer_test.cc | ||
| filename_test.cc | ||
| flush_job.cc | ||
| flush_job.h | ||
| flush_job_test.cc | ||
| flush_scheduler.cc | ||
| flush_scheduler.h | ||
| forward_iterator.cc | ||
| forward_iterator.h | ||
| forward_iterator_bench.cc | ||
| history_trimming_iterator.h | ||
| import_column_family_job.cc | ||
| import_column_family_job.h | ||
| import_column_family_test.cc | ||
| internal_stats.cc | ||
| internal_stats.h | ||
| job_context.h | ||
| kv_checksum.h | ||
| listener_test.cc | ||
| log_format.h | ||
| log_reader.cc | ||
| log_reader.h | ||
| log_test.cc | ||
| log_writer.cc | ||
| log_writer.h | ||
| logs_with_prep_tracker.cc | ||
| logs_with_prep_tracker.h | ||
| lookup_key.h | ||
| malloc_stats.cc | ||
| malloc_stats.h | ||
| manual_compaction_test.cc | ||
| memtable.cc | ||
| memtable.h | ||
| memtable_list.cc | ||
| memtable_list.h | ||
| memtable_list_test.cc | ||
| merge_context.h | ||
| merge_helper.cc | ||
| merge_helper.h | ||
| merge_helper_test.cc | ||
| merge_operator.cc | ||
| merge_test.cc | ||
| multi_cf_iterator.cc | ||
| multi_cf_iterator.h | ||
| multi_cf_iterator_test.cc | ||
| obsolete_files_test.cc | ||
| options_file_test.cc | ||
| output_validator.cc | ||
| output_validator.h | ||
| perf_context_test.cc | ||
| periodic_task_scheduler.cc | ||
| periodic_task_scheduler.h | ||
| periodic_task_scheduler_test.cc | ||
| pinned_iterators_manager.h | ||
| plain_table_db_test.cc | ||
| post_memtable_callback.h | ||
| pre_release_callback.h | ||
| prefix_test.cc | ||
| range_del_aggregator.cc | ||
| range_del_aggregator.h | ||
| range_del_aggregator_bench.cc | ||
| range_del_aggregator_test.cc | ||
| range_tombstone_fragmenter.cc | ||
| range_tombstone_fragmenter.h | ||
| range_tombstone_fragmenter_test.cc | ||
| read_callback.h | ||
| repair.cc | ||
| repair_test.cc | ||
| seqno_time_test.cc | ||
| seqno_to_time_mapping.cc | ||
| seqno_to_time_mapping.h | ||
| snapshot_checker.h | ||
| snapshot_impl.cc | ||
| snapshot_impl.h | ||
| table_cache.cc | ||
| table_cache.h | ||
| table_cache_sync_and_async.h | ||
| table_properties_collector.cc | ||
| table_properties_collector.h | ||
| table_properties_collector_test.cc | ||
| transaction_log_impl.cc | ||
| transaction_log_impl.h | ||
| trim_history_scheduler.cc | ||
| trim_history_scheduler.h | ||
| version_builder.cc | ||
| version_builder.h | ||
| version_builder_test.cc | ||
| version_edit.cc | ||
| version_edit.h | ||
| version_edit_handler.cc | ||
| version_edit_handler.h | ||
| version_edit_test.cc | ||
| version_set.cc | ||
| version_set.h | ||
| version_set_sync_and_async.h | ||
| version_set_test.cc | ||
| version_util.h | ||
| wal_edit.cc | ||
| wal_edit.h | ||
| wal_edit_test.cc | ||
| wal_manager.cc | ||
| wal_manager.h | ||
| wal_manager_test.cc | ||
| write_batch.cc | ||
| write_batch_base.cc | ||
| write_batch_internal.h | ||
| write_batch_test.cc | ||
| write_callback.h | ||
| write_callback_test.cc | ||
| write_controller.cc | ||
| write_controller.h | ||
| write_controller_test.cc | ||
| write_stall_stats.cc | ||
| write_stall_stats.h | ||
| write_thread.cc | ||
| write_thread.h | ||