dolysis/rocksdb
3
0
Fork 0
mirror of https://github.com/facebook/rocksdb.git synced 2024-11-27 11:43:49 +00:00
Code Issues Packages Projects Releases Wiki Activity
rocksdb/db
History
Andrew Kryczka ee1c802675 Add delay before flush in CompactRange to avoid write stalling 
Summary:
- Refactored logic for checking write stall condition to a helper function: `GetWriteStallConditionAndCause`. Now it is decoupled from the logic for updating WriteController / stats in `RecalculateWriteStallConditions`, so we can reuse it for predicting whether write stall will occur.
- Updated `CompactRange` to first check whether the one additional immutable memtable / L0 file would cause stalling before it flushes. If so, it waits until that is no longer true.
- Updated `bg_cv_` to be signaled on `SetOptions` calls. The stall conditions `CompactRange` cares about can change when (1) flush finishes, (2) compaction finishes, or (3) options dynamically change. The cv was already signaled for (1) and (2) but not yet for (3).
Closes https://github.com/facebook/rocksdb/pull/3381

Differential Revision: D6754983

Pulled By: ajkr

fbshipit-source-id: 5613e03f1524df7192dc6ae885d40fd8f091d972
2018-02-12 15:42:47 -08:00
..
builder.cc WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
builder.h Stream 2017-11-10 09:26:24 -08:00
c.cc add support for allow_ingest_behind in C API 2018-01-08 17:26:31 -08:00
c_test.c Added save points for transactions C API 2017-09-14 14:18:59 -07:00
column_family.cc Add delay before flush in CompactRange to avoid write stalling 2018-02-12 15:42:47 -08:00
column_family.h Add delay before flush in CompactRange to avoid write stalling 2018-02-12 15:42:47 -08:00
column_family_test.cc fix DBTest.AutomaticConflictsWithManualCompaction 2018-01-16 23:12:00 -08:00
compact_files_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
compacted_db_impl.cc Fix coverity issues column_family, compaction_db/iterator 2017-10-27 11:26:42 -07:00
compacted_db_impl.h Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
compaction.cc Improve fallocate size in compaction output 2018-01-22 16:43:46 -08:00
compaction.h Change RocksDB License 2017-07-15 16:11:23 -07:00
compaction_iteration_stats.h add counter for deletion dropping optimization 2017-08-19 14:10:08 -07:00
compaction_iterator.cc WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
compaction_iterator.h Added support for differential snapshots 2017-11-01 18:56:43 -07:00
compaction_iterator_test.cc WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
compaction_job.cc WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
compaction_job.h Make DBOption compaction_readahead_size dynamic 2017-11-16 17:57:25 -08:00
compaction_job_stats_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
compaction_job_test.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
compaction_picker.cc use bottommost compression when base level is bottommost 2017-11-09 17:42:00 -08:00
compaction_picker.h fix hanging after CompactFiles with L0 overlap 2017-09-13 15:41:38 -07:00
compaction_picker_test.cc Improve fallocate size in compaction output 2018-01-22 16:43:46 -08:00
compaction_picker_universal.cc Incorrect Universal Compaction reason 2018-01-26 11:12:40 -08:00
compaction_picker_universal.h Make Universal compaction options dynamic 2017-12-11 13:27:06 -08:00
comparator_db_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
convenience.cc Delete files in multiple ranges at once 2018-01-30 13:56:39 -08:00
corruption_test.cc fix corruption_test valgrind 2017-08-11 12:29:14 -07:00
cuckoo_table_db_test.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
db_basic_test.cc Handle error return from WriteBuffer() 2018-02-05 13:59:34 -08:00
db_blob_index_test.cc fix lite build 2017-10-17 08:57:09 -07:00
db_block_cache_test.cc Use block cache to track memory usage when ReadOptions.fill_cache=false 2018-01-29 14:43:10 -08:00
db_bloom_filter_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_compaction_filter_test.cc Split CompactionFilterWithValueChange 2017-10-20 15:42:07 -07:00
db_compaction_test.cc Add delay before flush in CompactRange to avoid write stalling 2018-02-12 15:42:47 -08:00
db_dynamic_level_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_encryption_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_filesnapshot.cc log flush reason for better debugging experience 2018-02-09 12:12:43 -08:00
db_flush_test.cc Handle error return from WriteBuffer() 2018-02-05 13:59:34 -08:00
db_impl.cc Add delay before flush in CompactRange to avoid write stalling 2018-02-12 15:42:47 -08:00
db_impl.h Add delay before flush in CompactRange to avoid write stalling 2018-02-12 15:42:47 -08:00
db_impl_compaction_flush.cc Add delay before flush in CompactRange to avoid write stalling 2018-02-12 15:42:47 -08:00
db_impl_debug.cc log flush reason for better debugging experience 2018-02-09 12:12:43 -08:00
db_impl_experimental.cc Inform caller when rocksdb is stalling writes 2017-10-05 18:11:43 -07:00
db_impl_files.cc fix live WALs purged while file deletions disabled 2018-01-17 17:42:04 -08:00
db_impl_open.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
db_impl_readonly.cc WritePrepared Txn: Iterator 2017-10-09 17:15:28 -07:00
db_impl_readonly.h Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_impl_write.cc Explictly fail writes if key or value is not smaller than 4GB 2018-02-09 14:57:54 -08:00
db_info_dumper.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_info_dumper.h Change RocksDB License 2017-07-15 16:11:23 -07:00
db_inplace_update_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_io_failure_test.cc Fix LITE unit tests 2017-07-26 21:11:47 -07:00
db_iter.cc Fix use-after-free in tailing iterator with merge operator 2018-02-02 21:26:28 -08:00
db_iter.h WritePrepared Txn: Return NotSupported on iterator refresh 2017-12-18 22:29:30 -08:00
db_iter_test.cc fix Seek with lower_bound 2017-11-29 22:56:29 -08:00
db_iterator_test.cc WritePrepared Txn: update compaction_iterator_test and db_iterator_test 2018-02-06 14:12:13 -08:00
db_log_iter_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_memtable_test.cc WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
db_merge_operator_test.cc WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
db_options_test.cc fix live WALs purged while file deletions disabled 2018-01-17 17:42:04 -08:00
db_properties_test.cc Add DB::Properties::kEstimateOldestKeyTime 2017-10-23 15:27:27 -07:00
db_range_del_test.cc Fix wrong smallest key of delete range tombstones 2017-08-29 18:41:35 -07:00
db_sst_test.cc Fix DeleteScheduler::MarkAsTrash() handling existing trash 2017-12-12 18:17:13 -08:00
db_statistics_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
db_table_properties_test.cc fix deletion-triggered compaction in table builder 2017-09-28 18:17:30 -07:00
db_tailing_iter_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
db_test.cc Explictly fail writes if key or value is not smaller than 4GB 2018-02-09 14:57:54 -08:00
db_test2.cc fix DBTest2.ReadAmpBitmapLiveInCacheAfterDBClose file ID support check 2018-01-30 16:50:42 -08:00
db_test_util.cc Fix multiple build failures 2018-01-16 17:30:39 -08:00
db_test_util.h WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
db_universal_compaction_test.cc Tests for dynamic universal compaction options 2018-01-29 16:42:45 -08:00
db_wal_test.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
db_write_test.cc BlobDB: Remove the need to get sequence number per write 2017-12-15 13:27:30 -08:00
dbformat.cc WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
dbformat.h WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
dbformat_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
deletefile_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
event_helpers.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
event_helpers.h Change RocksDB License 2017-07-15 16:11:23 -07:00
experimental.cc Replace dynamic_cast<> 2017-07-28 16:27:16 -07:00
external_sst_file_basic_test.cc optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
external_sst_file_ingestion_job.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
external_sst_file_ingestion_job.h optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
external_sst_file_test.cc Upgrade Appveyor to VS2017 2018-02-01 13:57:01 -08:00
fault_injection_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
file_indexer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
file_indexer_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
filename_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_job.cc log flush reason for better debugging experience 2018-02-09 12:12:43 -08:00
flush_job.h WritePrepared Txn: Compaction/Flush 2017-10-06 10:41:53 -07:00
flush_job_test.cc Add a histogram stat for memtable flush 2017-12-15 18:57:00 -08:00
flush_scheduler.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
flush_scheduler.h Change RocksDB License 2017-07-15 16:11:23 -07:00
forward_iterator.cc Fix use-after-free in tailing iterator with merge operator 2018-02-02 21:26:28 -08:00
forward_iterator.h Fix use-after-free in tailing iterator with merge operator 2018-02-02 21:26:28 -08:00
forward_iterator_bench.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
internal_stats.cc TableProperty::oldest_key_time defaults to 0 2017-10-27 15:00:05 -07:00
internal_stats.h Add DB::Properties::kEstimateOldestKeyTime 2017-10-23 15:27:27 -07:00
job_context.h Prevent unnecessary calls to PurgeObsoleteFiles 2018-01-12 13:27:08 -08:00
listener_test.cc Incorrect Universal Compaction reason 2018-01-26 11:12:40 -08:00
log_format.h Change RocksDB License 2017-07-15 16:11:23 -07:00
log_reader.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_reader.h Suppress lint in old files 2018-01-29 12:56:42 -08:00
log_test.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
log_writer.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
log_writer.h Change RocksDB License 2017-07-15 16:11:23 -07:00
malloc_stats.cc DB::DumpSupportInfo should log all supported compression types 2018-01-23 14:44:12 -08:00
malloc_stats.h Change RocksDB License 2017-07-15 16:11:23 -07:00
managed_iterator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
managed_iterator.h Change RocksDB License 2017-07-15 16:11:23 -07:00
manual_compaction_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
memtable.cc WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
memtable.h WritePrepared Txn: Duplicate Keys, Memtable part 2018-01-31 18:57:07 -08:00
memtable_list.cc Add DB::Properties::kEstimateOldestKeyTime 2017-10-23 15:27:27 -07:00
memtable_list.h Fix Flush() keep waiting after flush finish 2018-01-18 17:45:16 -08:00
memtable_list_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_context.h Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_helper.cc WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
merge_helper.h WritePrepared Txn: Support merge operator 2018-02-09 14:57:54 -08:00
merge_helper_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_operator.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
merge_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
options_file_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
perf_context_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
pinned_iterators_manager.h Change RocksDB License 2017-07-15 16:11:23 -07:00
plain_table_db_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
pre_release_callback.h WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
prefix_test.cc fix gflags namespace 2017-12-01 10:42:05 -08:00
range_del_aggregator.cc optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
range_del_aggregator.h optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
range_del_aggregator_test.cc optimize file ingestion checks for range deletion overlap 2017-11-28 11:27:02 -08:00
read_callback.h write-prepared txn: call IsInSnapshot 2017-09-11 09:14:48 -07:00
repair.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
repair_test.cc fix file numbers after repair 2017-10-10 13:12:37 -07:00
snapshot_checker.h WritePrepared Txn: Disable GC during recovery 2017-10-18 09:11:50 -07:00
snapshot_impl.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
snapshot_impl.h Disable options_settable_test in UBSAN and fix UBSAN failure in blob_… 2018-02-07 14:42:26 -08:00
table_cache.cc Make DBOption compaction_readahead_size dynamic 2017-11-16 17:57:25 -08:00
table_cache.h Change RocksDB License 2017-07-15 16:11:23 -07:00
table_properties_collector.cc fix duplicate definition of GetEntryType() 2017-11-01 22:56:17 -07:00
table_properties_collector.h Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
table_properties_collector_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
transaction_log_impl.cc WritePrepared Txn: Refactor conf params 2017-11-10 17:28:12 -08:00
transaction_log_impl.h WritePrepared Txn: Refactor conf params 2017-11-10 17:28:12 -08:00
version_builder.cc Fix coverity issues version, write_batch 2017-12-07 11:57:36 -08:00
version_builder.h Allow DB reopen with reduced options.num_levels 2017-08-24 16:10:54 -07:00
version_builder_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_edit.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
version_edit.h Fix coverity issues version, write_batch 2017-12-07 11:57:36 -08:00
version_edit_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
version_set.cc Suppress unused warnings 2018-02-02 12:27:07 -08:00
version_set.h WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
version_set_test.cc Revert "comment out unused parameters" 2017-07-21 18:26:26 -07:00
wal_manager.cc Fix memleak when DB::DeleteFile() 2018-01-11 18:57:33 -08:00
wal_manager.h Fix memleak when DB::DeleteFile() 2018-01-11 18:57:33 -08:00
wal_manager_test.cc WritePrepared Txn: PreReleaseCallback 2017-11-30 23:50:45 -08:00
write_batch.cc Explictly fail writes if key or value is not smaller than 4GB 2018-02-09 14:57:54 -08:00
write_batch_base.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_batch_internal.h Explictly fail writes if key or value is not smaller than 4GB 2018-02-09 14:57:54 -08:00
write_batch_test.cc Enable MSVC W4 with a few exceptions. Fix warnings and bugs 2017-10-19 10:57:12 -07:00
write_callback.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_callback_test.cc WritePrepared Txn: Duplicate Keys, Txn Part 2018-02-05 18:43:24 -08:00
write_controller.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller.h Change RocksDB License 2017-07-15 16:11:23 -07:00
write_controller_test.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
write_thread.cc Fix IOError on WAL write doesn't propagate to write group follower 2017-11-28 11:42:48 -08:00
write_thread.h WritePrepared Txn: Duplicate Keys, Txn Part 2018-02-05 18:43:24 -08:00