Yueh-Hsuan Chiang a7d4eb2f34 Fix a bug where flush does not happen when a manual compaction is running ... Summary: Currently, when rocksdb tries to run manual compaction to refit data into a level, there's a ReFitLevel() process that requires no bg work is currently running. When RocksDB plans to ReFitLevel(), it will do the following: 1. pause scheduling new bg work. 2. wait until all bg work finished 3. do the ReFitLevel() 4. unpause scheduling new bg work. However, as it pause scheduling new bg work at step one and waiting for all bg work finished in step 2, RocksDB will stop flushing until all bg work is done (which could take a long time.) This patch fix this issue by changing the way ReFitLevel() pause the background work: 1. pause scheduling compaction. 2. wait until all bg work finished. 3. pause scheduling flush 4. do ReFitLevel() 5. unpause both flush and compaction. The major difference is that. We only pause scheduling compaction in step 1 and wait for all bg work finished in step 2. This prevent flush being blocked for a long time. Although there's a very rare case that ReFitLevel() might be in starvation in step 2, but it's less likely the case as flush typically finish very fast. Test Plan: existing test. Reviewers: anthony, IslamAbdelRahman, kradhakrishnan, sdong Reviewed By: sdong Subscribers: dhruba, leveldb Differential Revision: https://reviews.facebook.net/D55029		2016-03-04 14:24:52 -08:00
..
auto_roll_logger.cc	Redo SyncPoints for flush while rolling test	2016-02-22 21:32:19 -08:00
auto_roll_logger.h	Redo SyncPoints for flush while rolling test	2016-02-22 21:32:19 -08:00
auto_roll_logger_test.cc	Redo SyncPoints for flush while rolling test	2016-02-22 21:32:19 -08:00
builder.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
builder.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
c.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
c_test.c	Deprecate CompactionFilterV2	2015-07-17 18:59:11 +02:00
column_family.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
column_family.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
column_family_test.cc	Add test to make sure DropColumnFamily doesn't impact existing iterators	2016-02-24 10:25:38 -08:00
compact_files_test.cc	Fixed CompactFiles() spuriously failing or corrupting DB	2016-02-22 13:54:58 -08:00
compacted_db_impl.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compacted_db_impl.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compaction.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compaction.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compaction_iterator.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compaction_iterator.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compaction_iterator_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compaction_job.cc	Subcompaction boundary keys should not terminate after an empty level	2016-03-02 15:45:07 -08:00
compaction_job.h	Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage	2016-02-17 15:20:23 -08:00
compaction_job_stats_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compaction_job_test.cc	Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage	2016-02-17 15:20:23 -08:00
compaction_picker.cc	Compaction always needs to be removed from level0_compactions_in_progress_ for universal compaction	2016-03-02 21:23:28 -08:00
compaction_picker.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
compaction_picker_test.cc	Add a new compaction priority that picks file whose overlapping ratio is smallest	2016-02-11 15:59:19 -08:00
comparator_db_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
convenience.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
corruption_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
cuckoo_table_db_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_compaction_filter_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_compaction_test.cc	Add a new compaction priority that picks file whose overlapping ratio is smallest	2016-02-11 15:59:19 -08:00
db_dynamic_level_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_filesnapshot.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_impl.cc	Fix a bug where flush does not happen when a manual compaction is running	2016-03-04 14:24:52 -08:00
db_impl.h	Fix a bug where flush does not happen when a manual compaction is running	2016-03-04 14:24:52 -08:00
db_impl_debug.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_impl_experimental.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_impl_readonly.cc	Add Iterator Property rocksdb.iterator.version_number	2016-03-02 16:23:59 -08:00
db_impl_readonly.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_info_dumper.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_info_dumper.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_inplace_update_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_iter.cc	Add Iterator Property rocksdb.iterator.version_number	2016-03-02 16:23:59 -08:00
db_iter.h	Add Iterator Property rocksdb.iterator.version_number	2016-03-02 16:23:59 -08:00
db_iter_test.cc	Add Iterator Property rocksdb.iterator.version_number	2016-03-02 16:23:59 -08:00
db_log_iter_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_properties_test.cc	Add DB Property "rocksdb.current_version_number"	2016-03-01 10:55:40 -08:00
db_table_properties_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
db_tailing_iter_test.cc	Add Iterator Property rocksdb.iterator.version_number	2016-03-02 16:23:59 -08:00
db_test.cc	Rename iterator property "rocksdb.iterator.is.key.pinned" => "rocksdb.iterator.is-key-pinned"	2016-03-01 13:47:12 -08:00
db_test2.cc	Add Iterator Property rocksdb.iterator.version_number	2016-03-02 16:23:59 -08:00
db_test_util.cc	Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage	2016-02-17 15:20:23 -08:00
db_test_util.h	Introduce SstFileManager::SetMaxAllowedSpaceUsage() to cap disk space usage	2016-02-17 15:20:23 -08:00
db_universal_compaction_test.cc	Make DBTestUniversalCompaction.IncreaseUniversalCompactionNumLevels more robust	2016-02-26 11:59:31 -08:00
db_wal_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
dbformat.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
dbformat.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
dbformat_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
deletefile_test.cc	Handle concurrent manifest update and backup creation	2016-02-29 12:56:55 -08:00
event_helpers.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
event_helpers.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
experimental.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
fault_injection_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
file_indexer.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
file_indexer.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
file_indexer_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
filename.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
filename.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
filename_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
flush_job.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
flush_job.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
flush_job_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
flush_scheduler.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
flush_scheduler.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
forward_iterator.cc	Add Iterator Property rocksdb.iterator.version_number	2016-03-02 16:23:59 -08:00
forward_iterator.h	Add Iterator Property rocksdb.iterator.version_number	2016-03-02 16:23:59 -08:00
forward_iterator_bench.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
inlineskiplist.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
inlineskiplist_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
internal_stats.cc	Add DB Property "rocksdb.current_version_number"	2016-03-01 10:55:40 -08:00
internal_stats.h	Add DB Property "rocksdb.current_version_number"	2016-03-01 10:55:40 -08:00
job_context.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
listener_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
log_format.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
log_reader.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
log_reader.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
log_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
log_writer.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
log_writer.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
managed_iterator.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
managed_iterator.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
manual_compaction_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
memtable.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
memtable.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
memtable_allocator.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
memtable_allocator.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
memtable_list.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
memtable_list.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
memtable_list_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
memtablerep_bench.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
merge_context.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
merge_helper.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
merge_helper.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
merge_helper_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
merge_operator.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
merge_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
options_file_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
perf_context_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
plain_table_db_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
prefix_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
repair.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
skiplist.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
skiplist_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
slice.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
snapshot_impl.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
snapshot_impl.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
table_cache.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
table_cache.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
table_properties_collector.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
table_properties_collector.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
table_properties_collector_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
transaction_log_impl.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
transaction_log_impl.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
version_builder.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
version_builder.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
version_builder_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
version_edit.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
version_edit.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
version_edit_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
version_set.cc	Handle concurrent manifest update and backup creation	2016-02-29 12:56:55 -08:00
version_set.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
version_set_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
wal_manager.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
wal_manager.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
wal_manager_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_batch.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_batch_base.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_batch_internal.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_batch_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_callback.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_callback_test.cc	Improve write_with_callback_test to sync WAL	2016-02-16 14:04:14 -08:00
write_controller.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_controller.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_controller_test.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_thread.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
write_thread.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
writebuffer.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
xfunc_test_points.cc	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00
xfunc_test_points.h	Updated all copyright headers to the new format.	2016-02-09 15:12:00 -08:00