rocksdb/monitoring
Zhongyi Xie 3886dddc3b force flushing stats CF to avoid holding old logs (#5509)
Summary:
WAL records RocksDB writes to all column families. When user flushes a a column family, the old WAL will not accept new writes but cannot be deleted yet because it may still contain live data for other column families. (See https://github.com/facebook/rocksdb/wiki/Write-Ahead-Log#life-cycle-of-a-wal for detailed explanation)
Because of this, if there is a column family that receive very infrequent writes and no manual flush is called for it, it could prevent a lot of WALs from being deleted. PR https://github.com/facebook/rocksdb/pull/5046 introduced persistent stats column family which is a good example of such column families. Depending on the config, it may have long intervals between writes, and user is unaware of it which makes it difficult to call manual flush for it.
This PR addresses the problem for persistent stats column family by forcing a flush for persistent stats column family when 1) another column family is flushed 2) persistent stats column family's log number is the smallest among all column families, this way persistent stats column family will  keep advancing its log number when necessary, allowing RocksDB to delete old WAL files.
Pull Request resolved: https://github.com/facebook/rocksdb/pull/5509

Differential Revision: D16045896

Pulled By: miasantreble

fbshipit-source-id: 286837b633e988417f0096ff38384742d3b40ef4
2019-07-01 11:56:43 -07:00
..
file_read_sample.h Change RocksDB License 2017-07-15 16:11:23 -07:00
histogram.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
histogram.h Scale histogram bucket size by constant factor 2017-08-21 17:10:40 -07:00
histogram_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
histogram_windowing.cc Support pragma once in all header files and cleanup some warnings (#4339) 2018-09-05 18:13:31 -07:00
histogram_windowing.h fix HistogramWindowingImpl copy-{assignment,constructor} 2017-09-01 11:11:56 -07:00
in_memory_stats_history.cc Persistent Stats: persist stats history to disk (#5046) 2019-06-17 15:21:50 -07:00
in_memory_stats_history.h Persistent Stats: persist stats history to disk (#5046) 2019-06-17 15:21:50 -07:00
instrumented_mutex.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
instrumented_mutex.h Change RocksDB License 2017-07-15 16:11:23 -07:00
iostats_context.cc Directly refernce perf_context internally. 2017-09-15 17:15:10 -07:00
iostats_context_imp.h Fix compilation errors for 32bits/LITE/ios build. (#5220) 2019-04-22 16:02:16 -07:00
iostats_context_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
perf_context.cc Introduce CPU timers for iterator seek and next (#5076) 2019-03-26 16:32:13 -07:00
perf_context_imp.h Introduce a CPU time counter in perf_context (#4741) 2018-12-20 12:03:44 -08:00
perf_level.cc Change RocksDB License 2017-07-15 16:11:23 -07:00
perf_level_imp.h Change RocksDB License 2017-07-15 16:11:23 -07:00
perf_step_timer.h Introduce a CPU time counter in perf_context (#4741) 2018-12-20 12:03:44 -08:00
persistent_stats_history.cc Persistent Stats: persist stats history to disk (#5046) 2019-06-17 15:21:50 -07:00
persistent_stats_history.h Persistent Stats: persist stats history to disk (#5046) 2019-06-17 15:21:50 -07:00
statistics.cc simplify include directive involving inttypes (#5402) 2019-06-06 13:56:07 -07:00
statistics.h Still implement StatisticsImpl::measureTime() (#5181) 2019-04-12 11:00:35 -07:00
statistics_test.cc Move test related files under util/ to test_util/ (#5377) 2019-05-30 11:25:51 -07:00
stats_history_test.cc force flushing stats CF to avoid holding old logs (#5509) 2019-07-01 11:56:43 -07:00
thread_status_impl.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
thread_status_updater.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
thread_status_updater.h Use RAII instead of pointers in cf_info_map 2017-09-28 14:26:47 -07:00
thread_status_updater_debug.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
thread_status_util.cc comment unused parameters to turn on -Wunused-parameter flag 2018-04-12 17:59:16 -07:00
thread_status_util.h Change RocksDB License 2017-07-15 16:11:23 -07:00
thread_status_util_debug.cc Change RocksDB License 2017-07-15 16:11:23 -07:00