rocksdb/monitoring
Guido Tagliavini Ponce 2af132c341 Fix overflow bug in standard deviation computation. (#10100)
Summary:
There was an overflow bug when computing the variance in the HistogramStat class.

This manifests, for instance, when running cache_bench with default arguments. This executes 32M lookups/inserts/deletes in a block cache, and then computes (among other things) the variance of the latencies. The variance is computed as ``variance = (cur_sum_squares * cur_num - cur_sum * cur_sum) / (cur_num * cur_num)``, where ``cum_sum_squares`` is the sum of the squares of the samples, ``cur_num`` is the number of samples, and ``cur_sum`` is the sum of the samples. Because the median latency in a typical run is around 3800 nanoseconds, both the ``cur_sum_squares * cur_num`` and ``cur_sum * cur_sum`` terms overflow as uint64_t.

Pull Request resolved: https://github.com/facebook/rocksdb/pull/10100

Test Plan: Added a unit test. Run ``make -j24 histogram_test && ./histogram_test``.

Reviewed By: pdillinger

Differential Revision: D36942738

Pulled By: guidotag

fbshipit-source-id: 0af5fb9e2a297a284e8e74c24e604d302906006e
2022-06-06 13:53:47 -07:00
..
file_read_sample.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
histogram.cc Fix overflow bug in standard deviation computation. (#10100) 2022-06-06 13:53:47 -07:00
histogram.h improve-histogram-performance: remove valueIndexMap_ (#8625) 2021-10-14 14:45:20 -07:00
histogram_test.cc Fix overflow bug in standard deviation computation. (#10100) 2022-06-06 13:53:47 -07:00
histogram_windowing.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
histogram_windowing.h Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00
in_memory_stats_history.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
in_memory_stats_history.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
instrumented_mutex.cc Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
instrumented_mutex.h Remove explicit padding from CacheAlignedInstrumentedMutex (#9809) 2022-04-05 18:32:05 -07:00
iostats_context.cc Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
iostats_context_imp.h Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
iostats_context_test.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
perf_context.cc Seek parallelization (#9994) 2022-05-20 16:09:33 -07:00
perf_context_imp.h Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
perf_level.cc Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
perf_level_imp.h Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
perf_step_timer.h make PerfStepTimer struct smaller by reordering members (#7931) 2021-03-08 21:33:15 -08:00
persistent_stats_history.cc Use std::numeric_limits<> (#9954) 2022-05-05 13:08:21 -07:00
persistent_stats_history.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
statistics.cc Multi file concurrency in MultiGet using coroutines and async IO (#9968) 2022-05-19 15:36:27 -07:00
statistics.h Add support for building on s390x platform (#8962) 2021-10-22 10:13:15 -07:00
statistics_test.cc Added a default Name method to Statistics (#8918) 2021-09-17 07:25:43 -07:00
stats_history_test.cc Persist the new MANIFEST after successfully syncing the new WAL during recovery (#9922) 2022-06-01 10:52:26 -07:00
thread_status_impl.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
thread_status_updater.cc Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
thread_status_updater.h Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
thread_status_updater_debug.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
thread_status_util.cc Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
thread_status_util.h Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
thread_status_util_debug.cc Add a SystemClock class to capture the time functions of an Env (#7858) 2021-01-25 22:09:11 -08:00