rocksdb/monitoring
Hui Xiao 9a034801ce Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444)
Summary:
**Context/Summary:**
- Similar to https://github.com/facebook/rocksdb/pull/11288 but for user read such as `Get(), MultiGet(), DBIterator::XXX(), Verify(File)Checksum()`.
   - For this, I refactored some user-facing `MultiGet` calls in `TransactionBase` and various types of `DB` so that it does not call a user-facing `Get()` but `GetImpl()` for passing the `ReadOptions::io_activity` check (see PR conversation)
   - New user read stats breakdown are guarded by `kExceptDetailedTimers` since measurement shows they have 4-5% regression to the upstream/main.

- Misc
   - More refactoring: with https://github.com/facebook/rocksdb/pull/11288, we complete passing `ReadOptions/IOOptions` to FS level. So we can now replace the previously [added](https://github.com/facebook/rocksdb/pull/9424) `rate_limiter_priority` parameter in `RandomAccessFileReader`'s `Read/MultiRead/Prefetch()` with `IOOptions::rate_limiter_priority`
   - Also, `ReadAsync()` call time is measured in `SST_READ_MICRO` now

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11444

Test Plan:
- CI fake db crash/stress test
- Microbenchmarking

**Build** `make clean && ROCKSDB_NO_FBCODE=1 DEBUG_LEVEL=0 make -jN db_basic_bench`
- google benchmark version: 604f6fd3f4
- db_basic_bench_base: upstream
- db_basic_bench_pr: db_basic_bench_base + this PR
- asyncread_db_basic_bench_base: upstream + [db basic bench patch for IteratorNext](https://github.com/facebook/rocksdb/compare/main...hx235:rocksdb:micro_bench_async_read)
- asyncread_db_basic_bench_pr: asyncread_db_basic_bench_base + this PR

**Test**

Get
```
TEST_TMPDIR=/dev/shm ./db_basic_bench_{null_stat|base|pr} --benchmark_filter=DBGet/comp_style:0/max_data:134217728/per_key_size:256/enable_statistics:1/negative_query:0/enable_filter:0/mmap:1/threads:1 --benchmark_repetitions=1000
```

Result
```
Coming soon
```

AsyncRead
```
TEST_TMPDIR=/dev/shm ./asyncread_db_basic_bench_{base|pr} --benchmark_filter=IteratorNext/comp_style:0/max_data:134217728/per_key_size:256/enable_statistics:1/async_io:1/include_detailed_timers:0 --benchmark_repetitions=1000 > syncread_db_basic_bench_{base|pr}.out
```

Result
```
Base:
1956,1956,1968,1977,1979,1986,1988,1988,1988,1990,1991,1991,1993,1993,1993,1993,1994,1996,1997,1997,1997,1998,1999,2001,2001,2002,2004,2007,2007,2008,

PR (2.3% regression, due to measuring `SST_READ_MICRO` that wasn't measured before):
1993,2014,2016,2022,2024,2027,2027,2028,2028,2030,2031,2031,2032,2032,2038,2039,2042,2044,2044,2047,2047,2047,2048,2049,2050,2052,2052,2052,2053,2053,
```

Reviewed By: ajkr

Differential Revision: D45918925

Pulled By: hx235

fbshipit-source-id: 58a54560d9ebeb3a59b6d807639692614dad058a
2023-08-08 17:26:50 -07:00
..
file_read_sample.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
histogram.cc clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
histogram.h clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
histogram_test.cc clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
histogram_windowing.cc clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
histogram_windowing.h clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
in_memory_stats_history.cc clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
in_memory_stats_history.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
instrumented_mutex.cc Inject spurious wakeup and sleep before acquiring db mutex to expose race condition (#10291) 2022-09-12 13:55:23 -07:00
instrumented_mutex.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
iostats_context.cc clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
iostats_context_imp.h Fix build error with NIOSTATS_CONTEXT (#10506) 2022-08-23 15:00:14 -07:00
iostats_context_test.cc clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
perf_context.cc Block fetch CPU time counters in perf context (#11342) 2023-04-15 11:09:44 -07:00
perf_context_imp.h Remove ROCKSDB_SUPPORT_THREAD_LOCAL define because it's a part of C++11 (#10015) 2022-05-18 15:25:19 -07:00
perf_level.cc clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
perf_level_imp.h clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
perf_step_timer.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
persistent_stats_history.cc clang format files under monitoring/ (#10857) 2022-10-24 20:45:54 -07:00
persistent_stats_history.h Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
statistics.cc Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444) 2023-08-08 17:26:50 -07:00
statistics_impl.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
statistics_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
stats_history_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
thread_status_impl.cc Replace namespace name "rocksdb" with ROCKSDB_NAMESPACE (#6433) 2020-02-20 12:09:57 -08:00
thread_status_updater.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
thread_status_updater.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
thread_status_updater_debug.cc Replace reinterpret_cast with static_cast_with_check (#7067) 2020-07-02 19:25:41 -07:00
thread_status_util.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
thread_status_util.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
thread_status_util_debug.cc Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444) 2023-08-08 17:26:50 -07:00