rocksdb/db/blob
Hui Xiao 9a034801ce Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444)
Summary:
**Context/Summary:**
- Similar to https://github.com/facebook/rocksdb/pull/11288 but for user read such as `Get(), MultiGet(), DBIterator::XXX(), Verify(File)Checksum()`.
   - For this, I refactored some user-facing `MultiGet` calls in `TransactionBase` and various types of `DB` so that it does not call a user-facing `Get()` but `GetImpl()` for passing the `ReadOptions::io_activity` check (see PR conversation)
   - New user read stats breakdown are guarded by `kExceptDetailedTimers` since measurement shows they have 4-5% regression to the upstream/main.

- Misc
   - More refactoring: with https://github.com/facebook/rocksdb/pull/11288, we complete passing `ReadOptions/IOOptions` to FS level. So we can now replace the previously [added](https://github.com/facebook/rocksdb/pull/9424) `rate_limiter_priority` parameter in `RandomAccessFileReader`'s `Read/MultiRead/Prefetch()` with `IOOptions::rate_limiter_priority`
   - Also, `ReadAsync()` call time is measured in `SST_READ_MICRO` now

Pull Request resolved: https://github.com/facebook/rocksdb/pull/11444

Test Plan:
- CI fake db crash/stress test
- Microbenchmarking

**Build** `make clean && ROCKSDB_NO_FBCODE=1 DEBUG_LEVEL=0 make -jN db_basic_bench`
- google benchmark version: 604f6fd3f4
- db_basic_bench_base: upstream
- db_basic_bench_pr: db_basic_bench_base + this PR
- asyncread_db_basic_bench_base: upstream + [db basic bench patch for IteratorNext](https://github.com/facebook/rocksdb/compare/main...hx235:rocksdb:micro_bench_async_read)
- asyncread_db_basic_bench_pr: asyncread_db_basic_bench_base + this PR

**Test**

Get
```
TEST_TMPDIR=/dev/shm ./db_basic_bench_{null_stat|base|pr} --benchmark_filter=DBGet/comp_style:0/max_data:134217728/per_key_size:256/enable_statistics:1/negative_query:0/enable_filter:0/mmap:1/threads:1 --benchmark_repetitions=1000
```

Result
```
Coming soon
```

AsyncRead
```
TEST_TMPDIR=/dev/shm ./asyncread_db_basic_bench_{base|pr} --benchmark_filter=IteratorNext/comp_style:0/max_data:134217728/per_key_size:256/enable_statistics:1/async_io:1/include_detailed_timers:0 --benchmark_repetitions=1000 > syncread_db_basic_bench_{base|pr}.out
```

Result
```
Base:
1956,1956,1968,1977,1979,1986,1988,1988,1988,1990,1991,1991,1993,1993,1993,1993,1994,1996,1997,1997,1997,1998,1999,2001,2001,2002,2004,2007,2007,2008,

PR (2.3% regression, due to measuring `SST_READ_MICRO` that wasn't measured before):
1993,2014,2016,2022,2024,2027,2027,2028,2028,2030,2031,2031,2032,2032,2038,2039,2042,2044,2044,2047,2047,2047,2048,2049,2050,2052,2052,2052,2053,2053,
```

Reviewed By: ajkr

Differential Revision: D45918925

Pulled By: hx235

fbshipit-source-id: 58a54560d9ebeb3a59b6d807639692614dad058a
2023-08-08 17:26:50 -07:00
..
blob_constants.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_contents.cc Major Cache refactoring, CPU efficiency improvement (#10975) 2023-01-11 14:20:40 -08:00
blob_contents.h Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
blob_counting_iterator.h Refactor AddRangeDels() + consider range tombstone during compaction file cutting (#11113) 2023-02-22 12:28:18 -08:00
blob_counting_iterator_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_fetcher.cc Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
blob_fetcher.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
blob_file_addition.cc Print blob file checksums as hex (#8437) 2021-06-22 09:49:44 -07:00
blob_file_addition.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_addition_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_file_builder.cc Changes and enhancements to compression stats, thresholds (#11388) 2023-04-21 21:57:40 -07:00
blob_file_builder.h Support prepopulating/warming the blob cache (#10298) 2022-07-17 07:13:59 -07:00
blob_file_builder_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_file_cache.cc Improve memory efficiency of many OptimisticTransactionDBs (#11439) 2023-05-24 11:57:15 -07:00
blob_file_cache.h Improve memory efficiency of many OptimisticTransactionDBs (#11439) 2023-05-24 11:57:15 -07:00
blob_file_cache_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
blob_file_completion_callback.h Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
blob_file_garbage.cc Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage.h Move BlobDB related files under db/ to db/blob/ (#6519) 2020-03-12 11:00:56 -07:00
blob_file_garbage_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_file_meta.cc Add BlobMetaData retrieval methods (#8273) 2021-06-28 08:13:29 -07:00
blob_file_meta.h Add BlobMetaData retrieval methods (#8273) 2021-06-28 08:13:29 -07:00
blob_file_reader.cc Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444) 2023-08-08 17:26:50 -07:00
blob_file_reader.h Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444) 2023-08-08 17:26:50 -07:00
blob_file_reader_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
blob_garbage_meter.cc Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_garbage_meter.h Add a class for measuring the amount of garbage generated during compaction (#8426) 2021-06-21 22:25:30 -07:00
blob_garbage_meter_test.cc Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
blob_index.h Remove local static string (#8103) 2022-08-05 23:03:51 -07:00
blob_log_format.cc Remove local static string (#8103) 2022-08-05 23:03:51 -07:00
blob_log_format.h Run clang-format on db/blob/ (#10856) 2022-10-24 16:00:32 -07:00
blob_log_sequential_reader.cc Group rocksdb.sst.read.micros stat by different user read IOActivity + misc (#11444) 2023-08-08 17:26:50 -07:00
blob_log_sequential_reader.h Fix a issue with initializing blob header buffer (#8537) 2021-08-02 17:15:06 -07:00
blob_log_writer.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
blob_log_writer.h Use SystemClock* instead of std::shared_ptr<SystemClock> in lower level routines (#8033) 2021-03-15 04:34:11 -07:00
blob_read_request.h Enable blob caching for MultiGetBlob in RocksDB (#10272) 2022-06-30 13:24:35 -07:00
blob_source.cc Change internal headers with duplicate names (#11408) 2023-05-17 11:27:09 -07:00
blob_source.h Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
blob_source_test.cc Group rocksdb.sst.read.micros stat by IOActivity flush and compaction (#11288) 2023-04-21 09:07:18 -07:00
db_blob_basic_test.cc Fix backward iteration issue when user defined timestamp is enabled in BlobDB (#11258) 2023-03-01 13:28:54 -08:00
db_blob_compaction_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_blob_corruption_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
db_blob_index_test.cc Remove RocksDB LITE (#11147) 2023-01-27 13:14:19 -08:00
prefetch_buffer_collection.cc Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00
prefetch_buffer_collection.h Support readahead during compaction for blob files (#9187) 2021-11-19 17:53:47 -08:00