dolysis
/
rocksdb
mirror of https://github.com/facebook/rocksdb.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
rocksdb/port
History
Peter Dillinger e34087c524 Add a temporary hook for custom yielding in long-running op (#13103) 
Summary:
This is a simplified version of https://github.com/facebook/rocksdb/issues/13096, which called for a way to hook into long-running loops completely within RocksDB to change their thread priority (or similar). The current prime hook point is `DBIter::FindNextUserEntryInternal` likely because of iterating over tombstones.

This is implemented using the weak symbol hack for ease of back-porting/patching, and while we get to know potential future requirements better for integration into the public API. (Consider potential relationships to `Env::GetThreadStatusUpdater()` and `TransactionDBMutexFactory`.)

Pull Request resolved: https://github.com/facebook/rocksdb/pull/13103

Test Plan:
Performance validated with db_bench and DEBUG_LEVEL=0: `./db_bench --benchmarks=fillseq,deleterandom,readseq[-X100] --value_size=1 --num=1000000`

No consistent difference seen; variances likely in how DB / executable / memory were laid out.

```
With an empty hook:
readseq [AVG    100 runs] : 1753018 (± 8850) ops/sec;   28.4 (± 0.1) MB/sec
readseq [MEDIAN 100 runs] : 1763746 ops/sec;   28.6 MB/sec
(recompile)
readseq [AVG    100 runs] : 1789019 (± 10260) ops/sec;   29.0 (± 0.2) MB/sec
readseq [MEDIAN 100 runs] : 1801849 ops/sec;   29.2 MB/sec

Base:
readseq [AVG    100 runs] : 1772196 (± 8240) ops/sec;   28.7 (± 0.1) MB/sec
readseq [MEDIAN 100 runs] : 1780453 ops/sec;   28.9 MB/sec
(recompile)
readseq [AVG    100 runs] : 1777637 (± 7613) ops/sec;   28.8 (± 0.1) MB/sec
readseq [MEDIAN 100 runs] : 1786657 ops/sec;   29.0 MB/sec

With a functional hook (count number of calls into it):
readseq [AVG    100 runs] : 1796733 (± 8854) ops/sec;   29.1 (± 0.1) MB/sec
readseq [MEDIAN 100 runs] : 1804690 ops/sec;   29.3 MB/sec
RocksDbThreadYield: 126915800
(recompile)
readseq [AVG    100 runs] : 1775371 (± 10529) ops/sec;   28.8 (± 0.2) MB/sec
readseq [MEDIAN 100 runs] : 1789046 ops/sec;   29.0 MB/sec
RocksDbThreadYield: 126977000

Base:
readseq [AVG    100 runs] : 1773071 (± 10657) ops/sec;   28.7 (± 0.2) MB/sec
readseq [MEDIAN 100 runs] : 1783414 ops/sec;   28.9 MB/sec
(recompile)
readseq [AVG    100 runs] : 1750852 (± 10184) ops/sec;   28.4 (± 0.2) MB/sec
readseq [MEDIAN 100 runs] : 1763587 ops/sec;   28.6 MB/sec
```

Reviewed By: george-reynya

Differential Revision: D65235379

Pulled By: pdillinger

fbshipit-source-id: 7829e4cc25a56d4c1801b8adf9c7f7aa49ab7aca
 		2024-10-30 20:37:28 -07:00
..
win Fix CondVar::TimedWait for Windows (#12815) 2024-07-08 21:38:21 -07:00
README reverting disastrous MOE commit, returning to r21 2011-04-19 23:11:15 +00:00
jemalloc_helper.h Fix deprecated dynamic exception in infra_asic_fpga/validation/freya/bmtc/ssp/src/gcc-arm-none-eabi-9-2019-q4-major/arm-none-eabi/include/c++/9.2.1/tr1/special_function_util.h +5 2024-06-24 09:30:20 -07:00
lang.h Remove extra semi colon from internal_repo_rocksdb/repo/port/lang.h 2024-01-23 09:41:29 -08:00
likely.h clang format files under port/ (#10849) 2022-10-24 16:56:01 -07:00
malloc.h Charge block cache for cache internal usage (#5797) 2019-09-16 15:26:21 -07:00
mmap.cc Improve / refactor anonymous mmap capabilities (#10810) 2022-10-17 17:10:16 -07:00
mmap.h More minor HCC refactoring + typed mmap (#11670) 2023-08-07 12:20:23 -07:00
port.h Add a temporary hook for custom yielding in long-running op (#13103) 2024-10-30 20:37:28 -07:00
port_dirent.h clang format files under port/ (#10849) 2022-10-24 16:56:01 -07:00
port_example.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
port_posix.cc Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
port_posix.h Remove unnecessary, confusing 'extern' (#12300) 2024-01-29 10:38:08 -08:00
stack_trace.cc Fix build on FreeBSD (#12714) 2024-05-30 17:48:17 -07:00
stack_trace.h Print stack traces on frozen tests in CI (#10828) 2022-10-18 00:35:35 -07:00
sys_time.h clang format files under port/ (#10849) 2022-10-24 16:56:01 -07:00
util_logger.h Use EnvLogger instead of PosixLogger (#10436) 2022-08-01 14:37:18 -07:00
xpress.h Change RocksDB License 2017-07-15 16:11:23 -07:00

README 

This directory contains interfaces and implementations that isolate the
rest of the package from platform details.

Code in the rest of the package includes "port.h" from this directory.
"port.h" in turn includes a platform specific "port_<platform>.h" file
that provides the platform specific implementation.

See port_posix.h for an example of what must be provided in a platform
specific header file.