mirror of
https://github.com/facebook/rocksdb.git
synced 2024-11-26 16:30:56 +00:00
aed7abbcca
Summary:
Running. TEST_TMPDIR=/dev/shm ./buck-out/gen/rocks/tools/rocks_db_bench --benchmarks=readwhilewriting --num=5000000 -benchmark_write_rate_limit=2000000 --threads=32
Collected esan data and reorder field. Accesses to 4th and 6th fields take majority of the access. Group them. Overall, this struct takes 10%+ of the total accesses in the program. (637773011/6107964986)
==2433831== class rocksdb::InlineSkipList
==2433831== size = 48, count = 637773011, ratio = 112412, array access = 0
==2433831== # 0: offset = 0, size = 2, count = 455137, type = i16
==2433831== # 1: offset = 2, size = 2, count = 6, type = i16
==2433831== # 2: offset = 4, size = 4, count = 182303, type = i32
==2433831== # 3: offset = 8, size = 8, count = 263953900, type = %"class.rocksdb::MemTableRep::KeyComparator"*
==2433831== # 4: offset = 16, size = 8, count = 136409, type = %"class.rocksdb::Allocator"*
==2433831== # 5: offset = 24, size = 8, count = 366628820, type = %"struct.rocksdb::InlineSkipList<const rocksdb::MemTableRep::KeyComparator &>::Node"*
==2433831== # 6: offset = 32, size = 4, count = 6280031, type = %"struct.std::atomic" = type { %"struct.std::__atomic_base" }
==2433831== # 7: offset = 40, size = 8, count = 136405, type = %"struct.rocksdb::InlineSkipList<const rocksdb::MemTableRep::KeyComparator &>::Splice"*
==2433831==EfficiencySanitizer: total struct field access count = 6107964986
Before re-ordering
[trentxintong@devbig460.frc2 ~/fbsource/fbcode]$ fgrep readwhilewriting
without-ro.log
readwhilewriting : 0.036 micros/op 27545605 ops/sec; 26.8 MB/s
(45954 of 5000000 found)
readwhilewriting : 0.036 micros/op 28024240 ops/sec; 27.2 MB/s
(43158 of 5000000 found)
readwhilewriting : 0.037 micros/op 27345145 ops/sec; 27.1 MB/s
(46725 of 5000000 found)
readwhilewriting : 0.037 micros/op 27072588 ops/sec; 27.3 MB/s
(42605 of 5000000 found)
readwhilewriting : 0.034 micros/op 29578781 ops/sec; 28.3 MB/s
(44294 of 5000000 found)
readwhilewriting : 0.035 micros/op 28528304 ops/sec; 27.7 MB/s
(44176 of 5000000 found)
readwhilewriting : 0.037 micros/op 27075497 ops/sec; 26.5 MB/s
(43763 of 5000000 found)
readwhilewriting : 0.036 micros/op 28024117 ops/sec; 27.1 MB/s
(40622 of 5000000 found)
readwhilewriting : 0.037 micros/op 27078709 ops/sec; 27.6 MB/s
(47774 of 5000000 found)
readwhilewriting : 0.034 micros/op 29020689 ops/sec; 28.1 MB/s
(45066 of 5000000 found)
AVERAGE()=27.37 MB/s
After re-ordering
[trentxintong@devbig460.frc2 ~/fbsource/fbcode]$ fgrep readwhilewriting
ro.log
readwhilewriting : 0.036 micros/op 27542409 ops/sec; 27.7 MB/s
(46163 of 5000000 found)
readwhilewriting : 0.036 micros/op 28021148 ops/sec; 28.2 MB/s
(46155 of 5000000 found)
readwhilewriting : 0.036 micros/op 28021035 ops/sec; 27.3 MB/s
(44039 of 5000000 found)
readwhilewriting : 0.036 micros/op 27538659 ops/sec; 27.5 MB/s
(46781 of 5000000 found)
readwhilewriting : 0.036 micros/op 28028604 ops/sec; 27.6 MB/s
(44689 of 5000000 found)
readwhilewriting : 0.036 micros/op 27541452 ops/sec; 27.3 MB/s
(43156 of 5000000 found)
readwhilewriting : 0.034 micros/op 29041338 ops/sec; 28.8 MB/s
(44895 of 5000000 found)
readwhilewriting : 0.036 micros/op 27784974 ops/sec; 26.3 MB/s
(39963 of 5000000 found)
readwhilewriting : 0.036 micros/op 27538892 ops/sec; 28.1 MB/s
(46570 of 5000000 found)
readwhilewriting : 0.038 micros/op 26622473 ops/sec; 27.0 MB/s
(43236 of 5000000 found)
AVERAGE()=27.58 MB/s
Closes https://github.com/facebook/rocksdb/pull/3855
Reviewed By: siying
Differential Revision: D8048781
Pulled By: trentxintong
fbshipit-source-id: bc9807a9845e2a92cb171ce1ecb5a2c8a51f1481
|
||
|---|---|---|
| buckifier | ||
| build_tools | ||
| cache | ||
| cmake | ||
| coverage | ||
| db | ||
| docs | ||
| env | ||
| examples | ||
| hdfs | ||
| include/rocksdb | ||
| java | ||
| memtable | ||
| monitoring | ||
| options | ||
| port | ||
| table | ||
| third-party | ||
| tools | ||
| util | ||
| utilities | ||
| .clang-format | ||
| .gitignore | ||
| .travis.yml | ||
| appveyor.yml | ||
| AUTHORS | ||
| CMakeLists.txt | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| DEFAULT_OPTIONS_HISTORY.md | ||
| DUMP_FORMAT.md | ||
| HISTORY.md | ||
| INSTALL.md | ||
| issue_template.md | ||
| LANGUAGE-BINDINGS.md | ||
| LICENSE.Apache | ||
| LICENSE.leveldb | ||
| Makefile | ||
| README.md | ||
| ROCKSDB_LITE.md | ||
| src.mk | ||
| TARGETS | ||
| thirdparty.inc | ||
| USERS.md | ||
| Vagrantfile | ||
| WINDOWS_PORT.md | ||
RocksDB: A Persistent Key-Value Store for Flash and RAM Storage
RocksDB is developed and maintained by Facebook Database Engineering Team. It is built on earlier work on LevelDB by Sanjay Ghemawat (sanjay@google.com) and Jeff Dean (jeff@google.com)
This code is a library that forms the core building block for a fast key value server, especially suited for storing data on flash drives. It has a Log-Structured-Merge-Database (LSM) design with flexible tradeoffs between Write-Amplification-Factor (WAF), Read-Amplification-Factor (RAF) and Space-Amplification-Factor (SAF). It has multi-threaded compactions, making it specially suitable for storing multiple terabytes of data in a single database.
Start with example usage here: https://github.com/facebook/rocksdb/tree/master/examples
See the github wiki for more explanation.
The public interface is in include/. Callers should not include or
rely on the details of any other header files in this package. Those
internal APIs may be changed without warning.
Design discussions are conducted in https://www.facebook.com/groups/rocksdb.dev/
License
RocksDB is dual-licensed under both the GPLv2 (found in the COPYING file in the root directory) and Apache 2.0 License (found in the LICENSE.Apache file in the root directory). You may select, at your option, one of the above-listed licenses.