Commit graph

5 commits

Author SHA1 Message Date
Islam AbdelRahman 8a1a603fdb Eliminate std::deque initialization while iterating over merge operands
Summary:
This patch is similar to D52563, When we iterate over a DB with merge operands we keep creating std::queue to store the operands, optimize this by reusing merge_operands_ data member

Before the patch

```
./db_bench --benchmarks="mergerandom,readseq,readseq,readseq,readseq" --db="/dev/shm/bench_merge_memcpy_on_the_fly/" --merge_operator="put" --merge_keys=10000 --num=10000

DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
mergerandom  :       3.757 micros/op 266141 ops/sec;   29.4 MB/s ( updates:10000)
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.413 micros/op 2423538 ops/sec;  268.1 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.451 micros/op 2219071 ops/sec;  245.5 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.420 micros/op 2382039 ops/sec;  263.5 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.408 micros/op 2452017 ops/sec;  271.3 MB/s

DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
mergerandom  :       3.947 micros/op 253376 ops/sec;   28.0 MB/s ( updates:10000)
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.441 micros/op 2266473 ops/sec;  250.7 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.471 micros/op 2122033 ops/sec;  234.8 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.440 micros/op 2271407 ops/sec;  251.3 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.429 micros/op 2331471 ops/sec;  257.9 MB/s
```

with the patch

```
./db_bench --benchmarks="mergerandom,readseq,readseq,readseq,readseq" --db="/dev/shm/bench_merge_memcpy_on_the_fly/" --merge_operator="put" --merge_keys=10000 --num=10000

DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
mergerandom  :       4.080 micros/op 245092 ops/sec;   27.1 MB/s ( updates:10000)
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.308 micros/op 3241843 ops/sec;  358.6 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.312 micros/op 3200408 ops/sec;  354.0 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.332 micros/op 3013962 ops/sec;  333.4 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.300 micros/op 3328017 ops/sec;  368.2 MB/s

DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
mergerandom  :       3.973 micros/op 251705 ops/sec;   27.8 MB/s ( updates:10000)
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.320 micros/op 3123752 ops/sec;  345.6 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.335 micros/op 2986641 ops/sec;  330.4 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.339 micros/op 2950047 ops/sec;  326.4 MB/s
DB path: [/dev/shm/bench_merge_memcpy_on_the_fly/]
readseq      :       0.319 micros/op 3131565 ops/sec;  346.4 MB/s
```

Test Plan: make check -j64

Reviewers: yhchiang, andrewkr, sdong

Reviewed By: sdong

Subscribers: andrewkr, dhruba

Differential Revision: https://reviews.facebook.net/D56031
2016-04-01 15:48:55 -07:00
Baraa Hamodi 21e95811d1 Updated all copyright headers to the new format. 2016-02-09 15:12:00 -08:00
agiardullo 711465ccec API to fetch from both a WriteBatchWithIndex and the db
Summary:
Added a couple functions to WriteBatchWithIndex to make it easier to query the value of a key including reading pending writes from a batch.  (This is needed for transactions).

I created write_batch_with_index_internal.h to use to store an internal-only helper function since there wasn't a good place in the existing class hierarchy to store this function (and it didn't seem right to stick this function inside WriteBatchInternal::Rep).

Since I needed to access the WriteBatchEntryComparator, I moved some helper classes from write_batch_with_index.cc into write_batch_with_index_internal.h/.cc.  WriteBatchIndexEntry, ReadableWriteBatch, and WriteBatchEntryComparator are all unchanged (just moved to a different file(s)).

Test Plan: Added new unit tests.

Reviewers: rven, yhchiang, sdong, igor

Reviewed By: igor

Subscribers: dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D38037
2015-05-11 14:51:51 -07:00
Yueh-Hsuan Chiang cda4006e87 Enhance partial merge to support multiple arguments
Summary:
* PartialMerge api now takes a list of operands instead of two operands.
* Add min_pertial_merge_operands to Options, indicating the minimum
  number of operands to trigger partial merge.
* This diff is based on Schalk's previous diff (D14601), but it also
  includes necessary changes such as updating the pure C api for
  partial merge.

Test Plan:
* make check all
* develop tests for cases where partial merge takes more than two
  operands.

TODOs (from Schalk):
* Add test with min_partial_merge_operands > 2.
* Perform benchmarks to measure the performance improvements (can probably
  use results of task #2837810.)
* Add description of problem to doc/index.html.
* Change wiki pages to reflect the interface changes.

Reviewers: haobo, igor, vamsi

Reviewed By: haobo

CC: leveldb, dhruba

Differential Revision: https://reviews.facebook.net/D16815
2014-03-24 17:57:13 -07:00
Siying Dong ef2211a9ca [RocksDB Performance Branch] Introduce MergeContext to Lazily Initialize merge operand list
Summary: In get operations, merge_operands is only used in few cases. Lazily initialize it can reduce average latency in some cases

Test Plan: make all check

Reviewers: haobo, kailiu, dhruba

Reviewed By: haobo

CC: igor, nkg-, leveldb

Differential Revision: https://reviews.facebook.net/D14415
2013-12-06 10:28:59 -08:00