mirror of https://github.com/google/benchmark.git
3 Commits
| Author | SHA1 | Message | Date |
|---|---|---|---|
Roman Lebedev
|
fbc31405b2
|
Random interleaving of benchmark repetitions - the sequel (fixes #1051) (#1163)
Inspired by the original implementation by Hai Huang @haih-g from https://github.com/google/benchmark/pull/1105. The original implementation had design deficiencies that weren't really addressable without redesign, so it was reverted. In essence, the original implementation consisted of two separateable parts: * reducing the amount time each repetition is run for, and symmetrically increasing repetition count * running the repetitions in random order While it worked fine for the usual case, it broke down when user would specify repetitions (it would completely ignore that request), or specified per-repetition min time (while it would still adjust the repetition count, it would not adjust the per-repetition time, leading to much greater run times) Here, like i was originally suggesting in the original review, i'm separating the features, and only dealing with a single one - running repetitions in random order. Now that the runs/repetitions are no longer in-order, the tooling may wish to sort the output, and indeed `compare.py` has been updated to do that: #1168. |
|
|
Dominic Hamon
|
e025dd5a54
|
Revert "Implementation of random interleaving. (#1105)" (#1161)
This reverts commit
|
|
|
haih-g
|
a6a738c1cc
|
Implementation of random interleaving. (#1105)
* Implementation of random interleaving. See http://github.com/google/benchmark/issues/1051 for the feature requests. Committer: Hai Huang (http://github.com/haih-g) On branch fr-1051 Changes to be committed: modified: include/benchmark/benchmark.h modified: src/benchmark.cc new file: src/benchmark_adjust_repetitions.cc new file: src/benchmark_adjust_repetitions.h modified: src/benchmark_api_internal.cc modified: src/benchmark_api_internal.h modified: src/benchmark_register.cc modified: src/benchmark_runner.cc modified: src/benchmark_runner.h modified: test/CMakeLists.txt new file: test/benchmark_random_interleaving_gtest.cc * Fix benchmark_random_interleaving_gtest.cc for fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark.cc modified: src/benchmark_runner.cc modified: test/benchmark_random_interleaving_gtest.cc * Fix macos build for fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_api_internal.cc modified: src/benchmark_api_internal.h modified: src/benchmark_runner.cc * Fix macos and windows build for fr-1051. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_runner.cc * Fix benchmark_random_interleaving_test.cc for macos and windows in fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: test/benchmark_random_interleaving_gtest.cc * Fix int type benchmark_random_interleaving_gtest for macos in fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: test/benchmark_random_interleaving_gtest.cc * Address dominichamon's comments 03/29 for fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark.cc modified: src/benchmark_api_internal.cc modified: src/benchmark_api_internal.h modified: test/benchmark_random_interleaving_gtest.cc * Address dominichamon's comment on default min_time / repetitions for fr-1051. Also change sentinel of random_interleaving_repetitions to -1. Hopefully it fixes the failures on Windows. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark.cc modified: src/benchmark_api_internal.cc modified: src/benchmark_api_internal.h * Fix windows test failures for fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_api_internal.cc modified: src/benchmark_runner.cc * Add license blurb for fr-1051. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_adjust_repetitions.cc modified: src/benchmark_adjust_repetitions.h * Switch to std::shuffle() for fr-1105. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark.cc * Change to 1e-9 in fr-1105 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_adjust_repetitions.cc * Fix broken build caused by bad merge for fr-1105. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_api_internal.cc modified: src/benchmark_runner.cc * Fix build breakage for fr-1051. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark.cc modified: src/benchmark_api_internal.cc modified: src/benchmark_api_internal.h modified: src/benchmark_register.cc modified: src/benchmark_runner.cc * Print out reports as they come in if random interleaving is disabled (fr-1051) Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark.cc * size_t, int64_t --> int in benchmark_runner for fr-1051. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_runner.cc modified: src/benchmark_runner.h * Address comments from dominichamon for fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark.cc modified: src/benchmark_adjust_repetitions.cc modified: src/benchmark_adjust_repetitions.h modified: src/benchmark_api_internal.cc modified: src/benchmark_api_internal.h modified: test/benchmark_random_interleaving_gtest.cc * benchmar_indices --> size_t to make CI pass: fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark.cc * Fix min_time not initialized issue for fr-1051. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_api_internal.cc modified: src/benchmark_api_internal.h * min_time --> MinTime in fr-1051. Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: src/benchmark_api_internal.cc modified: src/benchmark_api_internal.h modified: src/benchmark_runner.cc * Add doc for random interleaving for fr-1051 Committer: Hai Huang <haih@google.com> On branch fr-1051 Your branch is up to date with 'origin/fr-1051'. Changes to be committed: modified: README.md new file: docs/random_interleaving.md Co-authored-by: Dominic Hamon <dominichamon@users.noreply.github.com> |