* CMake: `get_git_version()`: just use `--dirty` flag of `git describe`
* CMake: move version normalization out of `get_git_version()`
Mainly, i want `get_git_version()` to return true version,
not something sanitized.
* JSON reporter: store library version and schema version in `context`
* Tools: discard inputs with unexpected `json_schema_version`
* Extract version string into `GetBenchmarkVersiom()`
---------
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
* Instead of directly comparing std::cout and GetOutputStream(), the underlying buffers are retreived via rdbuf(), and then compared.
* Instead of fflush(stdout), call out.flush().
Use out << FormatString() instead of vprintf
---------
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
Starting with Linux 6.6 [1], RDCYCLE is a privileged instruction on
RISC-V and can't be used directly from userland. There is a sysctl
option to change that as a transition period, but it will eventually
disappear.
Use RDTIME instead, which while less accurate has the advantage of being
synchronized between CPU (and thus monotonic) and of constant frequency.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=cc4c07c89aada16229084eeb93895c95b7eabaa3
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
There is no bug here, but it gave me a scare the other day.
It is not incorrect to use `IterationCount` here,
since it's just an `int64_t` either way,
but it's wildly confusing. Let's not do that.
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
* Increase the kMaxIterations limit
This fixes#1663. Note that as a result of this change, the columns in the console output can become misaligned if the actual iteration count is too high. This will be dealt with in a separate commit.
* Fix failing test on Windows
* Fix formatting
---------
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
* Make json and csv output consistent.
Currently, the --benchmark_format=csv option does not output the correct value for the cv statistics. Also, the json output should not contain a time unit for the cv statistics.
* fix formatting
* undo json change
---------
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
Previously, this could return the wrong result when there
was an even number of elements.
There were two `nth_element` calls. The second call could
change elements in `[center2, end])`, which was where
`center` pointed. Therefore, `*center` sometimes had the
wrong value after the second `nth_element` call.
Rewrite to use `max_element` instead of the second call to
`nth_element`. This avoids modifying the vector.
* perf_counters: Initialize once only when needed
This works around some performance problems running Android under QEMU.
Calling `pfm_initialize` was very slow, and was called during dynamic
initialization (before `main` or when loaded as a shared library).
This happened whenever benchmark was linked, even if no benchmarks
were run.
Instead, call `pfm_initialize` at most once, and only when one of:
1. `PerfCounters::Initialize` is called
2. `PerfCounters::Create` is called with a non-empty counter list
3. `PerfCounters::IsCounterSupported` is called
The return value of the first `pfm_initialize()` is saved and
returned from all subsequent `PerfCounters::Initialize` calls.
* perf_counters: Make success var const
* InitLibPfmOnce: Inline function
* State: Initialize counters with kAvgIteration in constructor
Previously, `counters` was updated in `PauseTiming()` with
`counters[name] += Counter(measurement, kAvgIteration)`.
The first `counters[name]` call inserts a counter with no flags.
There is no `operator+=` for `Counter`, so the insertion is done
by converting the `Counter` to a `double`, then constructing a
`Counter` to insert from the `double`, which drops the flags.
Pre-insert the `Counter` with the correct flags, then only
update `Counter::value`.
Introduced in 1c64a36 ([perf-counters] Fix pause/resume (#1643)).
* perf_counters_test.cc: Don't divide by iterations
Perf counters are now divided by iterations, so dividing again
in the test is wrong.
* State: Fix shadowed param error
* benchmark.cc: Fix clang-tidy error
---------
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
Change condition for `benchmarks_with_threads` from `benchmark.threads() > 0` to `> 1`. `threads()` appears to always be `>= 1`.
Introduced in fbc6efa (Refactoring of PerfCounters infrastructure (#1559))
* [perf-counters] Fix pause/resume
Using `state.PauseTiming() / state.ResumeTiming()` was broken.
Thanks [@virajbshah] for the the repro testcase.
* ran clang-format over the whole perf_counters_test.cc
* Remove check that perf counters are 0 on `Pause`, since `Pause`/`Resume`
sequences would cause a non-0 counter value
* both upper and lower bound for the with/without resume counters
---------
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
BENCHMARK_HAVE_STD_REGEX is not used but HAVE_STD_REGEX like the other two choices, i.e. HAVE_GNU_POSIX_REGEX and HAVE_POSIX_REGEX.
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
* Address warnings on NVIDIA nvc++
Types of warnings were being generated:
1. Deprecated warnings - solved by defining the relevant BENCHMARK_*
macros for nvc++ and adding pragma suppress on a couple of .cc files
2. Setup/TearDown const vs non-const partial override - solved by
adding non-const version
3. Static but not referenced - added diagnostic suppress for that file
* Modified manually to comply with CD/CI
* Revert partial override
* Suppress warnings from tests if compiler is NVHPC
---------
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
* [FR] Provide public accessors to benchmark name and arguments #1551
* Update AUTHORS and CONTRIBUTORS
* Update benchmark_register.cc
* Fix lint formatting
* Refactoring of PerfCounters infrastructure
The main feature in this pull request is the removal of the static
sharing of PerfCounters and instead creating them at the top
`RunBenchmarks()` function where all benchmark runners are created. A
single PerfCountersMeasurement object is created and then shared with
all the new BenchmarkRunners objects, one per existing benchmark.
Other features conflated here in this PR are:
- Added BENCHMARK_DONT_OPTIMIZE macro in global scope
- Removal of the `IsValid()` query, being replaced by checking the
number of remaining counters after validity tests
- Refactoring of all GTests to reflect the changes and new semantics
- extra comments throughout the new code to clarify intent
It was extremely hard to separate all those features in different PRs
as requested since they are so interdependent on each other so I'm just
pushing them altogether and asking for forgiveness.
This PR comes replacing PRs 1555 and 1558 which have been closed.
* Fixed whitespace issue with clang-format
My clang-format insists in deleting this single white space on line 601
while Github's clang format breaks when it is added. I had to disable
format-on-save to check-in this revert change.
I'm using clang 14.0.6.
* Filter performance counter names, not invalidate all
Currently, the performance counters are validated while they
are being created and one failure returns NoCounters(), ie it
effecitvely invalidates all the counters.
I would like to propose a new behavior: filter instead. If an
invalid name is added to the counter list, or if that particular
counter is not supported on this platform, that counter is dropped
from the list and an error messages is created, while all the
other counters remain active.
This will give testers a peace of mind that if one mistake is made
or if something is changed or removed from libpfm, their entire
test will not be invalidated. This feature gives more tolerance
with respect to versioning.
Another positive is that testers can now input a superset of all
desired counters for all platforms they support and just let
Benchmark drop all those that are not supported, although it will
create quite a lot of noise down the line, in which case perhaps
we should drop silently or make a consolidated, single error line
but this was not implemented in this change set.
* Removed unused helper type.
* Implement unlimited number of performance counters
Linux performance counters will limit the number of hardware
counters per reading group. For that reason the implementation of
PerfCounters is limited to 3. However if only software counters
are added, there is no reason to limit the counters. For hardware
counters, we create multiple groups and store a vector or leaders
in the PerfCounters object. When reading, there is an extra time
waste by iterating through all the group leaders. However this
should be the same performance as with today. Reading is done by
groups and it had to be heavily adjusted with the logic being
moved to PerfCounterValues. I created a test for x86-64 and took
care of filtering out the events in case it runs in a platform
that does not support those counters - the test will not fail. The
current tests were already failing (ReOpenExistingCounters,
CreateExistingMeasurements and MultiThreaded) on the main branch
and they continue to fail after this implementation - I did not
fix those not to conflate all here.
* Moved the PerfCounterValues::Read() implementation from header to body.
* Added missing implementation of PerfCounters::IsCounterSupported when HAVE_LIBPFM is not defined.
* Changed comments to reflect the implementation
* Removed arg name so it does not generate an error when HAVE_LIBPBM is not defined.
* Made loop counter a const reference for clang-tidy
* Added missig BENCHMARK_EXPORT to PerfCounterValues
* Fix CPU frequency estimation on riscv
* Cleanup code for CPU frequency estimation
* Fix use before definition of the macro
* Move the platform definitions back
* Fix compilation error on windows
* Remove unused sleep.h and sleep.cc
The previous code was triggering a warning in Debug builds where NDEBUG
is not defined and BM_CHECK() is included:
benchmark/src/benchmark_runner.cc: In function ‘benchmark::internal::BenchTimeType benchmark::internal::ParseBenchMinTime(const std::string&)’:
benchmark/src/benchmark_runner.cc:212:24: error: suggest parentheses around ‘&&’ within ‘||’ [-Werror=parentheses]
212 | (has_suffix && *p_end == 's' || *p_end == '\0'))
| ~~~~~~~~~~~^~~~~~~~~~~~~~~~
benchmark/src/check.h:82:4: note: in definition of macro ‘BM_CHECK’
82 | (b ? ::benchmark::internal::GetNullLogInstance() \
| ^
Add parenthesis around the && expression.
Also fix a spelling error and move the comma in the preceding comment to
improve clarity.
Tested:
- cmake -E make_directory build
- cmake -E chdir "build" cmake -DBENCHMARK_DOWNLOAD_DEPENDENCIES=on -DCMAKE_BUILD_TYPE=Debug ../
- cmake --build "build" --config Debug
- cmake -E chdir "build" ctest --build-config Debug
* Allow specifying number of iterations via --benchmark_min_time.
Make the flag accept two new suffixes:
+ <integer>x: number of iterations
+ <floag>s: minimum number of seconds.
This matches the internal benchmark API.
* forgot to change flag type to string
* used tagged union instead of std::variant, which is not available pre C++14
* update decl in benchmark_runner.h too
* fixed errors
* refactor
* backward compat
* typo
* use IterationCount type
* fixed test
* const_cast
* ret type
* remove extra _
* debug
* fixed bug from reporting that caused the new configs not to be included in the final report
* addressed review comments
* restore unnecessary changes in test/BUILD
* fix float comparisons warnings from Release builds
* clang format
* fix visibility warning
* remove misc file
* removed backup files
* addressed review comments
* fix shorten in warning
* use suffix for existing min_time specs to silent warnings in tests
* fix leaks
* use default min-time value in flag decl for consistency
* removed double kMinTimeDecl from benchmark.h
* dont need to preserve errno
* add death tests
* Add BENCHMARK_EXPORT to hopefully fix missing def errors
* only enable death tests in debug mode because bm_check is no-op in release mode
* guard death tests with additional support-check macros
* Add additional guard to prevent running in Release mode
---------
Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
* Include the benchmark's family-name in State
For compat with internal library, where State::name() returns the benchmark's family name.
* added missing files from prev commit
* fix field-init order error
* added test
Matthias Liedtke