Summary:
- When google benchmark is used as a submodule in a parent projects
whose min CMake version is 3.3.2 or later, the google benchmark
`CMakeLists.txt` triggers a warning regarding CMake policy 0063:
```
CMake Warning (dev) at tests/googlebenchmark/src/CMakeLists.txt:19 (add_library):
Policy CMP0063 is not set: Honor visibility properties for all target
types. Run "cmake --help-policy CMP0063" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
Target "benchmark" of type "STATIC_LIBRARY" has the following visibility
properties set for CXX:
CXX_VISIBILITY_PRESET
VISIBILITY_INLINES_HIDDEN
For compatibility CMake is not honoring them for this target.
This warning is for project developers. Use -Wno-dev to suppress it.
```
- Set CMake Policy 0063 to NEW if the policy is available. This will not
affect parent projects who include benchmark but do not have a CMake min
version of 3.3.2 or later, i.e. when this policy is introduced.
* Add FIXME in multiple_ranges_test.cc
* Improve handling of large bounds in AddRange.
Due to breaking the loop too early, AddRange
would miss a final multplier of 'mult' that
was within the numeric range of T.
* Enable negative values for Range argument
Fixes#762.
* Try to fix build of benchmark_gtest
* Try some more to fix build
* Attempt to fix format macros
* Attempt to resolve format errors for mingw32
* Review feedback
Put unit tests in benchmark::internal namespace
Fix error reporting in multiple_ranges_test.cc
* [JSON] add threads and repetitions to the json output, for better ide…
[Tests] explicitly check for thread == 1
[Tests] specifically mark all repetition checks
[JSON] add repetition_index reporting, but only for non-aggregates (i…
* [Formatting] Be very, very explicit about pointer alignment so clang-format can not put pointers/references on the wrong side of arguments.
[Benchmark::Run] Make sure to use explanatory sentinel variable rather than a magic number.
* Do not pass redundant information
It broke because the libc++ is being built as part of *this*
build, with old gcc+libstdc++ (4.8?), but LLVM is preparing
to switch to C++14, and gcc+libstdc++ <5 are soft-deprecated.
Just the gcc update doesn't cut it, clang still uses old libstdc++.
Created BenchmarkName class which holds the full benchmark
name and allows specifying and retrieving different components
of the name (e.g. ARGS, THREADS etc.)
Fixes#730.
- On qnx platform, cpu and cache info is stored in a syspage struct which
is different from other OS platform.
- The fix has been verified on an aarch64 target running qnx 7.0.
Fixes#774
Since pthread is required at least for GCC (according to the
documentation), this should be reflected by the pkg-config file.
The same is, for instance, also done by the gflags library:
1005485222/cmake/package.pc.in (L13)
If this policy isn't set, CMake emits a large warning when project() is
called from a cmake subdirectory.
This came up when the benchmark library was added to the LLVM build, and
it was reported in https://llvm.org/PR38874. This patch was the fix I
applied locally to fix the issue, and I wanted to send it upstream.
Some benchmarks are particularly sensitive and they run in less than
a nanosecond. In order for the console reporter to provide meaningful
output for such benchmarks it needs to be able to display the times
using more resolution than a single nanosecond.
This patch changes the console reporter to print at least three
significant digits for all results.
Unlike the initial attempt, this patch does not align the decimal point.
* Adding Host Name and test
* Addressing Review Comments
* Adding Test for JSON Reporter
* Adding HOST_NAME_MAX for MacOS systems
* Adding Explaination for MacOS HOST_NAME_MAX Addition
* Addressing Peer Review Comments
* Adding codecvt in windows header guard
* Changing name SystemInfo and adding empty message incase host name fetch fails
* Adding Comment on Struct SystemInfo
Unit-tests fail to build due to the following errors:
/home/cfx/Dev/google-benchmark/benchmark.git/test/string_util_gtest.cc:12:5: required from here
/home/cfx/Applications/googletest-1.8.1/include/gtest/gtest.h:1444:11: error: comparison between signed and unsigned integer expressions [-Werror=sign-compare]
if (lhs == rhs) {
~~~~^~~~~~
Fixes#741
std::tmpnam is deprecated and its use is discouraged. For our purposes
in the tests, we really just need a file name which is unlikely to
exist.
This patch converts the tests to using a dummy random file name
generator, which should hopefully avoid name conflicts.
It is incorrect to say that an aggregate is computed over
run's iterations, because those iterations already got averaged.
Similarly, if there are N repetitions with 1 iterations each,
an aggregate will be computed over N measurements, not 1.
Thus it is best to simply use the count of separate reports.
Fixes#586.
It is better to let the RunBenchmarks(), report() decide
whether to actually *only* output aggregates or not,
depending on whether there are actually aggregates.
It's subtle indeed.
Previously, `BenchmarkRunner()` always said that "if there are no repetitions,
then you should never output only the repetitions". And the `report()` simply assumed
that the `report_aggregates_only` bool it received makes sense, and simply used it.
Now, the logic is the same, but the blame has shifted.
`BenchmarkRunner()` always propagates what those benchmarks would have wanted
to happen wrt the aggregates. And the `report()` lambda has to actually consider
both the `report_aggregates_only` bool, and it's meaningfulness.
To put it in the context of the patch series - if the repetition count was `1`,
but `*_report_aggregates_only` was set to `true`, and we capture each iteration separately,
then we will compute the aggregates, but then output everything, both the iteration,
and aggregates, despite `*_report_aggregates_only` being set to `true`.
If benchmark added as cmake subproject, HandleGTest throws an error as does return absolute source dir.
Change it to , so it will be refering to it's own source dir.
Also see PR #703.
* Fix SOURCE_DIR in HandleGTest.cmake
If benchmark added as cmake subproject, HandleGTest throws an error as does return absolute source dir.
Change it to , so it will be refering to it's own source dir.
As prevously written, "--benchmark_color=auto" was treated as true,
because IsTruthyFlagValue("auto") returned true. The fix is to
rely on IsColorTerminal test only if the flag value is "auto",
and fall back to IsTruthyFlagValue otherwise. I also integrated
force_no_color check into the same block.
For several versions now, CMake by default refers to macOS’ Clang as AppleClang instead of just Clang, which would fail STREQUAL. Fixed by changing it to MATCHES.
Ok, so, i'm still trying to get to the state when it will be a trivial change to report all the separate iterations.
The old code (LHS of the diff) was rather convoluted i'd say.
I have tried to refactor it a bit into *small* logical chunks, with proper comments.
As far as i can tell, i preserved the intent of the code, what it was doing before.
The road forward still isn't clear, but i'm quite sure it's not with the old code :)
The State constructor should not be part of the public API. Adding a
utility method to BenchmarkInstance allows us to avoid leaking the
RunInThread method into the public API.
My knowledge of python is not great, so this is kinda horrible.
Two things:
1. If there were repetitions, for the RHS (i.e. the new value) we were always using the first repetition,
which naturally results in incorrect change reports for the second and following repetitions.
And what is even worse, that completely broke U test. :(
2. A better support for different repetition count for U test was missing.
It's important if we are to be able to report 'iteration as repetition',
since it is rather likely that the iteration count will mismatch.
Now, the rough idea on how this is implemented now. I think this is the right solution.
1. Get all benchmark names (in order) from the lhs benchmark.
2. While preserving the order, keep the unique names
3. Get all benchmark names (in order) from the rhs benchmark.
4. While preserving the order, keep the unique names
5. Intersect `2.` and `4.`, get the list of unique benchmark names that exist on both sides.
6. Now, we want to group (partition) all the benchmarks with the same name.
```
BM_FOO:
[lhs]: BM_FOO/repetition0 BM_FOO/repetition1
[rhs]: BM_FOO/repetition0 BM_FOO/repetition1 BM_FOO/repetition2
...
```
We also drop mismatches in `time_unit` here.
_(whose bright idea was it to store arbitrarily scaled timers in json **?!** )_
7. Iterate for each partition
7.1. Conditionally, diff the overlapping repetitions (the count of repetitions may be different.)
7.2. Conditionally, do the U test:
7.2.1. Get **all** the values of `"real_time"` field from the lhs benchmark
7.2.2. Get **all** the values of `"cpu_time"` field from the lhs benchmark
7.2.3. Get **all** the values of `"real_time"` field from the rhs benchmark
7.2.4. Get **all** the values of `"cpu_time"` field from the rhs benchmark
NOTE: the repetition count may be different, but we want *all* the values!
7.2.5. Do the rest of the u test stuff
7.2.6. Print u test
8. ???
9. **PROFIT**!
Fixes#677
Roman Lebedev