As @dominichamon and I have discussed, the current reporter interface
is poor at best. And something should be done to fix it.
I strongly suspect such a fix will require an entire reimagining
of the API, and therefore breaking backwards compatibility fully.
For that reason we should start deprecating and removing parts
that we don't intend to replace. One of these parts, I argue,
is the CSVReporter. I propose that the new reporter interface
should choose a single output format (JSON) and traffic entirely
in that. If somebody really wanted to replace the functionality
of the CSVReporter they would do so as an external tool which
transforms the JSON.
For these reasons I propose deprecating the CSVReporter.
* Add tests to verify assembler output -- Fix DoNotOptimize.
For things like `DoNotOptimize`, `ClobberMemory`, and even `KeepRunning()`,
it is important exactly what assembly they generate. However, we currently
have no way to test this. Instead it must be manually validated every
time a change occurs -- including a change in compiler version.
This patch attempts to introduce a way to test the assembled output automatically.
It's mirrors how LLVM verifies compiler output, and it uses LLVM FileCheck to run
the tests in a similar way.
The tests function by generating the assembly for a test in CMake, and then
using FileCheck to verify the // CHECK lines in the source file are found
in the generated assembly.
Currently, the tests only run on 64-bit x86 systems under GCC and Clang,
and when FileCheck is found on the system.
Additionally, this patch tries to improve the code gen from DoNotOptimize.
This should probably be a separate change, but I needed something to test.
* Disable assembly tests on Bazel for now
* Link FIXME to github issue
* Fix Tests on OS X
* fix strip_asm.py to work on both Linux and OS X like targets
This patch disables the -Winvalid-offsetof warning for GCC and Clang
when using it to check the cache lines of the State object.
Technically this usage of offsetof is undefined behavior until C++17.
However, all major compilers support this application as an extension,
as demonstrated by the passing static assert (If a compiler encounters UB
during evaluation of a constant expression, that UB must be diagnosed).
Unfortunately, Clang and GCC also produce a warning about it.
This patch temporarily suppresses the warning using #pragma's in the
source file (instead of globally suppressing the warning in the build systems).
This way the warning is ignored for both CMake and Bazel builds without
having to modify either build system.
* Improve State packing: put important members on first cache line.
This patch does a few different things to ensure commonly accessed
data is on the first cache line of the `State` object.
First, it moves the `error_occurred_` member to reside after
the `started_` and `finished_` bools, since there was internal
padding there that was unused.
Second, it moves `batch_leftover_` and `max_iterations` further up
in the struct declaration. These variables are used in the calculation
of `iterations()` which users might call within the loop. Therefore
it's more important they exist on the first cache line.
Finally, this patch turns the bool members into bitfields. Although
this shouldn't have much of an effect currently, because padding is
still needed between the last bool and the first size_t, it should
help in future changes that require more "bool like" members.
* Remove bitfield change for now
* Move bools (and their padding) to end of "first cache line" vars.
I think it makes the most sense to move the padding required
following the group of bools to the end of the variables we want
on the first cache line.
This also means that the `total_iterations_` variable, which is the
most accessed, has the same address as the State object.
* Fix static assertion after moving bools
* Add support for GTest based unit tests.
As Dominic and I have previously discussed, there is some
need/desire to improve the testing situation in Google Benchmark.
One step to fixing this problem is to make it easier to write
unit tests by adding support for GTest, which is what this patch does.
By default it looks for an installed version of GTest. However the
user can specify -DBENCHMARK_BUILD_EXTERNAL_GTEST=ON to instead
download, build, and use copy of gtest from source. This is
quite useful when Benchmark is being built in non-standard configurations,
such as against libc++ or in 32 bit mode.
* Add BENCHMARK_BUILD_32_BITS option and add builders to test it
* Attempt to fix travis configuration
* Make add_required_cxx_compiler_flag cause an error when the flag isn't supported
* add gcc-multilib dependancy on travis
* attempt to fix travis.yml parsing error
* Require g++-multilib instead of gcc-multilib
* Add 32 bit release configurations
* Attempt to fix libc++ travis build w/ 32 bits
* Work around CMake configuration failure on Travis
fixes#354
The build fails with ICC17 because of warnings and Werror. What is the correct solution to fix it?
Should a patch
disable Werror for ICC (or maybe all non known compilers)
disable the false postive warnings for all files. This could be done using:
add_cxx_compiler_flag(-wd2102) #ICC17u2: Many false positives for Wstrict-aliasing
add_cxx_compiler_flag(-wd2259) #ICC17u2: non-pointer conversion from "long" to "int" may lose significant bits (even for explicit static cast, sleep.cc(44))
add_cxx_compiler_flag(-wd654) #ICC17u2: overloaded virtual function "benchmark::Fixture::SetUp" is only partially overridden (because of deprecated overload)
disable warnings at file level or some other granularity
* Refactor benchmark.cc into benchmark_register.cc and benchmark_run.cc
The benchmark.cc file is getting really big and it contains a bunch of
unrelated components. This patch separates the files into two separate
parts. The "runtime" parts and the "registration" parts.
This patch also removes the PIMPL used by Benchmark. Previously we couldn't
have STL types in the interface but now we can. Therefore there is no reason
to keep BenchmarkImp.
* add missing include
* rework windows timers again
* Guard timespec on older Windows versions
* Remove old thread safety annotation workarounds
* Change to using per-thread timers
* fix bad assertions
* fix copy paste error on windows
* Fix thread safety annotations
* Make null-log thread safe
* remove remaining globals
* use chrono for walltime since it is thread safe
* consolidate timer functions
* Add missing ctime include
* Rename to be consistent with Google style
* Format patch using clang-format
* cleanup -Wthread-safety configuration
* Don't trust _POSIX_FEATURE macros because OS X lies.
* Fix OS X thread timings
* attempt to fix mingw build
* Attempt to make mingw work again
* Revert old mingw workaround
* improve diagnostics
* Drastically improve OS X measurements
* Use average real time instead of max
Add policy CMP0056, which honors the link flags in try_compile and
try_run. This allows for building against non-system libc++ by providing
the correct -L and rpath options in a containing project.
For example:
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -L ${LLVM_ROOT}/lib -l c++ -l c++abi")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-rpath,${LLVM_ROOT}/lib")
In addition to release, CMake supports RELWITHDEBINFO and MINSIZEREL
build configurations. In particular, debug info is necessary for many
profilers to do anything useful, making RELWITHDEBINFO important here.
MINSIZEREL was added for completeness' sake.
BENCHMARK_ENABLE_LTO=true was completely replacing
CMAKE_CXX_FLAGS_RELEASE; meaning neither CMake's release defaults nor
user customizations were being applied.
This change looks for CMake's default setting for MSVC++, /W3 (and any
other level should that change in the future), and removes it before
adding /W4. This stops the build for MSVC++ emitting warnings about
/W4 overriding /W3 earlier on the command line.
This also points out some important issues like:
warning: conversion to 'size_t {aka unsigned int}' from 'long long int' may alter its value [-Wconversion]
state.SetBytesProcessed(items_processed * sizeof(v));
This occurs on 32 bit systems where size_t is only 32 bits.
This patch adds a `coverage` target that allows coverage statisitics to be
retrieved for the project. It requires that lcov and gcov is installed and
that the generator is unix makefiles but this can be improved upon in
future releases.
To make it work use the coverage build type:
```
cmake -DCMAKE_BUILD_TYPE=Coverage .
make coverage
```
This improves development iterations when working in debug mode as you
don't have to fix up warnings to get the code to compile. Once a feature
is complete you can then run a release build and fix up all the warnings.
This patch adopts a new internal structure for how timings are performed.
Currently every iteration of a benchmark checks to see if it has been running
for an appropriate amount of time. Checking the clock introduces noise into
the timings and this can cause inconsistent output from each benchmark.
Now every iteration of a benchmark only checks an iteration count to see if it
should stop running. The iteration count is determined before hand by testing
the benchmark on a series of increasing iteration counts until a suitable count
is found. This increases the amount of time it takes to run the actual benchmarks
but it also greatly increases the accuracy of the results.
This patch introduces some breaking changes. The notable breaking changes are:
1. Benchmarks run on multiple threads no generate a report per thread. Instead
only a single report is generated.
2. ::benchmark::UseRealTime() was removed and replaced with State::UseRealTime().
This patch cleans up our use of generic macros and also merges changes in the
build system.
It adds options -DBENCHMARK_ENABLE_TESTING and -DBENCHMARK_ENABLE_SHARED.
Dominic Hamon