Commit graph

671 commits

Author SHA1 Message Date
Nicholas Hutchinson b826143ac2 Flush reporters' output streams after writing a benchmark run (#288)
If a reporter's output stream isn't line-buffered (e.g. it's not writing
to a terminal) then it can be some time before a write to it becomes
visible.

This is problematic if, say, you're wanting to use tail -f to view the
file written to via --benchmark_out. Or if the application crashes,
leaving you with no results.

Addressed by flushing the reporters' output streams whenever we invoke
methods that may write to them.
2016-09-11 15:36:14 -06:00
Eric c6f3f0eb9c Cleanup RunBenchmark code. (#289)
* Cleanup the code for generating and running benchmarks

* Rework calculation of real/manual time

* Add back TSAN builder
2016-09-06 02:28:35 -06:00
Eric d038472c18 Refactor benchmark.cc into benchmark_register.cc and benchmark.cc (#287)
* Refactor benchmark.cc into benchmark_register.cc and benchmark_run.cc

The benchmark.cc file is getting really big and it contains a bunch of
unrelated components. This patch separates the files into two separate
parts. The "runtime" parts and the "registration" parts.

This patch also removes the PIMPL used by Benchmark. Previously we couldn't
have STL types in the interface but now we can. Therefore there is no reason
to keep BenchmarkImp.

* add missing include

* rework windows timers again

* Guard timespec on older Windows versions

* Remove old thread safety annotation workarounds
2016-09-05 15:48:40 -06:00
Eric Fiselier 9c26168126 Document --benchmark_filter. Thanks to NAThompson for the original patch 2016-09-05 15:40:12 -06:00
Ronny 72be9523bb Add Benchmark::ThreadRange() version with increment instead of multiply (#283)
* add additive thread range

* add test
2016-09-03 13:56:46 -06:00
Eric Fiselier 60e88c21e4 Remove TSAN builder for now. 2016-09-03 02:11:42 -06:00
Eric Fiselier db376c8654 Tweak TSAN flags 2016-09-03 01:45:59 -06:00
Eric Fiselier 183999c002 Add Thread Sanitizer bot to Travis 2016-09-03 00:40:44 -06:00
Eric Fiselier 07ee194092 Fix wording because an idiot came up with it. 2016-09-03 00:19:37 -06:00
Eric Fiselier 309ecb737f Increase complexity_test runtime slightly less. 2016-09-03 00:18:02 -06:00
Eric Fiselier f261c68074 Try and add MSVC NORETURN attribute again 2016-09-03 00:13:20 -06:00
Eric Fiselier 756f069918 Increase complexity_test.cc runtime to attempt to avoid appveyor failures 2016-09-03 00:06:51 -06:00
Eric Fiselier 2314368cb5 Add another workaround for Appveyor warnings 2016-09-02 23:59:07 -06:00
Eric Fiselier 5224b63c35 Fix BENCHMARK_NORETURN on MSVC 2016-09-02 23:54:58 -06:00
Eric Fiselier a85f2a8e5b attempt to suppress apveyor warnings 2016-09-02 23:48:17 -06:00
Eric cba945e37d Make PauseTiming() and ResumeTiming() per thread. (#286)
* Change to using per-thread timers

* fix bad assertions

* fix copy paste error on windows

* Fix thread safety annotations

* Make null-log thread safe

* remove remaining globals

* use chrono for walltime since it is thread safe

* consolidate timer functions

* Add missing ctime include

* Rename to be consistent with Google style

* Format patch using clang-format

* cleanup -Wthread-safety configuration

* Don't trust _POSIX_FEATURE macros because OS X lies.

* Fix OS X thread timings

* attempt to fix mingw build

* Attempt to make mingw work again

* Revert old mingw workaround

* improve diagnostics

* Drastically improve OS X measurements

* Use average real time instead of max
2016-09-02 21:34:34 -06:00
Dominic Hamon 94c2a30a3e Add clang-format file 2016-09-02 14:49:59 -07:00
biojppm 6a28f1e968 Fixture: add non const Setup() and TearDown(). (#285)
* Fixture: add non const Setup() and TearDown().

This allows write-access to the State variable, which is important in
upcoming user-defined counter functionality.

* Fix const placement in the Fixture methods.

* Fixture: use const_cast instead of static_cast.
2016-09-01 10:51:48 -07:00
biojppm 83561f0580 CMake: add headers to the list of source files. (#284)
cmake does not require this, but IDEs such as VisualStudio or QtCreator
need this to add the headers to the generated project.
2016-08-31 15:28:43 -07:00
Eric Fiselier 61f570e82a Fix #202 - Document Windows dependency on shlwapi.lib 2016-08-30 03:41:58 -06:00
Eric df7bd92afd Add warning for large benchmark families (#279)
* Add warning for large benchmark families

* reflow comment
2016-08-29 12:43:55 -06:00
Eric 6e25917001 Add -DBENCHMARK_USE_LIBCXX:BOOL=<value> option. (#280)
* Add BENCHMARK_USE_LIBCXX option.

* Add comments

* re-add zero null pointer warning
2016-08-29 12:43:30 -06:00
Eric 49bbff2c66 Supply old RangePair and ArgPair API for C++03 compatibility. (#278)
* changes

* remove other changes

* remove unneeded test

* cleanup unused include
2016-08-29 10:59:46 -07:00
Eric Fiselier 2e0796e148 Cleanup and optimize Range generation. 2016-08-28 23:15:55 -06:00
Eric Fiselier db1af86d16 Fix out-of-bounds std::vector access.
In the `Ranges(...)` generation code a "control" vector which stores
the current index for each range passed to `Ranges`. Previously this vector
was incorrectly initialized to the size of the subranges not the number
of subranges.

Additionally this patch suppresses unused warnings generated by
`stream_init_anchor`.
2016-08-28 23:07:38 -06:00
Eric Fiselier cbcd7b656e Fix std::cout/std::cerr static initialization order fiasco.
The benchmark library internals write to std::cout/std::cerr during program
startup. This can cause segfaults when the user doesn't include <iostream> in
the benchmark (which init's the streams). This patch fixes this by emitting
a dynamic initializer in every TU which initializes the streams.
2016-08-28 22:48:48 -06:00
Eric Fiselier 78e22f10de Update libc++ install script to use new names 2016-08-28 14:26:54 -06:00
Eric Fiselier aaa25ac376 Improve diagnostic output for output tests. 2016-08-28 14:22:11 -06:00
Eric 0ed4456097 Refactor output test runner into standalone module. (#277)
* refactor

* Move default substitutions into library

* Move default substitutions to the *right* place in the library

* Fix init order issues that caused test failures

* improve diagnostics

* add missing include

* general cleanup

* Address review comments
2016-08-28 13:24:16 -06:00
Eric Fiselier 577baa0dbd Remove MinGW builder in preference of MinGW-w64.
The plain MinGW enviroment does not provide any threading supporting, including
in the C++ STL. The MinGW-w64 enviroment does not have this problem.

This patch removes the 32 bit bot since it's always going to fail.
2016-08-10 18:23:50 -06:00
Eric a11fb69c89 Add --benchmark_report_aggregates_only={true|false} flag for better summary output. (#267) 2016-08-10 18:20:54 -06:00
Eric Fiselier c9d747e5e6 Test different Appveyor MinGW installation 2016-08-09 14:36:15 -06:00
Eric Fiselier a7a7c56152 Workaround flaky complexity_test.cc test case.
See https://github.com/google/benchmark/issues/272
2016-08-09 14:14:15 -06:00
Eric 5eac66249c Add a "compare_bench.py" tooling script. (#266)
This patch adds the compare_bench.py utility which can be used to compare the result of benchmarks.
The program is invoked like:

$ compare_bench.py <old-benchmark> <new-benchmark> [benchmark options]...
Where <old-benchmark> and <new-benchmark> either specify a benchmark executable file, or a JSON output file. The type of the input file is automatically detected. If a benchmark executable is specified then the benchmark is run to obtain the results. Otherwise the results are simply loaded from the output file.
2016-08-09 12:33:57 -06:00
Eric de4ead7a53 Document compiler requirements (#269) 2016-08-09 12:31:44 -06:00
Eric Fiselier 96a5965b6e Use pre-installed MinGW on Appveyor
Currently out Appveyor CI downloads and stashes a custom MinGW installation.
However the builder already provides both 64 and 32 bit installations of MinGW.
This patch changes our CI to use those instead.

I'm hoping this will fix issues where the g++ is broken due to the Appveyor
package caching semantics.
2016-08-09 12:09:37 -06:00
Eric Fiselier 95dee3c699 Work around bogus GCC warning 2016-08-08 02:22:49 -06:00
Eric Fiselier ee54a3f03e Rework Appveyor config
Currently the Appveyor bot is a PIT. It never passes and it often hangs
or gives very poor output. This patch rewrites the configuration.

This patch also attempts to fix a flaky complexity test as a drive-by.
2016-08-08 02:04:59 -06:00
Eric Fiselier cfd717e43b Increase Appveyor timeout and fix build warning 2016-08-08 01:41:01 -06:00
Eric Fiselier 9e13b95b72 fix Appveyor config and add timeout 2016-08-08 00:04:59 -06:00
Eric Fiselier 85e7ed7eea Add libc++ and MSAN builders to to travis.
This patch adds three new build configurations to the travis CI.

* Clang 3.8 w/ libc++
* Clang 3.8 w/ libc++, UBSAN, ASAN
* Clang 3.8 w/ libc++, MSAN
2016-08-07 23:44:34 -06:00
Eric Fiselier 8228b3e8a7 Make Appveyor show test output on failure 2016-08-07 20:09:52 -06:00
Eric Fiselier 8267117a92 Add ASAN and UBSAN builders to travis for both GCC and Clang.
This patch adds new builders that test against GCC 6 and Clang 3.8 respectivly.
They also enable both address and undefined sanitizer. MSAN currently won't work
since it requires a sanitized STL.
2016-08-07 19:25:19 -06:00
Eric Fiselier c04f703ab4 Fix signed integer overflow UB in complexity computations.
Previously the FittingCurve functions for n^2 and n^3 did the calculation
using int types. This can overflow and cause UB. This patch changes the
calculations to use std::pow to prevent this.

Also re-enable VC 2013 appveyor bot since I *hope* this is what was causing
the failures.
2016-08-07 17:33:18 -06:00
Eric Fiselier 5121b8546b Temporarly disable VC 2013 appveyor build in order to test VC 2015 2016-08-07 17:09:31 -06:00
Eric Fiselier 9820035998 Whitespace change to trigger appveyor. 2016-08-07 16:31:43 -06:00
Eric Fiselier 1b57a2ad05 Add myself to AUTHORS.
This is secretly just a test commit to get appveyor to run again.
2016-08-07 15:19:20 -06:00
Eric Fiselier f30028b2a8 Attempt to fix VC 2013 build.
VC 2013 injects valid when  assigning an initializer list to std::set.
This attempts to work around this issue by using std::set's constructors
instead of the assignment operators.
2016-08-07 14:57:39 -06:00
Marcin Kolny dfe0260754 Support multiple ranges in the benchmark (#257)
* Support multiple ranges in the benchmark

google-benchmark library allows to provide up to two ranges to the
benchmark method (range_x and range_y). However, in many cases it's not
sufficient. The patch introduces multi-range features, so user can easily
define multiple ranges by passing a vector of integers, and access values
through the method range(i).

* Remove redundant API

Functions State::range_x() and State::range_y() have been removed. They should
be replaced by State::range(0) and State::range(1).
Functions Benchmark::ArgPair() and Benchmark::RangePair() have been removed.
They should be replaced by Benchmark::Args() and Benchmark::Ranges().
2016-08-04 12:30:14 -07:00
Eric Fiselier 7f1da4a68b Hide yet another test from MSVC 2013 2016-08-02 22:09:06 -06:00