Commit Graph

701 Commits

Author SHA1 Message Date
Eric df7bd92afd Add warning for large benchmark families (#279)
* Add warning for large benchmark families

* reflow comment
2016-08-29 12:43:55 -06:00
Eric 6e25917001 Add -DBENCHMARK_USE_LIBCXX:BOOL=<value> option. (#280)
* Add BENCHMARK_USE_LIBCXX option.

* Add comments

* re-add zero null pointer warning
2016-08-29 12:43:30 -06:00
Eric 49bbff2c66 Supply old `RangePair` and `ArgPair` API for C++03 compatibility. (#278)
* changes

* remove other changes

* remove unneeded test

* cleanup unused include
2016-08-29 10:59:46 -07:00
Eric Fiselier 2e0796e148 Cleanup and optimize Range generation. 2016-08-28 23:15:55 -06:00
Eric Fiselier db1af86d16 Fix out-of-bounds std::vector access.
In the `Ranges(...)` generation code a "control" vector which stores
the current index for each range passed to `Ranges`. Previously this vector
was incorrectly initialized to the size of the subranges not the number
of subranges.

Additionally this patch suppresses unused warnings generated by
`stream_init_anchor`.
2016-08-28 23:07:38 -06:00
Eric Fiselier cbcd7b656e Fix std::cout/std::cerr static initialization order fiasco.
The benchmark library internals write to std::cout/std::cerr during program
startup. This can cause segfaults when the user doesn't include <iostream> in
the benchmark (which init's the streams). This patch fixes this by emitting
a dynamic initializer in every TU which initializes the streams.
2016-08-28 22:48:48 -06:00
Eric Fiselier 78e22f10de Update libc++ install script to use new names 2016-08-28 14:26:54 -06:00
Eric Fiselier aaa25ac376 Improve diagnostic output for output tests. 2016-08-28 14:22:11 -06:00
Eric 0ed4456097 Refactor output test runner into standalone module. (#277)
* refactor

* Move default substitutions into library

* Move default substitutions to the *right* place in the library

* Fix init order issues that caused test failures

* improve diagnostics

* add missing include

* general cleanup

* Address review comments
2016-08-28 13:24:16 -06:00
Eric Fiselier 577baa0dbd Remove MinGW builder in preference of MinGW-w64.
The plain MinGW enviroment does not provide any threading supporting, including
in the C++ STL. The MinGW-w64 enviroment does not have this problem.

This patch removes the 32 bit bot since it's always going to fail.
2016-08-10 18:23:50 -06:00
Eric a11fb69c89 Add --benchmark_report_aggregates_only={true|false} flag for better summary output. (#267) 2016-08-10 18:20:54 -06:00
Eric Fiselier c9d747e5e6 Test different Appveyor MinGW installation 2016-08-09 14:36:15 -06:00
Eric Fiselier a7a7c56152 Workaround flaky complexity_test.cc test case.
See https://github.com/google/benchmark/issues/272
2016-08-09 14:14:15 -06:00
Eric 5eac66249c Add a "compare_bench.py" tooling script. (#266)
This patch adds the compare_bench.py utility which can be used to compare the result of benchmarks.
The program is invoked like:

$ compare_bench.py <old-benchmark> <new-benchmark> [benchmark options]...
Where <old-benchmark> and <new-benchmark> either specify a benchmark executable file, or a JSON output file. The type of the input file is automatically detected. If a benchmark executable is specified then the benchmark is run to obtain the results. Otherwise the results are simply loaded from the output file.
2016-08-09 12:33:57 -06:00
Eric de4ead7a53 Document compiler requirements (#269) 2016-08-09 12:31:44 -06:00
Eric Fiselier 96a5965b6e Use pre-installed MinGW on Appveyor
Currently out Appveyor CI downloads and stashes a custom MinGW installation.
However the builder already provides both 64 and 32 bit installations of MinGW.
This patch changes our CI to use those instead.

I'm hoping this will fix issues where the g++ is broken due to the Appveyor
package caching semantics.
2016-08-09 12:09:37 -06:00
Eric Fiselier 95dee3c699 Work around bogus GCC warning 2016-08-08 02:22:49 -06:00
Eric Fiselier ee54a3f03e Rework Appveyor config
Currently the Appveyor bot is a PIT. It never passes and it often hangs
or gives very poor output. This patch rewrites the configuration.

This patch also attempts to fix a flaky complexity test as a drive-by.
2016-08-08 02:04:59 -06:00
Eric Fiselier cfd717e43b Increase Appveyor timeout and fix build warning 2016-08-08 01:41:01 -06:00
Eric Fiselier 9e13b95b72 fix Appveyor config and add timeout 2016-08-08 00:04:59 -06:00
Eric Fiselier 85e7ed7eea Add libc++ and MSAN builders to to travis.
This patch adds three new build configurations to the travis CI.

* Clang 3.8 w/ libc++
* Clang 3.8 w/ libc++, UBSAN, ASAN
* Clang 3.8 w/ libc++, MSAN
2016-08-07 23:44:34 -06:00
Eric Fiselier 8228b3e8a7 Make Appveyor show test output on failure 2016-08-07 20:09:52 -06:00
Eric Fiselier 8267117a92 Add ASAN and UBSAN builders to travis for both GCC and Clang.
This patch adds new builders that test against GCC 6 and Clang 3.8 respectivly.
They also enable both address and undefined sanitizer. MSAN currently won't work
since it requires a sanitized STL.
2016-08-07 19:25:19 -06:00
Eric Fiselier c04f703ab4 Fix signed integer overflow UB in complexity computations.
Previously the FittingCurve functions for n^2 and n^3 did the calculation
using int types. This can overflow and cause UB. This patch changes the
calculations to use std::pow to prevent this.

Also re-enable VC 2013 appveyor bot since I *hope* this is what was causing
the failures.
2016-08-07 17:33:18 -06:00
Eric Fiselier 5121b8546b Temporarly disable VC 2013 appveyor build in order to test VC 2015 2016-08-07 17:09:31 -06:00
Eric Fiselier 9820035998 Whitespace change to trigger appveyor. 2016-08-07 16:31:43 -06:00
Eric Fiselier 1b57a2ad05 Add myself to AUTHORS.
This is secretly just a test commit to get appveyor to run again.
2016-08-07 15:19:20 -06:00
Eric Fiselier f30028b2a8 Attempt to fix VC 2013 build.
VC 2013 injects valid when  assigning an initializer list to std::set.
This attempts to work around this issue by using std::set's constructors
instead of the assignment operators.
2016-08-07 14:57:39 -06:00
Marcin Kolny dfe0260754 Support multiple ranges in the benchmark (#257)
* Support multiple ranges in the benchmark

google-benchmark library allows to provide up to two ranges to the
benchmark method (range_x and range_y). However, in many cases it's not
sufficient. The patch introduces multi-range features, so user can easily
define multiple ranges by passing a vector of integers, and access values
through the method range(i).

* Remove redundant API

Functions State::range_x() and State::range_y() have been removed. They should
be replaced by State::range(0) and State::range(1).
Functions Benchmark::ArgPair() and Benchmark::RangePair() have been removed.
They should be replaced by Benchmark::Args() and Benchmark::Ranges().
2016-08-04 12:30:14 -07:00
Eric Fiselier 7f1da4a68b Hide yet another test from MSVC 2013 2016-08-02 22:09:06 -06:00
Eric Fiselier 25eae60c4c Attempt to fix MSVC build failures 2016-08-02 19:33:41 -06:00
Eric 5f5ca31ce0 Add `RegisterBenchmark(name, func, args...)` for creating/registering benchmarks. (#259)
* Add RegisterBenchmark

* fix test inputs

* fix UB caused by unitialized value

* Add RegisterBenchmark

* fix test inputs

* fix UB caused by unitialized value

* Work around GCC 4.6/4.7/4.8 bug
2016-08-02 16:22:46 -07:00
Eric b7f8e355ee update travis, squash history (#265) 2016-08-02 14:47:29 -07:00
Eric Fiselier 44128d87d2 Add --benchmark_out=<filename> and --benchmark_out_format=<format> options.
These options allow you to write the output of a benchmark to the specified
file and with the specified format. The goal of this change is to help support
tooling.
2016-08-02 15:12:43 -06:00
Vadym f68e64c60a add support for step in DenseRange() (#260) 2016-07-25 21:13:09 +02:00
Eric Fiselier a2ca94dd10 Fix missing SYSTEM_INFO initializers 2016-07-23 16:22:13 -06:00
Eric Fiselier 3b2d0abd2b Use correct integer type for complexity. MSVC warns otherwise 2016-07-22 17:48:37 -06:00
Eric Fiselier 950c8b7440 Fix suprious failures in complexity_test.cc 2016-07-22 17:31:05 -06:00
Eric Fiselier 440df36e7a Improve failure mode in complexity_test.cc 2016-07-22 16:18:34 -06:00
Eric Fiselier 1bd62bd0be Revert "Workaround missing std::this_thread::sleep_for function in tests."
GCC 4.6 doesn't provide std::chrono::steady_clock and GCC 4.7 doesn't provide
std::this_thread::sleep_for. I would prefer to support GCC 4.7 but I'm
reverting this since the bots are GCC 4.6.

This reverts commit c5f454957d.
2016-07-22 15:42:26 -06:00
Eric Fiselier c5f454957d Workaround missing std::this_thread::sleep_for function in tests.
GCC 4.7 doesn't provide std::this_thread::sleep_for on my system.
This patch changes the tests to use a different method for wasting
cycles.
2016-07-22 15:35:26 -06:00
Sven ebd37b191c set cpuinfo_num_cpus on Windows (#254) 2016-07-13 13:00:32 -07:00
Elliott Hughes b805b7c6e0 Add missing `volatile`s to 32-bit ARM cycleclock assembler. (#253)
Without these, clang reorders these instructions as if they were
regular loads/stores which causes SIGILL from the kernel because
it performs all the loads before it starts testing the values.
2016-07-12 12:40:09 -07:00
Eric Fiselier 7e40ff9e35 Provide a better implementation of DoNotOptimize(...).
This implementation is less likely to ICE compilers, and is more correct.
It also acts as a memory barrier which will help prevent writes to global memory
from being optimized away.
2016-07-11 14:58:50 -06:00
Steve Downey 2149577f89 Add export linker flags policy to cmake (#251)
Add policy CMP0056, which honors the link flags in try_compile and
try_run. This allows for building against non-system libc++ by providing
the correct -L and rpath options in a containing project.

For example:

    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -L ${LLVM_ROOT}/lib -l c++ -l c++abi")
    set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -Wl,-rpath,${LLVM_ROOT}/lib")
2016-07-06 12:36:56 -07:00
Ismael 885ca41cf8 added check for SetComplexityN (#248) 2016-06-27 11:26:23 -07:00
Ismael a0783fd7f6 fix 244 (#247) 2016-06-27 11:25:43 -07:00
Nick d147797aac Add state.SetComplexityN to docs. (#245)
Add `state.SetComplexityN(state.range_x());` to documentation so that complexity is calculated correctly.
2016-06-27 11:24:13 -07:00
Ryan 1d53e5e0d8 Clarified output formats (#241) 2016-06-20 09:15:09 -07:00
Ismael 3fdd76bd14 fix issue 235 (#236) 2016-06-03 09:33:17 -07:00