Commit Graph

387 Commits

Author SHA1 Message Date
Joao Paulo Magalhaes 020bac985b Extend tabular counter tests to different counter sets. 2017-05-02 23:00:45 +01:00
Joao Paulo Magalhaes 2506044902 Add unit test for counter sets. 2017-05-02 22:14:49 +01:00
Joao Paulo Magalhaes cf20dc967f Add test for tabular output of rate counters. 2017-05-02 20:47:41 +01:00
Joao Paulo Magalhaes c69b385c9c Add first unit test for benchmark_tabular_counters. 2017-05-02 20:33:28 +01:00
Joao Paulo Magalhaes 17a012d754 Fix: --benchmark_counters_tabular was not being passed to tests. 2017-05-02 20:31:54 +01:00
Joao Paulo Magalhaes 77b9362b06 Add output_test.h to output_test_helper for VisualStudio editing. 2017-05-01 23:02:52 +01:00
Joao Paulo Magalhaes b57b2cfd77 Improve some comments. 2017-05-01 23:02:35 +01:00
Joao Paulo Magalhaes 64b5f3ff2d Make Results::GetTime() receive an enum. 2017-05-01 23:02:23 +01:00
Joao Paulo Magalhaes 62b1dd9c4a CHECK_BENCHMARK_RESULTS() was too inconspicuous. 2017-05-01 23:02:17 +01:00
Joao Paulo Magalhaes 3443ac2103 Fix brace formatting (Habits die hard!). 2017-05-01 23:02:11 +01:00
Joao Paulo Magalhaes 21600b966f Fix VS warning. 2017-05-01 23:01:57 +01:00
Joao Paulo Magalhaes 47226ccd56 CHECK(): rename EPS to FLOAT for consistency with googletest style. 2017-05-01 23:01:10 +01:00
Joao Paulo Magalhaes 2a2eb44b30 Fix VS2013 quirk. 2017-04-29 22:27:55 +01:00
Joao Paulo Magalhaes cdbcaaf2b6 Fix g++-4.8 compile errors. 2017-04-29 20:47:32 +01:00
Joao Paulo Magalhaes 8f69e4f6ce Remove whitespace. 2017-04-29 20:35:25 +01:00
Joao Paulo Magalhaes c81960a899 Add missing include of stringstream. 2017-04-29 20:31:44 +01:00
Joao Paulo Magalhaes b5effb30f9 Add missing include for providing std::function. 2017-04-29 20:24:26 +01:00
Joao Paulo Magalhaes 55876610f1 Remove unused prototype. 2017-04-29 20:02:19 +01:00
Joao Paulo Magalhaes 180719d0d6 Rename ResultsCheckerEntry to Results. 2017-04-29 20:01:18 +01:00
Joao Paulo Magalhaes f3b82a8eda Adopt standard style. 2017-04-29 19:44:13 +01:00
Joao Paulo Magalhaes 86249c57a5 Result checking: move some function definitions to source file. 2017-04-29 19:40:39 +01:00
Joao Paulo Magalhaes 03b0655d12 Fix expected values of user counters as rates in unit tests. 2017-04-29 19:35:43 +01:00
Joao Paulo Magalhaes 78548f8c6e Add (currently failing) tests for user counters with threads. 2017-04-29 19:26:34 +01:00
Joao Paulo Magalhaes 2814e9d8dc Fix ResultsCheckerEntry::NumThreads() 2017-04-29 19:25:51 +01:00
Joao Paulo Magalhaes 92034a8b84 Make result checkers execute on all regex-matching benchmarks. 2017-04-29 19:02:07 +01:00
Joao Paulo Magalhaes 738fcd9e6a Add log of the benchmark name when checking results. 2017-04-29 18:30:28 +01:00
Joao Paulo Magalhaes 1ce286f632 Avoid compiler-specific pragmas in result check macros.
- Epsilon is now understood as relative to expected value.
- Improve error messages for epsilon checks.
2017-04-29 18:26:30 +01:00
Joao Paulo Magalhaes da69e5de45 User counters: add more tests. 2017-04-28 20:45:30 +01:00
Joao Paulo Magalhaes 8c757a3bb9 Results check: add checks with epsilon. 2017-04-28 20:44:27 +01:00
Joao Paulo Magalhaes 1826feb164 ResultsCheckerEntry: add more getter functions. 2017-04-28 20:43:44 +01:00
Joao Paulo Magalhaes 2a8d0dd1b1 Use const char* instead of std::string in entry name lookup. 2017-04-28 20:42:28 +01:00
Joao Paulo Magalhaes e869e3749a Remove some whitespace. 2017-04-28 15:38:21 +01:00
Joao Paulo Magalhaes 6452883027 Unit testing: add facilities to check benchmark results.
This is needed for examining the values of user counters (needed
for #348). It is also needed for checking the values of standard
benchmark results like items_processed or complexities (for example,
checking the standard deviation is needed for unit testing #357
as discussed in #362).
2017-04-28 15:02:27 +01:00
Joao Paulo Magalhaes 693a43013d User counters: add more unit tests. ...
The tests are still missing a way to check actual validity of
numerical results; this will be done next. As they currently are,
the tests pass, but the problem detected with #378 is still
standing and the results with non-standard counters are wrong.
2017-04-27 22:11:40 +01:00
Joao Paulo Magalhaes 3c2d7f5348 User counter tests: first version. 2017-04-27 19:25:20 +01:00
Joao Paulo Magalhaes b273d9b7d5 Reporter tests: reuse csv header. 2017-04-27 19:24:06 +01:00
Dmitry Trifonov 7a74b74856 fix for android NDK r10e (#375) 2017-04-20 20:07:52 -06:00
Dmitry Trifonov 09b93ccc6a fix android compilation (#372)
* fix android compilation

* checking __GLIBCXX__ and __GLIBCPP__ macro in addition to __ANDROID__

* using vsnprintf instead of std::vsnprintf to compile on Android

* removed __GLIBCPP__ check on Android

* StringPrintF instead of std::to_string for Android
2017-04-18 09:48:07 -06:00
Eric 74b24058ad Add Benchmark::Iterations for explicit iteration count control - Fixes #370 (#373)
* Add Benchmark::Iterations for explicitly specifying the number of iterations to use.

* Document that benchmark::Iterations should not be used to limit benchmark runtimes
2017-04-17 21:29:28 -07:00
Eric Fiselier 7f87c98d36 Enable <cassert> by removing -DNDEBUG when running the tests.
In non-debug builds CMake automatically adds -DNDEBUG, this means
that uses of `assert` in the tests are disabled for non-debug builds.
Obviously we want these tests to run, regardless of configuration.

This patch strips -DNDEBUG during non-debug builds and adds
-UNDEBUG just to be sure.
2017-04-17 20:53:39 -06:00
rolandschulz 9b92ed76a8 Fix ICC compiler warnings (#358)
fixes #354

The build fails with ICC17 because of warnings and Werror. What is the correct solution to fix it?
Should a patch

disable Werror for ICC (or maybe all non known compilers)
disable the false postive warnings for all files. This could be done using:
add_cxx_compiler_flag(-wd2102) #ICC17u2: Many false positives for Wstrict-aliasing
add_cxx_compiler_flag(-wd2259) #ICC17u2: non-pointer conversion from "long" to "int" may lose significant bits (even for explicit static cast, sleep.cc(44))
add_cxx_compiler_flag(-wd654) #ICC17u2: overloaded virtual function "benchmark::Fixture::SetUp" is only partially overridden (because of deprecated overload)
disable warnings at file level or some other granularity
2017-03-27 18:30:54 -06:00
jpmag a9a66c85bb Add user-defined counters. (#262)
* Added user counters, and move use of bytes_processed and items_processed to user counter logic.

Each counter is a string-value pair. The counters were
made available through the State class. Two helper virtual
methods were added to the Fixture class to allow convenient
initialization and termination of the counters: InitState()
and TerminateState(). The reporting of the counters is buggy
and is still a work in progress, to be completed in the next commits.

* fix bad removal of BenchmarkCounters code during the merge

* add myself to AUTHORS/CONTRIBUTORS

* fix printing to std::cout in csv_reporter

* bytes_per_second and items_per_second are now in the UserCounters class

* add user counters to json reporter

* moving bytes_per_second and items_per_second to their old state

* console reporter dealing ok with user counters.

* update unit tests for user counters

* CSVReporter now prints user counters too.

* cleanup user counters

* reverted changes to cmake files which should have gone into later commits

* fixture_test: fix gcc 4.6 compilation

* remove ctor with default argument

see https://github.com/google/benchmark/pull/262#discussion_r72298055

* use (auto-defined) BENCHMARK_HAS_CXX11 instead of BENCHMARK_INITLIST.

https://github.com/google/benchmark/pull/262#discussion_r72298310

* leanify counters API

Discussions:
API complexity: https://github.com/google/benchmark/pull/262#discussion_r72298731
remove std::string dependency (WIP): https://github.com/google/benchmark/pull/262#discussion_r72298142
spacing & alignment: https://github.com/google/benchmark/pull/262#discussion_r72298422

* remove std::string dependency on public API - changed counter name storage to char*

* Counter ctor: use overloads instead of default arguments

discussion:
https://github.com/google/benchmark/pull/262#discussion_r72298055

* Use raw pointers to remove dependency on std::vector from public API .

For more info, see discussion at https://github.com/google/benchmark/pull/262#discussion_r72319678 .

* Move counter implementation from benchmark.cc to counter.cc.

    See discussion: https://github.com/google/benchmark/pull/262#discussion_r72298980 .

* Remove unused (commented-out) code.

* Moved thread counters to ThreadStats.

* Counters: fixed copy and move constructors.

* Counter: use an inplace buffer for small names.

* benchmark_test: move counters test out of CXX11 preprocessor conditional.

* Counter: fix VS2013 compilation error in char[] initialization.

* Fix typo.

* Expose counters from State.

See discussion: https://github.com/google/benchmark/pull/262#issuecomment-237156951

* Changed counters interface to map-like.

* Fix printing of user counters in ConsoleReporter.

* Applied clang-format to counter.cc and console_reporter.cc.

Command was `clang-format -style=Google -i counter.cc console_reporter.cc`
I also applied to all other files, but the changes were very
far-reaching so I rolled those back.

* Rename Counter::Flags_e to Counter::Flags

* Fix use of reserved names in Counter and BenchmarkCounters.

* Counter: Fix move ctor bug + change order of members.

* Fixture: remove tentative methods InitState() and TerminateState().

* Update fixture_test to the new Fixture interface.

* BenchmarkCounters: fixed a bug in the move ctor. Remove call to CHECK_LT().

CHECK_LT() was making the size_t lookup take ~double the time of a string lookup!

* BenchmarkCounters: add option to not print zero counters (defaults to false).

* Add test to compare counter storage and access with std::map.

* README: clarify cost of counter access modes.

* move counter access test to an own test.

* BenchmarkCounters: add move Insert()

* Counters access test: add accelerated lookup by name.

* Fix old range syntax.

* Fix missing include of cstdio

* Fix Visual Studio warning

* VS2013 and lower: fix use of snprintf()

* VS2013: fix use of char[] as a member of std::pair<>.

* change counter storage to std::map

* Remove skipZeroCounters logic

* Fix VS compilation error.

* Implemented request changes to PR #262.

* PR #262: More requested changes.

* README: cleanup counter text.

* PR #262: remove clang-format changes for preexisting code

* Complexity+Counters: fix counter flags which were being ignored.

* Document all Counter::Flag members

* fixed loss of counter values

* ConsoleReporter: remove tabular printing of user counters.

* ConsoleReporter: header printing should not be contingent on user counter names.

* Minor white space and alignment fixes.

* cxx03_test + counters: reuse the BM_empty() function.

* user counters: add note to README on how counters are gathered across threads
2017-03-01 17:23:42 -07:00
Marek Kurdej 0064c56abd Add tests for reporters (#307)
* Test bytes_per_second and items_per_second.

* Test SetLabel.

* Reformat.

* Make State::error_occurred_ private.

* Fix tests with floats.

* Merge private blocks
2016-10-28 09:13:57 -07:00
Marek Kurdej 3f23832a09 Allow calling Args and ArgNames in any order. 2016-10-26 09:36:39 +02:00
Marek Kurdej c1c01b2cd3 Handle the case when the argument name is an empty string. 2016-10-25 09:45:35 +02:00
Marek Kurdej 17e1c405dd Add ArgName() and ArgNames() methods to name arguments/ranges. 2016-10-24 09:57:40 +02:00
Marek Kurdej 8aeb20f16b Add tests. 2016-10-21 14:59:06 +02:00
Eric Fiselier 36a251ab3e Fix #300. Emit RMS as a float not a percentage in JSON 2016-10-07 22:26:01 -06:00
Eric Fiselier 2555035f44 Use correct RE for floating point numbers in reporter_output_test.cc 2016-10-07 21:56:22 -06:00
Dominic Hamon 1100e91907 Simplify clang-format and apply to tests (#302) 2016-10-07 11:04:50 -07:00
Ronny 72be9523bb Add Benchmark::ThreadRange() version with increment instead of multiply (#283)
* add additive thread range

* add test
2016-09-03 13:56:46 -06:00
Eric Fiselier 309ecb737f Increase complexity_test runtime slightly less. 2016-09-03 00:18:02 -06:00
Eric Fiselier 756f069918 Increase complexity_test.cc runtime to attempt to avoid appveyor failures 2016-09-03 00:06:51 -06:00
biojppm 83561f0580 CMake: add headers to the list of source files. (#284)
cmake does not require this, but IDEs such as VisualStudio or QtCreator
need this to add the headers to the generated project.
2016-08-31 15:28:43 -07:00
Eric 6e25917001 Add -DBENCHMARK_USE_LIBCXX:BOOL=<value> option. (#280)
* Add BENCHMARK_USE_LIBCXX option.

* Add comments

* re-add zero null pointer warning
2016-08-29 12:43:30 -06:00
Eric 49bbff2c66 Supply old `RangePair` and `ArgPair` API for C++03 compatibility. (#278)
* changes

* remove other changes

* remove unneeded test

* cleanup unused include
2016-08-29 10:59:46 -07:00
Eric Fiselier db1af86d16 Fix out-of-bounds std::vector access.
In the `Ranges(...)` generation code a "control" vector which stores
the current index for each range passed to `Ranges`. Previously this vector
was incorrectly initialized to the size of the subranges not the number
of subranges.

Additionally this patch suppresses unused warnings generated by
`stream_init_anchor`.
2016-08-28 23:07:38 -06:00
Eric Fiselier aaa25ac376 Improve diagnostic output for output tests. 2016-08-28 14:22:11 -06:00
Eric 0ed4456097 Refactor output test runner into standalone module. (#277)
* refactor

* Move default substitutions into library

* Move default substitutions to the *right* place in the library

* Fix init order issues that caused test failures

* improve diagnostics

* add missing include

* general cleanup

* Address review comments
2016-08-28 13:24:16 -06:00
Eric a11fb69c89 Add --benchmark_report_aggregates_only={true|false} flag for better summary output. (#267) 2016-08-10 18:20:54 -06:00
Eric Fiselier a7a7c56152 Workaround flaky complexity_test.cc test case.
See https://github.com/google/benchmark/issues/272
2016-08-09 14:14:15 -06:00
Eric Fiselier ee54a3f03e Rework Appveyor config
Currently the Appveyor bot is a PIT. It never passes and it often hangs
or gives very poor output. This patch rewrites the configuration.

This patch also attempts to fix a flaky complexity test as a drive-by.
2016-08-08 02:04:59 -06:00
Eric Fiselier 8267117a92 Add ASAN and UBSAN builders to travis for both GCC and Clang.
This patch adds new builders that test against GCC 6 and Clang 3.8 respectivly.
They also enable both address and undefined sanitizer. MSAN currently won't work
since it requires a sanitized STL.
2016-08-07 19:25:19 -06:00
Eric Fiselier f30028b2a8 Attempt to fix VC 2013 build.
VC 2013 injects valid when  assigning an initializer list to std::set.
This attempts to work around this issue by using std::set's constructors
instead of the assignment operators.
2016-08-07 14:57:39 -06:00
Marcin Kolny dfe0260754 Support multiple ranges in the benchmark (#257)
* Support multiple ranges in the benchmark

google-benchmark library allows to provide up to two ranges to the
benchmark method (range_x and range_y). However, in many cases it's not
sufficient. The patch introduces multi-range features, so user can easily
define multiple ranges by passing a vector of integers, and access values
through the method range(i).

* Remove redundant API

Functions State::range_x() and State::range_y() have been removed. They should
be replaced by State::range(0) and State::range(1).
Functions Benchmark::ArgPair() and Benchmark::RangePair() have been removed.
They should be replaced by Benchmark::Args() and Benchmark::Ranges().
2016-08-04 12:30:14 -07:00
Eric Fiselier 7f1da4a68b Hide yet another test from MSVC 2013 2016-08-02 22:09:06 -06:00
Eric 5f5ca31ce0 Add `RegisterBenchmark(name, func, args...)` for creating/registering benchmarks. (#259)
* Add RegisterBenchmark

* fix test inputs

* fix UB caused by unitialized value

* Add RegisterBenchmark

* fix test inputs

* fix UB caused by unitialized value

* Work around GCC 4.6/4.7/4.8 bug
2016-08-02 16:22:46 -07:00
Eric Fiselier 44128d87d2 Add --benchmark_out=<filename> and --benchmark_out_format=<format> options.
These options allow you to write the output of a benchmark to the specified
file and with the specified format. The goal of this change is to help support
tooling.
2016-08-02 15:12:43 -06:00
Eric Fiselier 950c8b7440 Fix suprious failures in complexity_test.cc 2016-07-22 17:31:05 -06:00
Eric Fiselier 440df36e7a Improve failure mode in complexity_test.cc 2016-07-22 16:18:34 -06:00
Eric Fiselier 1bd62bd0be Revert "Workaround missing std::this_thread::sleep_for function in tests."
GCC 4.6 doesn't provide std::chrono::steady_clock and GCC 4.7 doesn't provide
std::this_thread::sleep_for. I would prefer to support GCC 4.7 but I'm
reverting this since the bots are GCC 4.6.

This reverts commit c5f454957d.
2016-07-22 15:42:26 -06:00
Eric Fiselier c5f454957d Workaround missing std::this_thread::sleep_for function in tests.
GCC 4.7 doesn't provide std::this_thread::sleep_for on my system.
This patch changes the tests to use a different method for wasting
cycles.
2016-07-22 15:35:26 -06:00
Ismael 3fdd76bd14 fix issue 235 (#236) 2016-06-03 09:33:17 -07:00
Ismael 8ba94b4c18 changed global string to const char * 2016-06-02 22:40:21 +02:00
Ismael 240ba4e64e changed BigOFunc argument from size_t to int 2016-06-02 22:21:52 +02:00
Ismael 109f528a40 removed functional library not needed 2016-06-02 19:48:53 +02:00
Ismael 212cfe1c2e removed check on automatic fit, to avoid random convergence misfits breaking the build 2016-06-02 19:01:10 +02:00
Ismael 8c73d49b77 fixed reporter_output_test 2016-06-01 23:13:10 +02:00
Ismael 867f9145a0 added lambdas to complexity report 2016-06-01 23:08:01 +02:00
Eric Fiselier 74a278e206 Fix flaky test 2016-05-30 17:13:41 -06:00
Eric 1b263fe6d9 Cleanup reporters (#226)
* Move ComputeStats call out of the reporters

* Cleanup adjusted time calculations in reporters

* Move ComputeBigO call out of reporters

* Remove ReportComplexity interface using ReportRuns instead

* Factor out reporting of basic context information

* Attempt to fix GCC 4.6 build errors

* Move ComputeStats to complexity.cc
2016-05-27 16:45:25 -06:00
Eric 238e558fdb Allow benchmarks to take arbitrary arguments. (#221)
* Add lambda benchmarks

* Remove lambda capture since the lambda is not at a block scope

* Remove LambdaBenchmark helper since FunctionBenchmark can be used with non-capturing lambas

* Add lambda benchmarks

* Remove lambda capture since the lambda is not at a block scope

* Remove LambdaBenchmark helper since FunctionBenchmark can be used with non-capturing lambas

* Add more docs for BENCHMARK_CAPTURE.

* Fix use of misnamed parameter

* Guard BENCHMARK_CAPTURE tests against non-c++11 compilers

* Move tests out of basic_test.cc
2016-05-27 12:37:10 -07:00
Eric 5686bf1b38 Change reporters to use a specified output and error stream. Add tests for output. (#219)
* Add test for reporter output.

* setup err_stream tests

* Fix warnings in tests

* whitespace

* Fix build errors caused by super pedantic compilers

* Pass streams by pointer not non-const reference
2016-05-27 12:34:37 -07:00
Dominic Hamon d6bc7e1581 Merge branch 'update_complexity' of git://github.com/ismaelJimenez/benchmark into ismaelJimenez-update_complexity 2016-05-26 14:01:28 -07:00
Albert Pretorius 74e82e822f Force DoNotOptimize operand to memory for both gcc and clang 2016-05-26 07:43:07 +01:00
Ismael 290ac9ee0e updated complexity_test.cc to new interface for auto 2016-05-25 23:19:32 +02:00
Ismael e246699f25 added auto as default value for complexity 2016-05-25 21:18:56 +02:00
Dominic Hamon 9686f1769e Merge branch 'skip_with_error' of git://github.com/efcs/benchmark into efcs-skip_with_error 2016-05-25 09:00:31 -07:00
Eric Fiselier 84bc4d703b Add a per benchmark 'Repetitions' option. 2016-05-24 21:52:23 -06:00
Eric Fiselier b195f82554 Merge branch 'master' into skip_with_error 2016-05-24 15:45:02 -06:00
Eric Fiselier 1080b17bf5 Fix build error with libc++ 2016-05-24 15:09:31 -06:00
Eric Fiselier bdeb38718e merge 2016-05-24 15:01:07 -06:00
Dominic Hamon 2440b752fd Formatting updates 2016-05-24 13:26:32 -07:00
Dominic Hamon a86545874a Merge branch 'ismaelJimenez-complexity' 2016-05-24 13:15:55 -07:00
Dominic Hamon 31cdabf6bb Merge pull request #216 from efcs/diagnostic-checks
Add checks that <Resume|Pause>Timing functions are not called outside of the KeepRunning() loop. Fixes #204
2016-05-24 13:09:41 -07:00
Ismael 36a9ae197f added SetComplexityN 2016-05-24 19:56:49 +02:00
Eric Fiselier 9341d705a1 Change --benchmark_list_tests to respect the benchmark filter.
This behavior mirrors how GTest handles these two flags.
2016-05-24 00:42:11 -06:00
Eric Fiselier e0de8171c6 Change RunSpecifiedBenchmarks to return the number of benchmarks run. Fixes #145 2016-05-23 23:35:09 -06:00
Eric Fiselier c7108df977 Only compile and run the C++03 test when -std=c++03 is supported. Fixes #164 2016-05-23 20:44:10 -06:00
Eric Fiselier 43017f8b15 Add SkipWithError(...) function. 2016-05-23 19:24:56 -06:00
Eric Fiselier 029f37446d Allow test to compile with -fno-exceptions 2016-05-23 15:38:30 -06:00
Eric Fiselier 481e06e65a Address review comments 2016-05-23 15:27:43 -06:00
Eric Fiselier 751e07d42d add newline 2016-05-23 15:07:54 -06:00
Eric Fiselier 69d1a52407 Add checks that <Resume|Pause>Timing functions are not called outside of the benchmark. Fixes #204 2016-05-23 15:05:55 -06:00
Ismael 43ef17441c refactor names 2016-05-23 20:50:35 +02:00
Ismael ac05c04533 refactor MinimalLEastSq 2016-05-23 20:12:54 +02:00
Ismael 290bd60289 Refactor for pull request 2016-05-21 11:51:42 +02:00
Ismael 2e5c397b48 implemented complexity reporting 2016-05-21 08:55:43 +02:00
Ismael 872ff01a49 addaptation of minimal_leastsq library 2016-05-20 16:49:39 +02:00
Ismael b73dc22944 implemented Complexity for O(1) 2016-05-18 21:25:00 +02:00
Ismael 27f0baa190 Added test file and Complexity() interface 2016-05-18 19:59:34 +02:00
Ismael c60eefdbb7 added option to change range multiplier 2016-05-14 15:56:34 +02:00
Billy Robert O'Neal III df9ab80113 Use nanoseconds instead of duration<double, milli>
MSVC++ before 2015 Update 2 has a bug in sleep_for where it tries to
implicitly += the input with a nanoseconds variable. Work around this by
using nanoseconds directly (which can be implicitly +='d with
chrono::nanoseconds).
2016-05-10 18:02:02 -07:00
Jussi Knuuttila e253a28402 Manual timing support. 2016-05-05 23:24:13 +03:00
Kai Wolf fb733897c5 Remove sleep.h dependency for tests 2016-05-02 19:53:45 +02:00
Kai Wolf 1043f45bed Add explanatory comment why src headers are needed in test/CMakeLists.txt 2016-03-29 22:10:07 +02:00
Kai Wolf 7c69b36078 Add an additional parameter for time units 2016-03-26 01:04:17 +01:00
Ian Henriksen 0dca953b13 Include cstdlib in map_test to make sure std::rand is available when
building with libc++.
2016-02-24 12:31:31 -07:00
Mohamed Amin JABRI 53aca9bc71 Pass const State to Fixture::TearDown. Fix memory leak in fixture_test 2016-02-24 17:36:34 +09:00
Dominic Hamon 31e71be77c Merge pull request #181 from google/map_test
Pass const state to Fixture::SetUp. Add map_test.
2016-02-16 02:08:29 -08:00
Jakob Buchgraber 007efee751 Add number of threads to State.
Having access to the thread count from within a benchmark is useful,
for when one wants to distribute a workload dynamically among the
benchmarks running in parallel e.g when using ThreadRange() or
ThreadPerCpu().
2016-02-15 21:39:15 +01:00
Dominic Hamon 53068f974c Pass const state to Fixture::SetUp. Add map_test. 2016-02-15 03:04:19 -08:00
Anton Danielsson c8f4690fd4 Sorted include of headers. 2015-10-09 08:47:36 +02:00
Anton Danielsson 6abd53777b Use stringstream instead of atoi to avoid sign error.
The sane thing here would be to use std::stoul but this is not available in the android-ndk...
2015-10-09 08:47:36 +02:00
Anton Danielsson df0df4aba9 Fixed tests so they build on android.
- Added missing header to test/benchmark_test.cc
- Changed std::stoul to std::atol in test/filter_test.cc because of a
  limitation in the android-ndk (http://stackoverflow.com/questions/17950814/how-to-use-stdstoul-and-stdstoull-in-android)
2015-10-09 08:47:35 +02:00
izaid 14c4ce5ebf Removed unneeded #include <cstddef> 2015-10-06 15:10:15 +01:00
Eric Fiselier ed0a2eb741 use size_t instead of int for the iteration count 2015-10-01 15:08:44 -06:00
Greg Miller 2463339d0b Updates Initialize() to work with an argv as 'char**' 2015-09-30 16:14:50 -04:00
Eli Bendersky 1dd40c99c0 Custom arguments function usage with Apply - fix doc and add test. 2015-09-23 12:47:54 -07:00
David Coeurjolly 68e69213e7 Fixing cmake issue with empty cmake_build_type 2015-05-22 23:15:19 +02:00
Matt Clarkson 36c1eb6975 Coverage support with LCOV and GCOV
This patch adds a `coverage` target that allows coverage statisitics to be
retrieved for the project. It requires that lcov and gcov is installed and
that the generator is unix makefiles but this can be improved upon in
future releases.

To make it work use the coverage build type:

```
cmake -DCMAKE_BUILD_TYPE=Coverage .
make coverage
```
2015-05-13 09:17:38 +01:00
Eric Fiselier 9c25a67c1a address more review comments 2015-04-06 19:04:12 -04:00
Eric Fiselier 12f4405870 add ability to use benchmark filters 2015-04-06 17:00:06 -04:00
Eric Fiselier e97a96f9e7 redo filter test 2015-03-31 00:05:02 -04:00
Eric Fiselier 9d47552020 merge new changes in basic_test.cc 2015-03-27 16:37:53 -04:00
Eric Fiselier 9ed1082c33 Merge branch 'master' into new-benchmark-interface 2015-03-27 16:36:40 -04:00
Eric Fiselier e428b9eec3 Add 'benchmark::DoNotOptimize(...)' to help users prevent optimizations 2015-03-27 16:35:46 -04:00
Eric Fiselier 4f9ce87e8c Merge branch 'master' into new-benchmark-interface 2015-03-27 16:26:36 -04:00
Eric Fiselier c5f238b18c Fix column width calculation and remove duplicate test 2015-03-27 12:28:22 -04:00
Eric Fiselier 4bf6ceb50d Change the available benchmark options 2015-03-26 23:37:26 -04:00
Eric Fiselier 1b9e55c281 Remove walltime implementation tests and expose less internals 2015-03-26 17:56:27 -04:00
Eric Fiselier 21207a4df5 improve walltime test 2015-03-26 15:25:08 -04:00
Eric Fiselier c0755c4226 actually add new test 2015-03-26 14:57:03 -04:00
Eric Fiselier 725f1f066d add walltime benchmark and fix unused variable. 2015-03-26 14:56:52 -04:00
Eric Fiselier be993acbb3 remove unneeded test repetitions 2015-03-26 14:33:48 -04:00
Eric Fiselier d0df7ef2de cleanup whitespace and includes 2015-03-26 14:30:16 -04:00
Eric Fiselier a3308c6d86 Introduce use of std::chrono and remove Walltime printing 2015-03-26 14:26:07 -04:00
Eric Fiselier 482c19d92e Handle creation of C++03 flags differently 2015-03-18 17:08:15 -04:00
Eric Fiselier d40cb97713 Don't treat CMAKE_CXX_FLAGS as a list 2015-03-18 17:05:53 -04:00
Eric Fiselier daa8a67aa5 add C++03 test and update README 2015-03-18 16:34:43 -04:00
Eric Fiselier 20f1c0e2a8 Apply reporter interface changes. Make report methods non-const and add a Finalize method. 2015-03-17 16:16:36 -04:00
Eric Fiselier e45252e6f5 move ConsoleReporter out of internal namespace 2015-03-17 14:15:16 -04:00
Eric Fiselier d71660227b fix minimal_benchmark.h include 2015-03-17 14:02:02 -04:00
Eric Fiselier ef79d0af9f move basic_test over to minimal benchmark 2015-03-17 13:48:33 -04:00
Eric Fiselier 279e502a05 add test for benchmark with two template parameters 2015-03-17 13:32:20 -04:00
Eric Fiselier efe591015d address review comments 2015-03-17 12:31:46 -04:00
Eric Fiselier 66bf7c8f71 add floating point comparison warnings 2015-03-12 20:27:29 -04:00
Eric Fiselier 7a767012f1 Adopt new benchmark timing internals.
This patch adopts a new internal structure for how timings are performed.
Currently every iteration of a benchmark checks to see if it has been running
for an appropriate amount of time. Checking the clock introduces noise into
the timings and this can cause inconsistent output from each benchmark.

Now every iteration of a benchmark only checks an iteration count to see if it
should stop running. The iteration count is determined before hand by testing
the benchmark on a series of increasing iteration counts until a suitable count
is found. This increases the amount of time it takes to run the actual benchmarks
but it also greatly increases the accuracy of the results.

This patch introduces some breaking changes. The notable breaking changes are:
1. Benchmarks run on multiple threads no generate a report per thread. Instead
   only a single report is generated.
2. ::benchmark::UseRealTime() was removed and replaced with State::UseRealTime().
2015-03-12 18:03:33 -04:00
Dominic Hamon d68127d8ad Enable UseRealTime and fix documentation for SetLabel.
Fixes #89

UseRealTime was defined in the internal namespace by mistake.
Similarly, documentation suggested that benchmark::SetLabel should be
used to set a label, and a function was declared but not defined, while
actually the call should be 'state.SetLabel'.
2015-03-11 09:47:15 -07:00
Eric Fiselier e975efdb7a remove third_party directory. update test args 2015-03-10 14:38:45 -04:00
Dominic Hamon 3b40f0a7a3 Add filter test, remove re test, and googletest deps 2015-03-09 20:30:14 -07:00
Eric Fiselier a187aa0803 Third step in moving towards adopting new timer semantics 2015-03-06 17:01:05 -05:00
Eric Fiselier f947cebe15 remove ifdefs around BM_Factorial 2015-03-06 16:59:32 -05:00
Eric Fiselier 8f96f50553 comment out test to prevent failures. 2015-03-06 16:53:52 -05:00
Eric Fiselier 8ed7d7664b The second step towards merging the timer changes.
This patch does two things:

1. It overhalls the static initialization in Walltime to be simplier. It uses
   a static variable inside WallTime::Now() to initialize the timer.

2. Add a logging mechanism so that the -v flag actually has meaning and
   reimplement the CHECK macros to allow extra messages to be streamed in.
2015-03-06 14:07:24 -05:00
Eric Fiselier b1f34d3880 address comments 2015-03-06 13:20:47 -05:00
Eric Fiselier 5b41e128b3 Step one towards merging timer changes.
This patch cleans up our use of generic macros and also merges changes in the
build system.

It adds options -DBENCHMARK_ENABLE_TESTING and -DBENCHMARK_ENABLE_SHARED.
2015-03-06 12:35:00 -05:00
Dominic Hamon 3968ff45ba Fix #72 by avoiding 64-to-32-bit shortenings 2015-02-18 11:18:40 -08:00
Dominic Hamon b40b66ab3b Actually fix issue with CHECK 2014-12-02 12:58:14 -08:00
Dominic Hamon d31977bb69 Added gtest CHECK symbol to benchmark test 2014-12-02 12:51:46 -08:00
Lei Xu 0f78cebba5 Use #ifdef to protect calling Factorial in benchmark_test.cc 2014-11-13 16:09:21 -08:00
Dominic Hamon 7e290fb965 Fix release builds 2014-11-13 13:12:28 -08:00
Dominic Hamon a3b5e44c52 Remove CHECK_* from public API.
CHECK_* are now private and used internally in the library. The test
uses have been replaced with asserts.

Fixes #62.
2014-10-31 21:54:55 -07:00
Niklas Hofmann b04211cc88 Merge branch 'master' of github.com:google/benchmark 2014-10-25 18:35:35 +02:00
Niklas Hofmann 3e9264dcf1 added googletest dependency to tests (allows compilation with -j) 2014-10-16 19:17:08 +02:00
Dominic Hamon a13627bc3a Move pthread dependency to test CMakeLists.txt 2014-10-11 14:57:45 -07:00
Matt Clarkson 0fd0be9789 Added more complicated regex test patterns 2014-08-22 14:55:46 +01:00
Matt Clarkson 1176936966 Check the number of benchmark tests ran
Previously the benchmark_test program executed the benchmark tests to make sure
the API was working but was not checking the number of tests that were
completed. If the regular expression matching breaks, zero tests could be ran.
Similarly, if one of the APIs breaks and doesn't run the correct amount of tests
then `make test` will catch this.
2014-08-15 11:09:23 +01:00
Matt Clarkson 373cc41100 C++11 concurrency instead of pthread 2014-08-06 10:50:21 -07:00
Paul Redmond 52b3004ae3 fix linker error by reordering link libraries 2014-07-23 10:35:42 -04:00
Chris Kennelly fa908926c7 Partially resolve google/benchmark#17 by fixing regular expression leak.
This adds a unit test to validate the wrapper without running the entirety of
benchmark_test.
2014-04-23 00:56:17 -07:00
Chris Kennelly e38fde6450 Setup targets from each directory, rather than the top-level. 2014-04-23 00:47:07 -07:00
Shuo Chen 664159372c Use a new container in each round of push_back test.
Otherwise containers keep growing and uses gigabytes of memory.
2014-04-17 10:04:02 -07:00
Dominic Hamon 3a01f8320b Add items/bytes processed to SetInsert benchmark test 2014-01-07 14:31:14 -08:00
Dominic Hamon 2923a481ed Multithreaded tests are reenabled 2014-01-07 13:15:26 -08:00
Dominic Hamon 15bf66750c Really drop multithreading support 2013-12-20 15:10:46 -08:00
Dominic Hamon 403f354423 Initial commit
Benchmark library builds and runs but only single-threaded. Multithreaded
support needs a bit more love.

Currently requires some C++11 support (g++ 4.6.3 seems to work).
2013-12-18 16:58:23 -08:00