dolysis
/
benchmark
mirror of https://github.com/google/benchmark.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
benchmark/test/CMakeLists.txt

127 lines
4.6 KiB
CMake
Raw Normal View History

Step one towards merging timer changes. This patch cleans up our use of generic macros and also merges changes in the build system. It adds options -DBENCHMARK_ENABLE_TESTING and -DBENCHMARK_ENABLE_SHARED.
2015-03-06 17:35:00 +00:00
# Enable the tests
Move pthread dependency to test CMakeLists.txt
2014-10-11 21:57:45 +00:00
find_package(Threads REQUIRED)
Refactor output test runner into standalone module. (#277) * refactor * Move default substitutions into library * Move default substitutions to the *right* place in the library * Fix init order issues that caused test failures * improve diagnostics * add missing include * general cleanup * Address review comments
2016-08-28 19:24:16 +00:00
add_library(output_test_helper STATIC output_test_helper.cc)
Step one towards merging timer changes. This patch cleans up our use of generic macros and also merges changes in the build system. It adds options -DBENCHMARK_ENABLE_TESTING and -DBENCHMARK_ENABLE_SHARED.
2015-03-06 17:35:00 +00:00
macro(compile_benchmark_test name)
add_executable(${name} "${name}.cc")
Add filter test, remove re test, and googletest deps
2015-03-10 03:30:14 +00:00
target_link_libraries(${name} benchmark ${CMAKE_THREAD_LIBS_INIT})
Step one towards merging timer changes. This patch cleans up our use of generic macros and also merges changes in the build system. It adds options -DBENCHMARK_ENABLE_TESTING and -DBENCHMARK_ENABLE_SHARED.
2015-03-06 17:35:00 +00:00
endmacro(compile_benchmark_test)
Refactor output test runner into standalone module. (#277) * refactor * Move default substitutions into library * Move default substitutions to the *right* place in the library * Fix init order issues that caused test failures * improve diagnostics * add missing include * general cleanup * Address review comments
2016-08-28 19:24:16 +00:00
macro(compile_output_test name)
add_executable(${name} "${name}.cc")
target_link_libraries(${name} output_test_helper benchmark ${CMAKE_THREAD_LIBS_INIT})
endmacro(compile_output_test)
Setup targets from each directory, rather than the top-level.
2014-04-23 07:47:07 +00:00
# Demonstration executable
Step one towards merging timer changes. This patch cleans up our use of generic macros and also merges changes in the build system. It adds options -DBENCHMARK_ENABLE_TESTING and -DBENCHMARK_ENABLE_SHARED.
2015-03-06 17:35:00 +00:00
compile_benchmark_test(benchmark_test)
redo filter test
2015-03-31 04:05:02 +00:00
add_test(benchmark benchmark_test --benchmark_min_time=0.01)
Partially resolve google/benchmark#17 by fixing regular expression leak. This adds a unit test to validate the wrapper without running the entirety of benchmark_test.
2014-04-23 07:56:17 +00:00
Add filter test, remove re test, and googletest deps
2015-03-10 03:30:14 +00:00
compile_benchmark_test(filter_test)
redo filter test
2015-03-31 04:05:02 +00:00
macro(add_filter_test name filter expect)
add_test(${name} filter_test --benchmark_min_time=0.01 --benchmark_filter=${filter} ${expect})
Change --benchmark_list_tests to respect the benchmark filter. This behavior mirrors how GTest handles these two flags.
2016-05-24 06:42:11 +00:00
add_test(${name}_list_only filter_test --benchmark_list_tests --benchmark_filter=${filter} ${expect})
redo filter test
2015-03-31 04:05:02 +00:00
endmacro(add_filter_test)
add_filter_test(filter_simple "Foo" 3)
add_filter_test(filter_suffix "BM_.*" 4)
add_filter_test(filter_regex_all ".*" 5)
add_filter_test(filter_regex_blank "" 5)
add_filter_test(filter_regex_none "monkey" 0)
add_filter_test(filter_regex_wildcard ".*Foo.*" 3)
add_filter_test(filter_regex_begin "^BM_.*" 4)
add_filter_test(filter_regex_begin2 "^N" 1)
add_filter_test(filter_regex_end ".*Ba$" 1)
Adopt new benchmark timing internals. This patch adopts a new internal structure for how timings are performed. Currently every iteration of a benchmark checks to see if it has been running for an appropriate amount of time. Checking the clock introduces noise into the timings and this can cause inconsistent output from each benchmark. Now every iteration of a benchmark only checks an iteration count to see if it should stop running. The iteration count is determined before hand by testing the benchmark on a series of increasing iteration counts until a suitable count is found. This increases the amount of time it takes to run the actual benchmarks but it also greatly increases the accuracy of the results. This patch introduces some breaking changes. The notable breaking changes are: 1. Benchmarks run on multiple threads no generate a report per thread. Instead only a single report is generated. 2. ::benchmark::UseRealTime() was removed and replaced with State::UseRealTime().
2015-03-12 22:03:33 +00:00
Change the available benchmark options
2015-03-27 03:37:26 +00:00
compile_benchmark_test(options_test)
redo filter test
2015-03-31 04:05:02 +00:00
add_test(options_benchmarks options_test --benchmark_min_time=0.01)
Change the available benchmark options
2015-03-27 03:37:26 +00:00
Adopt new benchmark timing internals. This patch adopts a new internal structure for how timings are performed. Currently every iteration of a benchmark checks to see if it has been running for an appropriate amount of time. Checking the clock introduces noise into the timings and this can cause inconsistent output from each benchmark. Now every iteration of a benchmark only checks an iteration count to see if it should stop running. The iteration count is determined before hand by testing the benchmark on a series of increasing iteration counts until a suitable count is found. This increases the amount of time it takes to run the actual benchmarks but it also greatly increases the accuracy of the results. This patch introduces some breaking changes. The notable breaking changes are: 1. Benchmarks run on multiple threads no generate a report per thread. Instead only a single report is generated. 2. ::benchmark::UseRealTime() was removed and replaced with State::UseRealTime().
2015-03-12 22:03:33 +00:00
compile_benchmark_test(basic_test)
redo filter test
2015-03-31 04:05:02 +00:00
add_test(basic_benchmark basic_test --benchmark_min_time=0.01)
add C++03 test and update README
2015-03-18 20:34:43 +00:00
Add SkipWithError(...) function.
2016-05-24 01:24:56 +00:00
compile_benchmark_test(diagnostics_test)
add_test(diagnostics_test diagnostics_test --benchmark_min_time=0.01)
compile_benchmark_test(skip_with_error_test)
add_test(skip_with_error_test skip_with_error_test --benchmark_min_time=0.01)
Force DoNotOptimize operand to memory for both gcc and clang
2016-05-25 06:31:20 +00:00
compile_benchmark_test(donotoptimize_test)
add_test(donotoptimize_test donotoptimize_test --benchmark_min_time=0.01)
add ability to use benchmark filters
2015-04-06 21:00:06 +00:00
compile_benchmark_test(fixture_test)
add_test(fixture_test fixture_test --benchmark_min_time=0.01)
Add `RegisterBenchmark(name, func, args...)` for creating/registering benchmarks. (#259) * Add RegisterBenchmark * fix test inputs * fix UB caused by unitialized value * Add RegisterBenchmark * fix test inputs * fix UB caused by unitialized value * Work around GCC 4.6/4.7/4.8 bug
2016-08-02 23:22:46 +00:00
compile_benchmark_test(register_benchmark_test)
add_test(register_benchmark_test register_benchmark_test --benchmark_min_time=0.01)
Pass const state to Fixture::SetUp. Add map_test.
2016-02-15 11:04:19 +00:00
compile_benchmark_test(map_test)
add_test(map_test map_test --benchmark_min_time=0.01)
Support multiple ranges in the benchmark (#257) * Support multiple ranges in the benchmark google-benchmark library allows to provide up to two ranges to the benchmark method (range_x and range_y). However, in many cases it's not sufficient. The patch introduces multi-range features, so user can easily define multiple ranges by passing a vector of integers, and access values through the method range(i). * Remove redundant API Functions State::range_x() and State::range_y() have been removed. They should be replaced by State::range(0) and State::range(1). Functions Benchmark::ArgPair() and Benchmark::RangePair() have been removed. They should be replaced by Benchmark::Args() and Benchmark::Ranges().
2016-08-04 19:30:14 +00:00
compile_benchmark_test(multiple_ranges_test)
add_test(multiple_ranges_test multiple_ranges_test --benchmark_min_time=0.01)
Refactor output test runner into standalone module. (#277) * refactor * Move default substitutions into library * Move default substitutions to the *right* place in the library * Fix init order issues that caused test failures * improve diagnostics * add missing include * general cleanup * Address review comments
2016-08-28 19:24:16 +00:00
compile_output_test(reporter_output_test)
Change reporters to use a specified output and error stream. Add tests for output. (#219) * Add test for reporter output. * setup err_stream tests * Fix warnings in tests * whitespace * Fix build errors caused by super pedantic compilers * Pass streams by pointer not non-const reference
2016-05-27 19:34:37 +00:00
add_test(reporter_output_test reporter_output_test --benchmark_min_time=0.01)
Only compile and run the C++03 test when -std=c++03 is supported. Fixes #164
2016-05-24 02:44:10 +00:00
check_cxx_compiler_flag(-std=c++03 BENCHMARK_HAS_CXX03_FLAG)
if (BENCHMARK_HAS_CXX03_FLAG)
set(CXX03_FLAGS "${CMAKE_CXX_FLAGS}")
string(REPLACE "-std=c++11" "-std=c++03" CXX03_FLAGS "${CXX03_FLAGS}")
string(REPLACE "-std=c++0x" "-std=c++03" CXX03_FLAGS "${CXX03_FLAGS}")
compile_benchmark_test(cxx03_test)
set_target_properties(cxx03_test
PROPERTIES COMPILE_FLAGS "${CXX03_FLAGS}")
add_test(cxx03 cxx03_test --benchmark_min_time=0.01)
endif()
Coverage support with LCOV and GCOV This patch adds a `coverage` target that allows coverage statisitics to be retrieved for the project. It requires that lcov and gcov is installed and that the generator is unix makefiles but this can be improved upon in future releases. To make it work use the coverage build type: ``` cmake -DCMAKE_BUILD_TYPE=Coverage . make coverage ```
2015-04-30 13:28:04 +00:00
Refactor output test runner into standalone module. (#277) * refactor * Move default substitutions into library * Move default substitutions to the *right* place in the library * Fix init order issues that caused test failures * improve diagnostics * add missing include * general cleanup * Address review comments
2016-08-28 19:24:16 +00:00
compile_output_test(complexity_test)
Added test file and Complexity() interface
2016-05-18 17:59:34 +00:00
add_test(complexity_benchmark complexity_test --benchmark_min_time=0.01)
Coverage support with LCOV and GCOV This patch adds a `coverage` target that allows coverage statisitics to be retrieved for the project. It requires that lcov and gcov is installed and that the generator is unix makefiles but this can be improved upon in future releases. To make it work use the coverage build type: ``` cmake -DCMAKE_BUILD_TYPE=Coverage . make coverage ```
2015-04-30 13:28:04 +00:00
# Add the coverage command(s)
Fixing cmake issue with empty cmake_build_type
2015-05-22 21:15:19 +00:00
if(CMAKE_BUILD_TYPE)
string(TOLOWER ${CMAKE_BUILD_TYPE} CMAKE_BUILD_TYPE_LOWER)
endif()
Coverage support with LCOV and GCOV This patch adds a `coverage` target that allows coverage statisitics to be retrieved for the project. It requires that lcov and gcov is installed and that the generator is unix makefiles but this can be improved upon in future releases. To make it work use the coverage build type: ``` cmake -DCMAKE_BUILD_TYPE=Coverage . make coverage ```
2015-04-30 13:28:04 +00:00
if (${CMAKE_BUILD_TYPE_LOWER} MATCHES "coverage")
find_program(GCOV gcov)
find_program(LCOV lcov)
find_program(GENHTML genhtml)
find_program(CTEST ctest)
if (GCOV AND LCOV AND GENHTML AND CTEST AND HAVE_CXX_FLAG_COVERAGE)
add_custom_command(
OUTPUT ${CMAKE_BINARY_DIR}/lcov/index.html
COMMAND ${LCOV} -q -z -d .
COMMAND ${LCOV} -q --no-external -c -b "${CMAKE_SOURCE_DIR}" -d . -o before.lcov -i
COMMAND ${CTEST} --force-new-ctest-process
COMMAND ${LCOV} -q --no-external -c -b "${CMAKE_SOURCE_DIR}" -d . -o after.lcov
COMMAND ${LCOV} -q -a before.lcov -a after.lcov --output-file final.lcov
COMMAND ${LCOV} -q -r final.lcov "'${CMAKE_SOURCE_DIR}/test/*'" -o final.lcov
COMMAND ${GENHTML} final.lcov -o lcov --demangle-cpp --sort -p "${CMAKE_BINARY_DIR}" -t benchmark
Added test file and Complexity() interface
2016-05-18 17:59:34 +00:00
DEPENDS filter_test benchmark_test options_test basic_test fixture_test cxx03_test complexity_test
Coverage support with LCOV and GCOV This patch adds a `coverage` target that allows coverage statisitics to be retrieved for the project. It requires that lcov and gcov is installed and that the generator is unix makefiles but this can be improved upon in future releases. To make it work use the coverage build type: ``` cmake -DCMAKE_BUILD_TYPE=Coverage . make coverage ```
2015-04-30 13:28:04 +00:00
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
COMMENT "Running LCOV"
)
add_custom_target(coverage
DEPENDS ${CMAKE_BINARY_DIR}/lcov/index.html
COMMENT "LCOV report at lcov/index.html"
)
message(STATUS "Coverage command added")
else()
if (HAVE_CXX_FLAG_COVERAGE)
set(CXX_FLAG_COVERAGE_MESSAGE supported)
else()
set(CXX_FLAG_COVERAGE_MESSAGE unavailable)
endif()
message(WARNING
"Coverage not available:\n"
" gcov: ${GCOV}\n"
" lcov: ${LCOV}\n"
" genhtml: ${GENHTML}\n"
" ctest: ${CTEST}\n"
" --coverage flag: ${CXX_FLAG_COVERAGE_MESSAGE}")
endif()
endif()