benchmark/test
Roman Lebedev 1e96bb0ab5
Support windows MSYS2 environments (#1704)
* [CI] Attempt to add windows MSYS2-based coverage

* Mark decl of `State::KeepRunningInternal()` as `inline`

Maybe helps with
```
D:\a\_temp\msys64\ucrt64\bin\g++.exe -DHAVE_STD_REGEX -DHAVE_STEADY_CLOCK -DTEST_BENCHMARK_LIBRARY_HAS_NO_ASSERTIONS -ID:/a/benchmark/benchmark/include -Wall  -Wextra  -Wshadow  -Wfloat-equal  -Wold-style-cast  -Werror  -pedantic  -pedantic-errors  -fstrict-aliasing  -Wno-deprecated-declarations  -Wno-deprecated  -Wstrict-aliasing  -Wno-unused-variable -std=c++11 -fvisibility=hidden -fno-keep-inline-dllexport   -UNDEBUG -MD -MT test/CMakeFiles/benchmark_test.dir/benchmark_test.cc.obj -MF test\CMakeFiles\benchmark_test.dir\benchmark_test.cc.obj.d -o test/CMakeFiles/benchmark_test.dir/benchmark_test.cc.obj -c D:/a/benchmark/benchmark/test/benchmark_test.cc
In file included from D:/a/benchmark/benchmark/test/benchmark_test.cc:1:
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1007:37: error: 'bool benchmark::State::KeepRunningInternal(benchmark::IterationCount, bool)' redeclared without dllimport attribute after being referenced with dll linkage [-Werror]
 1007 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunningInternal(IterationCount n,
      |                                     ^~~~~
```

* Mark more `State`'s member function decls as `inline`

```
[27/110] Building CXX object test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj
FAILED: test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj
D:\a\_temp\msys64\clang32\bin\clang++.exe -DHAVE_STD_REGEX -DHAVE_STEADY_CLOCK -DHAVE_THREAD_SAFETY_ATTRIBUTES -DTEST_BENCHMARK_LIBRARY_HAS_NO_ASSERTIONS -ID:/a/benchmark/benchmark/include -Wall  -Wextra  -Wshadow  -Wfloat-equal  -Wold-style-cast  -Werror  -pedantic  -pedantic-errors  -Wshorten-64-to-32  -fstrict-aliasing  -Wno-deprecated-declarations  -Wno-deprecated  -Wstrict-aliasing  -Wthread-safety  -Wno-unused-variable -std=c++11 -fvisibility=hidden -fvisibility-inlines-hidden   -UNDEBUG -MD -MT test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj -MF test\CMakeFiles\spec_arg_verbosity_test.dir\spec_arg_verbosity_test.cc.obj.d -o test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj -c D:/a/benchmark/benchmark/test/spec_arg_verbosity_test.cc
In file included from D:/a/benchmark/benchmark/test/spec_arg_verbosity_test.cc:5:
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:999:44: error: 'benchmark::State::KeepRunning' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes]
  999 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunning() {
      |                                            ^
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1003:44: error: 'benchmark::State::KeepRunningBatch' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes]
 1003 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunningBatch(IterationCount n) {
      |                                            ^
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1075:60: error: 'benchmark::State::begin' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes]
 1075 | inline BENCHMARK_ALWAYS_INLINE State::StateIterator State::begin() {
      |                                                            ^
D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1078:60: error: 'benchmark::State::end' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes]

 1078 | inline BENCHMARK_ALWAYS_INLINE State::StateIterator State::end() {

      |                                                            ^

```

* StatisticsTest.CV: don't require precise FP match, tolerate some abs error

We get ever so slightly different results on windows with GCC.
```
71: Test command: D:\a\benchmark\benchmark\_build\test\statistics_gtest.exe
71: Working Directory: D:/a/benchmark/benchmark/_build/test
71: Test timeout computed to be: 10000000
71: Running main() from gmock_main.cc
71: [==========] Running 4 tests from 1 test suite.
71: [----------] Global test environment set-up.
71: [----------] 4 tests from StatisticsTest
71: [ RUN      ] StatisticsTest.Mean
71: [       OK ] StatisticsTest.Mean (0 ms)
71: [ RUN      ] StatisticsTest.Median
71: [       OK ] StatisticsTest.Median (0 ms)
71: [ RUN      ] StatisticsTest.StdDev
71: [       OK ] StatisticsTest.StdDev (0 ms)
71: [ RUN      ] StatisticsTest.CV
71: D:/a/benchmark/benchmark/test/statistics_gtest.cc:31: Failure
71: Expected equality of these values:
71:   benchmark::StatisticsCV({2.5, 2.4, 3.3, 4.2, 5.1})
71:     Which is: 0.32888184094918088
71:   0.32888184094918121
71: [  FAILED  ] StatisticsTest.CV (0 ms)
71: [----------] 4 tests from StatisticsTest (0 ms total)
```

* Fix DLL path discovery for tests
2023-11-23 17:47:04 +03:00
..
AssemblyTests.cmake Partially revert "Do not depend on unversioned python binary (#1496)" (#1501) 2022-10-13 10:03:29 +01:00
BUILD Set -Wno-unused-variable for tests (#1682) 2023-10-20 12:51:32 +00:00
CMakeLists.txt Support windows MSYS2 environments (#1704) 2023-11-23 17:47:04 +03:00
args_product_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
basic_test.cc get rid of warnings in tests (#1562) 2023-03-06 14:47:54 +00:00
benchmark_gtest.cc Ensure we don't need benchmark installed to pass c++ feature checks (#1456) 2022-08-04 15:33:35 +01:00
benchmark_min_time_flag_iters_test.cc Fix code triggering -Wsign-conversion (#1596) 2023-05-11 08:40:05 +01:00
benchmark_min_time_flag_time_test.cc Fix code triggering -Wsign-conversion (#1596) 2023-05-11 08:40:05 +01:00
benchmark_name_gtest.cc Introduce warmup phase to BenchmarkRunner (#1130) (#1399) 2022-05-23 13:50:17 +01:00
benchmark_random_interleaving_gtest.cc Expose default display reporter creation in public API (#1344) 2022-02-11 10:23:05 +00:00
benchmark_setup_teardown_test.cc Fix typos found by codespell (#1519) 2023-01-10 12:25:32 +00:00
benchmark_test.cc get rid of some deprecation warnings from tests 2023-02-06 16:38:53 +00:00
clobber_memory_assembly_test.cc Address warnings on NVIDIA nvc++ (#1573) 2023-03-13 12:34:12 +00:00
commandlineflags_gtest.cc Add `benchmark_context` flag that allows per-run custom context. (#1127) 2021-05-04 14:36:11 +01:00
complexity_test.cc Fixed compiler warnings (#1697) 2023-11-10 10:09:50 +00:00
cxx03_test.cc format tests with clang-format (#1282) 2021-11-10 16:22:31 +00:00
diagnostics_test.cc Disable debug-only test in release builds to avoid expected failures. (#1595) 2023-05-10 10:18:43 +01:00
display_aggregates_only_test.cc Introduce Coefficient of variation aggregate (#1220) 2021-09-03 18:44:10 +01:00
donotoptimize_assembly_test.cc Address warnings on NVIDIA nvc++ (#1573) 2023-03-13 12:34:12 +00:00
donotoptimize_test.cc Fix pass rvalue to DoNotOptimize (#1608) 2023-06-19 08:35:52 +01:00
filter_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
fixture_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
internal_threading_test.cc format tests with clang-format (#1282) 2021-11-10 16:22:31 +00:00
link_main_test.cc get rid of warnings in tests (#1562) 2023-03-06 14:47:54 +00:00
map_test.cc get rid of warnings in tests (#1562) 2023-03-06 14:47:54 +00:00
memory_manager_test.cc get rid of warnings in tests (#1562) 2023-03-06 14:47:54 +00:00
min_time_parse_gtest.cc Allow specifying number of iterations via --benchmark_min_time. (#1525) 2023-02-07 11:45:18 +00:00
multiple_ranges_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
options_test.cc Bug fix variable 'actual_iterations' set but not used (#1517) 2022-12-19 14:03:11 +00:00
output_test.h Convert uses of `const char*` to `std::string` (#1567) 2023-03-14 13:10:27 +00:00
output_test_helper.cc Make json and csv output consistent. (#1662) 2023-09-26 13:43:23 +01:00
perf_counters_gtest.cc perf-counters: Make tests pass on Android (#1653) 2023-08-21 15:04:50 +01:00
perf_counters_test.cc State: Initialize counters with kAvgIteration in constructor (#1652) 2023-08-21 15:35:42 +01:00
register_benchmark_test.cc Convert uses of `const char*` to `std::string` (#1567) 2023-03-14 13:10:27 +00:00
repetitions_test.cc Statistics: add support for percentage unit in addition to time (#1219) 2021-09-03 15:36:56 +01:00
report_aggregates_only_test.cc Introduce Coefficient of variation aggregate (#1220) 2021-09-03 18:44:10 +01:00
reporter_output_test.cc Make json and csv output consistent. (#1662) 2023-09-26 13:43:23 +01:00
skip_with_error_test.cc Convert uses of `const char*` to `std::string` (#1567) 2023-03-14 13:10:27 +00:00
spec_arg_test.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
spec_arg_verbosity_test.cc Add option to get the verbosity provided by commandline flag -v (#1330) (#1397) 2022-05-17 17:59:36 +01:00
state_assembly_test.cc Iteration counts should be `uint64_t` globally. (#817) 2019-05-13 12:33:11 +03:00
statistics_gtest.cc Support windows MSYS2 environments (#1704) 2023-11-23 17:47:04 +03:00
string_util_gtest.cc bug: Inconsistent suffixes console reporter 1009 (#1631) 2023-08-01 08:47:09 +01:00
templated_fixture_test.cc format tests with clang-format (#1282) 2021-11-10 16:22:31 +00:00
time_unit_gtest.cc Fix Clang-Tidy warnings related to modernize-use-override (#1523) 2023-01-09 17:52:18 +00:00
user_counters_tabular_test.cc Make json and csv output consistent. (#1662) 2023-09-26 13:43:23 +01:00
user_counters_test.cc get rid of warnings in tests (#1562) 2023-03-06 14:47:54 +00:00
user_counters_thousands_test.cc Add tests for Human Readable functionality (#1632) 2023-07-14 13:56:01 +01:00