dolysis
/
benchmark
mirror of https://github.com/google/benchmark.git
3
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
benchmark/.github/workflows/build-and-test.yml

162 lines
4.5 KiB
YAML
Raw Permalink Normal View History

Create build-and-test.yml (#1013) First attempt at a non-travis/non appveyor CI solution
2020-08-18 07:53:21 +00:00
name: build-and-test
on:
bump cmake dep and docs (#1468) * bump cmake dep and docs
2022-09-14 14:11:37 +00:00
push:
branches: [ main ]
pull_request:
branches: [ main ]
Create build-and-test.yml (#1013) First attempt at a non-travis/non appveyor CI solution
2020-08-18 07:53:21 +00:00
jobs:
add g++-6 to ubuntu-14.04 (#1144) * add g++-6 to ubuntu-14.04 * fix syntax * fix yamllint errors for build-and-test * fix 'add-apt-repository' command not found * make 'run tests' explicit * enable testing and run both release and debug * oops
2021-05-07 13:23:05 +00:00
# TODO: add 32-bit builds (g++ and clang++) for ubuntu
# (requires g++-multilib and libc6:i386)
Add multiple compiler support to build-and-test workflow (#1128) * Add 32-bit build support to build-and-test * attempt different yaml multiline string format * syntax fixes to yaml * switch to getting alternative compilers working * remove done TODO * trying to separate out windows * oops, typo. * add TODOs for missing builds wrt travis
2021-04-30 13:22:15 +00:00
# TODO: add coverage build (requires lcov)
# TODO: add clang + libc++ builds for ubuntu
Create build-and-test.yml (#1013) First attempt at a non-travis/non appveyor CI solution
2020-08-18 07:53:21 +00:00
job:
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
name: ${{ matrix.os }}.${{ matrix.build_type }}.${{ matrix.lib }}.${{ matrix.compiler }}
Create build-and-test.yml (#1013) First attempt at a non-travis/non appveyor CI solution
2020-08-18 07:53:21 +00:00
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
remove best effort support for ubuntu 18.04 (#1537) * remove deprecated ubuntu-18.04 target * update docs * force an apt update for perfcounters
2023-02-06 15:37:26 +00:00
os: [ubuntu-22.04, ubuntu-20.04, macos-latest]
Create build-and-test.yml (#1013) First attempt at a non-travis/non appveyor CI solution
2020-08-18 07:53:21 +00:00
build_type: ['Release', 'Debug']
add compiler to build-and-test and create min-cmake CI bot (#1550) * add compiler to build-and-test and create min-cmake CI bot * fix CXX env var * downgrade msvc generator for cmake-3.10 * assume windows users have the latest cmake
2023-02-28 12:40:40 +00:00
compiler: ['g++', 'clang++']
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
lib: ['shared', 'static']
Create build-and-test.yml (#1013) First attempt at a non-travis/non appveyor CI solution
2020-08-18 07:53:21 +00:00
steps:
update actions/checkout to v4
2024-07-17 15:47:54 +00:00
- uses: actions/checkout@v4
Create pylint.yml (#1039) * Create pylint.yml * improve file matching * fix some pylint issues * run on PR and push (force on master only) * more pylint fixes * suppress noisy exit code and filter to fatals * add conan as a dep so the module is importable * fix lint error on unreachable branch
2020-09-09 08:43:26 +00:00
add compiler to build-and-test and create min-cmake CI bot (#1550) * add compiler to build-and-test and create min-cmake CI bot * fix CXX env var * downgrade msvc generator for cmake-3.10 * assume windows users have the latest cmake
2023-02-28 12:40:40 +00:00
- uses: lukka/get-cmake@latest
add g++-6 to ubuntu-14.04 (#1144) * add g++-6 to ubuntu-14.04 * fix syntax * fix yamllint errors for build-and-test * fix 'add-apt-repository' command not found * make 'run tests' explicit * enable testing and run both release and debug * oops
2021-05-07 13:23:05 +00:00
- name: create build environment
run: cmake -E make_directory ${{ runner.workspace }}/_build
Create pylint.yml (#1039) * Create pylint.yml * improve file matching * fix some pylint issues * run on PR and push (force on master only) * more pylint fixes * suppress noisy exit code and filter to fatals * add conan as a dep so the module is importable * fix lint error on unreachable branch
2020-09-09 08:43:26 +00:00
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
- name: setup cmake initial cache
run: touch compiler-cache.cmake
add g++-6 to ubuntu-14.04 (#1144) * add g++-6 to ubuntu-14.04 * fix syntax * fix yamllint errors for build-and-test * fix 'add-apt-repository' command not found * make 'run tests' explicit * enable testing and run both release and debug * oops
2021-05-07 13:23:05 +00:00
- name: configure cmake
env:
CXX: ${{ matrix.compiler }}
shell: bash
working-directory: ${{ runner.workspace }}/_build
run: >
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
cmake -C ${{ github.workspace }}/compiler-cache.cmake
$GITHUB_WORKSPACE
add g++-6 to ubuntu-14.04 (#1144) * add g++-6 to ubuntu-14.04 * fix syntax * fix yamllint errors for build-and-test * fix 'add-apt-repository' command not found * make 'run tests' explicit * enable testing and run both release and debug * oops
2021-05-07 13:23:05 +00:00
-DBENCHMARK_DOWNLOAD_DEPENDENCIES=ON
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
-DBUILD_SHARED_LIBS=${{ matrix.lib == 'shared' }}
add g++-6 to ubuntu-14.04 (#1144) * add g++-6 to ubuntu-14.04 * fix syntax * fix yamllint errors for build-and-test * fix 'add-apt-repository' command not found * make 'run tests' explicit * enable testing and run both release and debug * oops
2021-05-07 13:23:05 +00:00
-DCMAKE_BUILD_TYPE=${{ matrix.build_type }}
add compiler to build-and-test and create min-cmake CI bot (#1550) * add compiler to build-and-test and create min-cmake CI bot * fix CXX env var * downgrade msvc generator for cmake-3.10 * assume windows users have the latest cmake
2023-02-28 12:40:40 +00:00
-DCMAKE_CXX_COMPILER=${{ env.CXX }}
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
-DCMAKE_CXX_VISIBILITY_PRESET=hidden
-DCMAKE_VISIBILITY_INLINES_HIDDEN=ON
Create pylint.yml (#1039) * Create pylint.yml * improve file matching * fix some pylint issues * run on PR and push (force on master only) * more pylint fixes * suppress noisy exit code and filter to fatals * add conan as a dep so the module is importable * fix lint error on unreachable branch
2020-09-09 08:43:26 +00:00
add g++-6 to ubuntu-14.04 (#1144) * add g++-6 to ubuntu-14.04 * fix syntax * fix yamllint errors for build-and-test * fix 'add-apt-repository' command not found * make 'run tests' explicit * enable testing and run both release and debug * oops
2021-05-07 13:23:05 +00:00
- name: build
shell: bash
working-directory: ${{ runner.workspace }}/_build
run: cmake --build . --config ${{ matrix.build_type }}
Create pylint.yml (#1039) * Create pylint.yml * improve file matching * fix some pylint issues * run on PR and push (force on master only) * more pylint fixes * suppress noisy exit code and filter to fatals * add conan as a dep so the module is importable * fix lint error on unreachable branch
2020-09-09 08:43:26 +00:00
add g++-6 to ubuntu-14.04 (#1144) * add g++-6 to ubuntu-14.04 * fix syntax * fix yamllint errors for build-and-test * fix 'add-apt-repository' command not found * make 'run tests' explicit * enable testing and run both release and debug * oops
2021-05-07 13:23:05 +00:00
- name: test
shell: bash
working-directory: ${{ runner.workspace }}/_build
run: ctest -C ${{ matrix.build_type }} -VV
Add ubuntu-14.04 build and test workflow (#1131) * Add ubuntu-14.04 build and test workflow * avoid '.' in job name * no need for fail fast * fix workflow syntax * install some stuff * better compiler installations * update before install * just say yes * trying to match up some paths * Difference between runner and github context in container? * Try some judicious logging * cmake 3.5+ required * specific compiler versions * need git for googletest * Disable testing on old compilers * disable testing properly
2021-05-07 11:39:04 +00:00
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
msvc:
name: ${{ matrix.os }}.${{ matrix.build_type }}.${{ matrix.lib }}.${{ matrix.msvc }}
runs-on: ${{ matrix.os }}
defaults:
run:
shell: powershell
strategy:
fail-fast: false
matrix:
msvc:
- VS-16-2019
- VS-17-2022
arch:
- x64
build_type:
- Debug
- Release
lib:
- shared
- static
include:
- msvc: VS-16-2019
os: windows-2019
generator: 'Visual Studio 16 2019'
- msvc: VS-17-2022
os: windows-2022
generator: 'Visual Studio 17 2022'
steps:
update actions/checkout to v4
2024-07-17 15:47:54 +00:00
- uses: actions/checkout@v4
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
add compiler to build-and-test and create min-cmake CI bot (#1550) * add compiler to build-and-test and create min-cmake CI bot * fix CXX env var * downgrade msvc generator for cmake-3.10 * assume windows users have the latest cmake
2023-02-28 12:40:40 +00:00
- uses: lukka/get-cmake@latest
annotate and export public symbols (#1321)
2022-02-14 10:48:53 +00:00
- name: configure cmake
run: >
cmake -S . -B _build/
-A ${{ matrix.arch }}
-G "${{ matrix.generator }}"
-DBENCHMARK_DOWNLOAD_DEPENDENCIES=ON
-DBUILD_SHARED_LIBS=${{ matrix.lib == 'shared' }}
- name: build
run: cmake --build _build/ --config ${{ matrix.build_type }}
- name: test
run: ctest --test-dir _build/ -C ${{ matrix.build_type }} -VV
Support windows MSYS2 environments (#1704) * [CI] Attempt to add windows MSYS2-based coverage * Mark decl of `State::KeepRunningInternal()` as `inline` Maybe helps with ``` D:\a\_temp\msys64\ucrt64\bin\g++.exe -DHAVE_STD_REGEX -DHAVE_STEADY_CLOCK -DTEST_BENCHMARK_LIBRARY_HAS_NO_ASSERTIONS -ID:/a/benchmark/benchmark/include -Wall -Wextra -Wshadow -Wfloat-equal -Wold-style-cast -Werror -pedantic -pedantic-errors -fstrict-aliasing -Wno-deprecated-declarations -Wno-deprecated -Wstrict-aliasing -Wno-unused-variable -std=c++11 -fvisibility=hidden -fno-keep-inline-dllexport -UNDEBUG -MD -MT test/CMakeFiles/benchmark_test.dir/benchmark_test.cc.obj -MF test\CMakeFiles\benchmark_test.dir\benchmark_test.cc.obj.d -o test/CMakeFiles/benchmark_test.dir/benchmark_test.cc.obj -c D:/a/benchmark/benchmark/test/benchmark_test.cc In file included from D:/a/benchmark/benchmark/test/benchmark_test.cc:1: D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1007:37: error: 'bool benchmark::State::KeepRunningInternal(benchmark::IterationCount, bool)' redeclared without dllimport attribute after being referenced with dll linkage [-Werror] 1007 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunningInternal(IterationCount n, | ^~~~~ ``` * Mark more `State`'s member function decls as `inline` ``` [27/110] Building CXX object test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj FAILED: test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj D:\a\_temp\msys64\clang32\bin\clang++.exe -DHAVE_STD_REGEX -DHAVE_STEADY_CLOCK -DHAVE_THREAD_SAFETY_ATTRIBUTES -DTEST_BENCHMARK_LIBRARY_HAS_NO_ASSERTIONS -ID:/a/benchmark/benchmark/include -Wall -Wextra -Wshadow -Wfloat-equal -Wold-style-cast -Werror -pedantic -pedantic-errors -Wshorten-64-to-32 -fstrict-aliasing -Wno-deprecated-declarations -Wno-deprecated -Wstrict-aliasing -Wthread-safety -Wno-unused-variable -std=c++11 -fvisibility=hidden -fvisibility-inlines-hidden -UNDEBUG -MD -MT test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj -MF test\CMakeFiles\spec_arg_verbosity_test.dir\spec_arg_verbosity_test.cc.obj.d -o test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj -c D:/a/benchmark/benchmark/test/spec_arg_verbosity_test.cc In file included from D:/a/benchmark/benchmark/test/spec_arg_verbosity_test.cc:5: D:/a/benchmark/benchmark/include/benchmark/benchmark.h:999:44: error: 'benchmark::State::KeepRunning' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes] 999 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunning() { | ^ D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1003:44: error: 'benchmark::State::KeepRunningBatch' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes] 1003 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunningBatch(IterationCount n) { | ^ D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1075:60: error: 'benchmark::State::begin' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes] 1075 | inline BENCHMARK_ALWAYS_INLINE State::StateIterator State::begin() { | ^ D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1078:60: error: 'benchmark::State::end' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes] 1078 | inline BENCHMARK_ALWAYS_INLINE State::StateIterator State::end() { | ^ ``` * StatisticsTest.CV: don't require precise FP match, tolerate some abs error We get ever so slightly different results on windows with GCC. ``` 71: Test command: D:\a\benchmark\benchmark\_build\test\statistics_gtest.exe 71: Working Directory: D:/a/benchmark/benchmark/_build/test 71: Test timeout computed to be: 10000000 71: Running main() from gmock_main.cc 71: [==========] Running 4 tests from 1 test suite. 71: [----------] Global test environment set-up. 71: [----------] 4 tests from StatisticsTest 71: [ RUN ] StatisticsTest.Mean 71: [ OK ] StatisticsTest.Mean (0 ms) 71: [ RUN ] StatisticsTest.Median 71: [ OK ] StatisticsTest.Median (0 ms) 71: [ RUN ] StatisticsTest.StdDev 71: [ OK ] StatisticsTest.StdDev (0 ms) 71: [ RUN ] StatisticsTest.CV 71: D:/a/benchmark/benchmark/test/statistics_gtest.cc:31: Failure 71: Expected equality of these values: 71: benchmark::StatisticsCV({2.5, 2.4, 3.3, 4.2, 5.1}) 71: Which is: 0.32888184094918088 71: 0.32888184094918121 71: [ FAILED ] StatisticsTest.CV (0 ms) 71: [----------] 4 tests from StatisticsTest (0 ms total) ``` * Fix DLL path discovery for tests
2023-11-23 14:47:04 +00:00
msys2:
name: ${{ matrix.os }}.${{ matrix.build_type }}.${{ matrix.lib }}.${{ matrix.msys2.msystem }}
runs-on: ${{ matrix.os }}
defaults:
run:
shell: msys2 {0}
strategy:
fail-fast: false
matrix:
os: [ windows-latest ]
msys2:
- { msystem: MINGW64, arch: x86_64, family: GNU, compiler: g++ }
- { msystem: MINGW32, arch: i686, family: GNU, compiler: g++ }
- { msystem: CLANG64, arch: x86_64, family: LLVM, compiler: clang++ }
- { msystem: CLANG32, arch: i686, family: LLVM, compiler: clang++ }
- { msystem: UCRT64, arch: x86_64, family: GNU, compiler: g++ }
build_type:
- Debug
- Release
lib:
- shared
- static
steps:
update actions/checkout to v4
2024-07-17 15:47:54 +00:00
- uses: actions/checkout@v4
use docker container for ubuntu-16.04 builds (#1265) * use docker container for ubuntu-16.04 builds * install some bits * no sudo in docker container * cmake, not cmake3 * include perfcounters * still no sudo in docker containers * yes please, apt
2021-10-27 15:29:37 +00:00
Support windows MSYS2 environments (#1704) * [CI] Attempt to add windows MSYS2-based coverage * Mark decl of `State::KeepRunningInternal()` as `inline` Maybe helps with ``` D:\a\_temp\msys64\ucrt64\bin\g++.exe -DHAVE_STD_REGEX -DHAVE_STEADY_CLOCK -DTEST_BENCHMARK_LIBRARY_HAS_NO_ASSERTIONS -ID:/a/benchmark/benchmark/include -Wall -Wextra -Wshadow -Wfloat-equal -Wold-style-cast -Werror -pedantic -pedantic-errors -fstrict-aliasing -Wno-deprecated-declarations -Wno-deprecated -Wstrict-aliasing -Wno-unused-variable -std=c++11 -fvisibility=hidden -fno-keep-inline-dllexport -UNDEBUG -MD -MT test/CMakeFiles/benchmark_test.dir/benchmark_test.cc.obj -MF test\CMakeFiles\benchmark_test.dir\benchmark_test.cc.obj.d -o test/CMakeFiles/benchmark_test.dir/benchmark_test.cc.obj -c D:/a/benchmark/benchmark/test/benchmark_test.cc In file included from D:/a/benchmark/benchmark/test/benchmark_test.cc:1: D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1007:37: error: 'bool benchmark::State::KeepRunningInternal(benchmark::IterationCount, bool)' redeclared without dllimport attribute after being referenced with dll linkage [-Werror] 1007 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunningInternal(IterationCount n, | ^~~~~ ``` * Mark more `State`'s member function decls as `inline` ``` [27/110] Building CXX object test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj FAILED: test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj D:\a\_temp\msys64\clang32\bin\clang++.exe -DHAVE_STD_REGEX -DHAVE_STEADY_CLOCK -DHAVE_THREAD_SAFETY_ATTRIBUTES -DTEST_BENCHMARK_LIBRARY_HAS_NO_ASSERTIONS -ID:/a/benchmark/benchmark/include -Wall -Wextra -Wshadow -Wfloat-equal -Wold-style-cast -Werror -pedantic -pedantic-errors -Wshorten-64-to-32 -fstrict-aliasing -Wno-deprecated-declarations -Wno-deprecated -Wstrict-aliasing -Wthread-safety -Wno-unused-variable -std=c++11 -fvisibility=hidden -fvisibility-inlines-hidden -UNDEBUG -MD -MT test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj -MF test\CMakeFiles\spec_arg_verbosity_test.dir\spec_arg_verbosity_test.cc.obj.d -o test/CMakeFiles/spec_arg_verbosity_test.dir/spec_arg_verbosity_test.cc.obj -c D:/a/benchmark/benchmark/test/spec_arg_verbosity_test.cc In file included from D:/a/benchmark/benchmark/test/spec_arg_verbosity_test.cc:5: D:/a/benchmark/benchmark/include/benchmark/benchmark.h:999:44: error: 'benchmark::State::KeepRunning' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes] 999 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunning() { | ^ D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1003:44: error: 'benchmark::State::KeepRunningBatch' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes] 1003 | inline BENCHMARK_ALWAYS_INLINE bool State::KeepRunningBatch(IterationCount n) { | ^ D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1075:60: error: 'benchmark::State::begin' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes] 1075 | inline BENCHMARK_ALWAYS_INLINE State::StateIterator State::begin() { | ^ D:/a/benchmark/benchmark/include/benchmark/benchmark.h:1078:60: error: 'benchmark::State::end' redeclared inline; 'dllimport' attribute ignored [-Werror,-Wignored-attributes] 1078 | inline BENCHMARK_ALWAYS_INLINE State::StateIterator State::end() { | ^ ``` * StatisticsTest.CV: don't require precise FP match, tolerate some abs error We get ever so slightly different results on windows with GCC. ``` 71: Test command: D:\a\benchmark\benchmark\_build\test\statistics_gtest.exe 71: Working Directory: D:/a/benchmark/benchmark/_build/test 71: Test timeout computed to be: 10000000 71: Running main() from gmock_main.cc 71: [==========] Running 4 tests from 1 test suite. 71: [----------] Global test environment set-up. 71: [----------] 4 tests from StatisticsTest 71: [ RUN ] StatisticsTest.Mean 71: [ OK ] StatisticsTest.Mean (0 ms) 71: [ RUN ] StatisticsTest.Median 71: [ OK ] StatisticsTest.Median (0 ms) 71: [ RUN ] StatisticsTest.StdDev 71: [ OK ] StatisticsTest.StdDev (0 ms) 71: [ RUN ] StatisticsTest.CV 71: D:/a/benchmark/benchmark/test/statistics_gtest.cc:31: Failure 71: Expected equality of these values: 71: benchmark::StatisticsCV({2.5, 2.4, 3.3, 4.2, 5.1}) 71: Which is: 0.32888184094918088 71: 0.32888184094918121 71: [ FAILED ] StatisticsTest.CV (0 ms) 71: [----------] 4 tests from StatisticsTest (0 ms total) ``` * Fix DLL path discovery for tests
2023-11-23 14:47:04 +00:00
- name: Install Base Dependencies
uses: msys2/setup-msys2@v2
with:
cache: false
msystem: ${{ matrix.msys2.msystem }}
update: true
install: >-
git
base-devel
pacboy: >-
cc:p
cmake:p
ninja:p
- name: configure cmake
env:
CXX: ${{ matrix.msys2.compiler }}
run: >
cmake -S . -B _build/
-GNinja
-DBENCHMARK_DOWNLOAD_DEPENDENCIES=ON
-DBUILD_SHARED_LIBS=${{ matrix.lib == 'shared' }}
- name: build
run: cmake --build _build/ --config ${{ matrix.build_type }}
- name: test
run: ctest --test-dir _build/ -C ${{ matrix.build_type }} -VV