benchmark/test/benchmark_min_time_flag_ite...

67 lines
1.7 KiB
C++
Raw Normal View History

Allow specifying number of iterations via --benchmark_min_time. (#1525) * Allow specifying number of iterations via --benchmark_min_time. Make the flag accept two new suffixes: + <integer>x: number of iterations + <floag>s: minimum number of seconds. This matches the internal benchmark API. * forgot to change flag type to string * used tagged union instead of std::variant, which is not available pre C++14 * update decl in benchmark_runner.h too * fixed errors * refactor * backward compat * typo * use IterationCount type * fixed test * const_cast * ret type * remove extra _ * debug * fixed bug from reporting that caused the new configs not to be included in the final report * addressed review comments * restore unnecessary changes in test/BUILD * fix float comparisons warnings from Release builds * clang format * fix visibility warning * remove misc file * removed backup files * addressed review comments * fix shorten in warning * use suffix for existing min_time specs to silent warnings in tests * fix leaks * use default min-time value in flag decl for consistency * removed double kMinTimeDecl from benchmark.h * dont need to preserve errno * add death tests * Add BENCHMARK_EXPORT to hopefully fix missing def errors * only enable death tests in debug mode because bm_check is no-op in release mode * guard death tests with additional support-check macros * Add additional guard to prevent running in Release mode --------- Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-02-07 11:45:18 +00:00
#include <cassert>
#include <cstdlib>
#include <cstring>
#include <iostream>
#include <string>
#include <vector>
#include "benchmark/benchmark.h"
// Tests that we can specify the number of iterations with
// --benchmark_min_time=<NUM>x.
namespace {
class TestReporter : public benchmark::ConsoleReporter {
public:
virtual bool ReportContext(const Context& context) BENCHMARK_OVERRIDE {
return ConsoleReporter::ReportContext(context);
};
virtual void ReportRuns(const std::vector<Run>& report) BENCHMARK_OVERRIDE {
assert(report.size() == 1);
iter_nums_.push_back(report[0].iterations);
ConsoleReporter::ReportRuns(report);
};
TestReporter() {}
virtual ~TestReporter() {}
const std::vector<benchmark::IterationCount>& GetIters() const {
return iter_nums_;
}
Allow specifying number of iterations via --benchmark_min_time. (#1525) * Allow specifying number of iterations via --benchmark_min_time. Make the flag accept two new suffixes: + <integer>x: number of iterations + <floag>s: minimum number of seconds. This matches the internal benchmark API. * forgot to change flag type to string * used tagged union instead of std::variant, which is not available pre C++14 * update decl in benchmark_runner.h too * fixed errors * refactor * backward compat * typo * use IterationCount type * fixed test * const_cast * ret type * remove extra _ * debug * fixed bug from reporting that caused the new configs not to be included in the final report * addressed review comments * restore unnecessary changes in test/BUILD * fix float comparisons warnings from Release builds * clang format * fix visibility warning * remove misc file * removed backup files * addressed review comments * fix shorten in warning * use suffix for existing min_time specs to silent warnings in tests * fix leaks * use default min-time value in flag decl for consistency * removed double kMinTimeDecl from benchmark.h * dont need to preserve errno * add death tests * Add BENCHMARK_EXPORT to hopefully fix missing def errors * only enable death tests in debug mode because bm_check is no-op in release mode * guard death tests with additional support-check macros * Add additional guard to prevent running in Release mode --------- Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-02-07 11:45:18 +00:00
private:
std::vector<benchmark::IterationCount> iter_nums_;
Allow specifying number of iterations via --benchmark_min_time. (#1525) * Allow specifying number of iterations via --benchmark_min_time. Make the flag accept two new suffixes: + <integer>x: number of iterations + <floag>s: minimum number of seconds. This matches the internal benchmark API. * forgot to change flag type to string * used tagged union instead of std::variant, which is not available pre C++14 * update decl in benchmark_runner.h too * fixed errors * refactor * backward compat * typo * use IterationCount type * fixed test * const_cast * ret type * remove extra _ * debug * fixed bug from reporting that caused the new configs not to be included in the final report * addressed review comments * restore unnecessary changes in test/BUILD * fix float comparisons warnings from Release builds * clang format * fix visibility warning * remove misc file * removed backup files * addressed review comments * fix shorten in warning * use suffix for existing min_time specs to silent warnings in tests * fix leaks * use default min-time value in flag decl for consistency * removed double kMinTimeDecl from benchmark.h * dont need to preserve errno * add death tests * Add BENCHMARK_EXPORT to hopefully fix missing def errors * only enable death tests in debug mode because bm_check is no-op in release mode * guard death tests with additional support-check macros * Add additional guard to prevent running in Release mode --------- Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-02-07 11:45:18 +00:00
};
} // end namespace
static void BM_MyBench(benchmark::State& state) {
for (auto s : state) {
}
}
BENCHMARK(BM_MyBench);
int main(int argc, char** argv) {
// Make a fake argv and append the new --benchmark_min_time=<foo> to it.
int fake_argc = argc + 1;
const char** fake_argv = new const char*[static_cast<size_t>(fake_argc)];
Allow specifying number of iterations via --benchmark_min_time. (#1525) * Allow specifying number of iterations via --benchmark_min_time. Make the flag accept two new suffixes: + <integer>x: number of iterations + <floag>s: minimum number of seconds. This matches the internal benchmark API. * forgot to change flag type to string * used tagged union instead of std::variant, which is not available pre C++14 * update decl in benchmark_runner.h too * fixed errors * refactor * backward compat * typo * use IterationCount type * fixed test * const_cast * ret type * remove extra _ * debug * fixed bug from reporting that caused the new configs not to be included in the final report * addressed review comments * restore unnecessary changes in test/BUILD * fix float comparisons warnings from Release builds * clang format * fix visibility warning * remove misc file * removed backup files * addressed review comments * fix shorten in warning * use suffix for existing min_time specs to silent warnings in tests * fix leaks * use default min-time value in flag decl for consistency * removed double kMinTimeDecl from benchmark.h * dont need to preserve errno * add death tests * Add BENCHMARK_EXPORT to hopefully fix missing def errors * only enable death tests in debug mode because bm_check is no-op in release mode * guard death tests with additional support-check macros * Add additional guard to prevent running in Release mode --------- Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-02-07 11:45:18 +00:00
for (int i = 0; i < argc; ++i) fake_argv[i] = argv[i];
fake_argv[argc] = "--benchmark_min_time=4x";
benchmark::Initialize(&fake_argc, const_cast<char**>(fake_argv));
TestReporter test_reporter;
const size_t returned_count =
benchmark::RunSpecifiedBenchmarks(&test_reporter, "BM_MyBench");
assert(returned_count == 1);
// Check the executed iters.
const std::vector<benchmark::IterationCount> iters = test_reporter.GetIters();
Allow specifying number of iterations via --benchmark_min_time. (#1525) * Allow specifying number of iterations via --benchmark_min_time. Make the flag accept two new suffixes: + <integer>x: number of iterations + <floag>s: minimum number of seconds. This matches the internal benchmark API. * forgot to change flag type to string * used tagged union instead of std::variant, which is not available pre C++14 * update decl in benchmark_runner.h too * fixed errors * refactor * backward compat * typo * use IterationCount type * fixed test * const_cast * ret type * remove extra _ * debug * fixed bug from reporting that caused the new configs not to be included in the final report * addressed review comments * restore unnecessary changes in test/BUILD * fix float comparisons warnings from Release builds * clang format * fix visibility warning * remove misc file * removed backup files * addressed review comments * fix shorten in warning * use suffix for existing min_time specs to silent warnings in tests * fix leaks * use default min-time value in flag decl for consistency * removed double kMinTimeDecl from benchmark.h * dont need to preserve errno * add death tests * Add BENCHMARK_EXPORT to hopefully fix missing def errors * only enable death tests in debug mode because bm_check is no-op in release mode * guard death tests with additional support-check macros * Add additional guard to prevent running in Release mode --------- Co-authored-by: dominic <510002+dmah42@users.noreply.github.com>
2023-02-07 11:45:18 +00:00
assert(!iters.empty() && iters[0] == 4);
delete[] fake_argv;
return 0;
}