luxolus
/
open-consul
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity

ca: require that tests that use Vault are named correctly 

Previously we were using two different criteria to decide where to run a
test.  The main `go-test` job would skip Vault tests based on the
presence of the `vault` binary, but the `test-connect-ca-providers` job
would run tests based on the name.

This led to a scenario where a test may never run in CI.

To fix this problem I added a name check to the function we use to skip
the test. This should ensure that any test that requires vault is named
correctly to be run as part of the `test-connect-ca-providers` job.

At the same time I relaxed the regex we use. I verified this runs the
same tests using `go test --list Vault`.  I made this change because a
bunch of tests in `agent/connect/ca` used `Vault` in the name, without
the underscores. Instead of changing a bunch of test names, this seemed
easier.

With this approach, the worst case is that we run a few extra tests in
the `test-connect-ca-providers` job, which doesn't seem like a problem.
This commit is contained in:
Daniel Nephin 2022-02-28 16:13:53 -05:00
parent 3804677570
commit bb7f2f15b3
2 changed files with 12 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
GNUmakefile
View File

@ -330,15 +330,15 @@ ifeq ("$(CIRCLECI)","true")
# Run in CI
gotestsum --format=short-verbose --junitfile "$(TEST_RESULTS_DIR)/gotestsum-report.xml" -- -cover -coverprofile=coverage.txt ./agent/connect/ca
# Run leader tests that require Vault
gotestsum --format=short-verbose --junitfile "$(TEST_RESULTS_DIR)/gotestsum-report-leader.xml" -- -cover -coverprofile=coverage-leader.txt -run '.*_Vault_' ./agent/consul
gotestsum --format=short-verbose --junitfile "$(TEST_RESULTS_DIR)/gotestsum-report-leader.xml" -- -cover -coverprofile=coverage-leader.txt -run Vault ./agent/consul
# Run agent tests that require Vault
gotestsum --format=short-verbose --junitfile "$(TEST_RESULTS_DIR)/gotestsum-report-agent.xml" -- -cover -coverprofile=coverage-agent.txt -run '.*_Vault_' ./agent
gotestsum --format=short-verbose --junitfile "$(TEST_RESULTS_DIR)/gotestsum-report-agent.xml" -- -cover -coverprofile=coverage-agent.txt -run Vault ./agent
else
# Run locally
@echo "Running /agent/connect/ca tests in verbose mode"
@go test -v ./agent/connect/ca
@go test -v ./agent/consul -run '.*_Vault_'
@go test -v ./agent -run '.*_Vault_'
@go test -v ./agent/consul -run Vault
@go test -v ./agent -run Vault
endif
proto: $(PROTOGOFILES) $(PROTOGOBINFILES)

8
agent/connect/ca/testing.go
View File

@ -6,6 +6,7 @@ import (
"io/ioutil"
"os"
"os/exec"
"strings"
"sync"
"github.com/hashicorp/go-hclog"
@ -86,6 +87,13 @@ func TestConsulProvider(t testing.T, d ConsulProviderStateDelegate) *ConsulProvi
// These tests may be skipped in CI. They are run as part of a separate
// integration test suite.
func SkipIfVaultNotPresent(t testing.T) {
// Try to safeguard against tests that will never run in CI.
// This substring should match the pattern used by the
// test-connect-ca-providers CI job.
if !strings.Contains(t.Name(), "Vault") {
t.Fatalf("test name must contain Vault, otherwise CI will never run it")
}
vaultBinaryName := os.Getenv("VAULT_BINARY_NAME")
if vaultBinaryName == "" {
vaultBinaryName = "vault"