open-consul/sdk/testutil
Daniel Nephin 80ff174880 testutil: NewLogBuffer - buffer logs until a test fails
Replaces #7559

Running tests in parallel, with background goroutines, results in test output not being associated with the correct test. `go test` does not make any guarantees about output from goroutines being attributed to the correct test case.

Attaching log output from background goroutines also cause data races.  If the goroutine outlives the test, it will race with the test being marked done. Previously this was noticed as a panic when logging, but with the race detector enabled it is shown as a data race.

The previous solution did not address the problem of correct test attribution because test output could still be hidden when it was associated with a test that did not fail. You would have to look at all of the log output to find the relevant lines. It also made debugging test failures more difficult because each log line was very long.

This commit attempts a new approach. Instead of printing all the logs, only print when a test fails. This should work well when there are a small number of failures, but may not work well when there are many test failures at the same time. In those cases the failures are unlikely a result of a specific test, and the log output is likely less useful.

All of the logs are printed from the test goroutine, so they should be associated with the correct test.

Also removes some test helpers that were not used, or only had a single caller. Packages which expose many functions with similar names can be difficult to use correctly.

Related:
https://github.com/golang/go/issues/38458 (may be fixed in go1.15)
https://github.com/golang/go/issues/38382#issuecomment-612940030
2020-07-21 12:50:40 -04:00
..
retry Simplified code in various places (#6176) 2019-07-20 09:37:19 -04:00
README.md testutil: NewLogBuffer - buffer logs until a test fails 2020-07-21 12:50:40 -04:00
assertions.go sdk: extracting testutil.RequireErrorContains from various places it was duplicated (#7753) 2020-05-01 11:56:34 -05:00
io.go Centralized Config CLI (#5731) 2019-04-30 16:27:16 -07:00
server.go testutil: NewLogBuffer - buffer logs until a test fails 2020-07-21 12:50:40 -04:00
server_methods.go Move internal/ to sdk/ (#5568) 2019-03-27 08:54:56 -04:00
server_wrapper.go Move internal/ to sdk/ (#5568) 2019-03-27 08:54:56 -04:00
testlog.go testutil: NewLogBuffer - buffer logs until a test fails 2020-07-21 12:50:40 -04:00

README.md

Consul Testing Utilities

This package provides some generic helpers to facilitate testing in Consul.

TestServer

TestServer is a harness for managing Consul agents and initializing them with test data. Using it, you can form test clusters, create services, add health checks, manipulate the K/V store, etc. This test harness is completely decoupled from Consul's core and API client, meaning it can be easily imported and used in external unit tests for various applications. It works by invoking the Consul CLI, which means it is a requirement to have Consul installed in the $PATH.

Following is an example usage:

package my_program

import (
	"testing"

	"github.com/hashicorp/consul/consul/structs"
	"github.com/hashicorp/consul/sdk/testutil"
)

func TestFoo_bar(t *testing.T) {
	// Create a test Consul server
	srv1, err := testutil.NewTestServerConfigT(t, nil)
	if err != nil {
		t.Fatal(err)
	}
	defer srv1.Stop()

	// Create a secondary server, passing in configuration
	// to avoid bootstrapping as we are forming a cluster.
	srv2, err := testutil.NewTestServerConfigT(t, func(c *testutil.TestServerConfig) {
		c.Bootstrap = false
	})
	if err != nil {
		t.Fatal(err)
	}
	defer srv2.Stop()

	// Join the servers together
	srv1.JoinLAN(t, srv2.LANAddr)

	// Create a test key/value pair
	srv1.SetKV(t, "foo", []byte("bar"))

	// Create lots of test key/value pairs
	srv1.PopulateKV(t, map[string][]byte{
		"bar": []byte("123"),
		"baz": []byte("456"),
	})

	// Create a service
	srv1.AddService(t, "redis", structs.HealthPassing, []string{"master"})

	// Create a service that will be accessed in target source code
	srv1.AddAccessibleService("redis", structs.HealthPassing, "127.0.0.1", 6379, []string{"master"})

	// Create a service check
	srv1.AddCheck(t, "service:redis", "redis", structs.HealthPassing)

	// Create a node check
	srv1.AddCheck(t, "mem", "", structs.HealthCritical)

	// The HTTPAddr field contains the address of the Consul
	// API on the new test server instance.
	println(srv1.HTTPAddr)

	// All functions also have a wrapper method to limit the passing of "t"
	wrap := srv1.Wrap(t)
	wrap.SetKV("foo", []byte("bar"))
}