129 lines
4.3 KiB
Go
129 lines
4.3 KiB
Go
// Copyright (c) HashiCorp, Inc.
|
|
// SPDX-License-Identifier: MPL-2.0
|
|
|
|
/*
|
|
Package framework implements a model for developing end-to-end test suites. The
|
|
model includes a top level Framework which TestSuites can be added to. TestSuites
|
|
include conditions under which the suite will run and a list of TestCase
|
|
implementations to run. TestCases can be implemented with methods that run
|
|
before/after each and all tests.
|
|
|
|
# Writing Tests
|
|
|
|
Tests follow a similar patterns as go tests. They are functions that must start
|
|
with 'Test' and instead of a *testing.T argument, a *framework.F is passed and
|
|
they must have a receiver that implements the TestCase interface.
|
|
A crude example as follows:
|
|
|
|
// foo_test.go
|
|
type MyTestCase struct {
|
|
framework.TC
|
|
}
|
|
|
|
func (tc *MyTestCase) TestMyFoo(f *framework.F) {
|
|
f.T().Log("bar")
|
|
}
|
|
|
|
func TestCalledFromGoTest(t *testing.T){
|
|
framework.New().AddSuites(&framework.TestSuite{
|
|
Component: "foo",
|
|
Cases: []framework.TestCase{
|
|
new(MyTestCase),
|
|
},
|
|
}).Run(t)
|
|
}
|
|
|
|
Test cases should embed the TC struct which satisfies the TestCase interface.
|
|
Optionally a TestCase can also implement the Name() function which returns
|
|
a string to name the test case. By default the name is the name of the struct
|
|
type, which in the above example would be "MyTestCase"
|
|
|
|
Test cases may also optionally implement additional interfaces to define setup
|
|
and teardown logic:
|
|
|
|
BeforeEachTest
|
|
AfterEachTest
|
|
BeforeAllTests
|
|
AfterAllTests
|
|
|
|
The test case struct allows you to setup and teardown state in the struct that
|
|
can be consumed by the tests. For example:
|
|
|
|
type ComplexNomadTC struct {
|
|
framework.TC
|
|
jobID string
|
|
}
|
|
|
|
func (tc *ComplexNomadTC) BeforeEach(f *framework.F){
|
|
// Do some complex job setup with a unique prefix string
|
|
jobID, err := doSomeComplexSetup(tc.Nomad(), f.ID())
|
|
f.NoError(err)
|
|
f.Set("jobID", jobID)
|
|
}
|
|
|
|
func (tc *ComplexNomadTC) TestSomeScenario(f *framework.F){
|
|
jobID := f.Value("jobID").(string)
|
|
doTestThingWithJob(f, tc.Nomad(), jobID)
|
|
}
|
|
|
|
func (tc *ComplexNomadTC) TestOtherScenario(f *framework.F){
|
|
jobID := f.Value("jobID").(string)
|
|
doOtherTestThingWithJob(f, tc.Nomad(), jobID)
|
|
}
|
|
|
|
func (tc *ComplexNomadTC) AfterEach(f *framework.F){
|
|
jobID := f.Value("jobID").(string)
|
|
_, _, err := tc.Nomad().Jobs().Deregister(jobID, true, nil)
|
|
f.NoError(err)
|
|
}
|
|
|
|
As demonstrated in the previous example, TC also exposes functions that return
|
|
configured api clients including Nomad, Consul and Vault. If Consul or Vault
|
|
are not provisioned their respective getter functions will return nil.
|
|
|
|
# Testify Integration
|
|
|
|
Test cases expose a T() function to fetch the current *testing.T context.
|
|
While this means the author is able to most other testing libraries,
|
|
github.com/stretch/testify is recommended and integrated into the framework.
|
|
The TC struct also embeds testify assertions that are preconfigured with the
|
|
current testing context. Additionally TC comes with a Require() method that
|
|
yields a testify Require if that flavor is desired.
|
|
|
|
func (tc *MyTestCase) TestWithTestify() {
|
|
err := someErrFunc()
|
|
tc.NoError(err)
|
|
// Or tc.Require().NoError(err)
|
|
}
|
|
|
|
# Parallelism
|
|
|
|
The test framework honors go test's parallel feature under certain conditions.
|
|
A TestSuite can be created with the Parallel field set to true to enable
|
|
parallel execution of the test cases of the suite. Tests within a test case
|
|
will be executed sequentially unless f.T().Parallel() is called. Note that if
|
|
multiple tests are to be executed in parallel, access to TC is not syncronized.
|
|
The *framework.F offers a way to store state between before/after each method if
|
|
desired.
|
|
|
|
func (tc *MyTestCase) BeforeEach(f *framework.F){
|
|
jobID, _ := doSomeComplexSetup(tc.Nomad(), f.ID())
|
|
f.Set("jobID", jobID)
|
|
}
|
|
|
|
func (tc *MyTestCase) TestParallel(f *framework.F){
|
|
f.T().Parallel()
|
|
jobID := f.Value("jobID").(string)
|
|
}
|
|
|
|
Since test cases have the potential to work with a shared Nomad cluster in parallel
|
|
any resources created or destroyed must be prefixed with a unique identifier for
|
|
each test case. The framework.F struct exposes an ID() function that will return a
|
|
string that is unique with in a test. Therefore, multiple tests with in the case
|
|
can reliably create unique IDs between tests and setup/teardown. The string
|
|
returned is 8 alpha numeric characters.
|
|
*/
|
|
|
|
// Deprecated: no longer use e2e/framework for new tests; see TestExample for new e2e test structure.
|
|
package framework
|