open-nomad/e2e
Seth Hoenig 3ce4f52740
Merge pull request #12446 from shoenig/no-pkg-err
cleanup: purge github.com/pkg/errors
2022-04-04 09:22:44 -05:00
..
affinities e2e: setup consul ACLs a little more correctly 2020-01-31 19:06:11 -06:00
bin e2e/bin/run: run & update only attempt to contact linux servers (#8517) 2020-07-24 10:52:12 -04:00
cli chore: bump golangci-lint from v1.24 to v1.39 2021-04-03 09:50:23 +02:00
clientstate e2e: have e2e use ci.Parallel 2022-03-17 08:37:34 -05:00
connect E2E: ensure ConnectACLsE2ETest has clean state before starting (#12334) 2022-03-21 11:05:02 -04:00
consul E2E with HCP Consul/Vault (#12267) 2022-03-18 09:27:28 -04:00
consulacls cleanup: purge github.com/pkg/errors 2022-04-01 19:24:02 -05:00
consultemplate e2e: account for new job stop CLI exit behaviour. 2022-02-01 14:16:37 +01:00
csi E2E: ensure that CSI EBS tests are isolated from each other (#12443) 2022-04-04 09:44:55 -04:00
deployment e2e: setup consul ACLs a little more correctly 2020-01-31 19:06:11 -06:00
disconnectedclients E2E disconnected clients test refactor (#12402) 2022-03-30 09:12:44 -04:00
e2eutil E2E disconnected clients test refactor (#12402) 2022-03-30 09:12:44 -04:00
eval_priority core: allow setting and propagation of eval priority on job de/registration (#11532) 2021-11-23 09:23:31 +01:00
events e2e: deflake events 2021-01-21 10:25:42 -05:00
example E2E: move example test to use golangs stdlib test runner (#12383) 2022-03-25 14:44:16 -04:00
execagent Apply some suggested fixes from staticcheck (#9598) 2020-12-10 07:29:18 -08:00
framework e2e: have e2e use ci.Parallel 2022-03-17 08:37:34 -05:00
isolation drivers/exec+java: Add task configuration to restore previous PID/IPC isolation behavior 2021-02-08 14:26:35 -06:00
lifecycle chore: fix incorrect docstring formatting. 2021-08-30 11:08:12 +02:00
metrics E2E with HCP Consul/Vault (#12267) 2022-03-18 09:27:28 -04:00
namespaces e2e: account for new job stop CLI exit behaviour. 2022-02-01 14:16:37 +01:00
networking e2e: use operator api for Networking suite validation (#12180) 2022-03-03 15:17:29 -05:00
nodedrain E2E with HCP Consul/Vault (#12267) 2022-03-18 09:27:28 -04:00
nomad09upgrade ci: missing import for nomad09upgrade 2022-03-17 08:49:15 -05:00
nomadexec e2e: stop suppressing unexpected EOF errors 2021-05-24 13:35:08 -04:00
oversubscription Revert "Return SchedulerConfig instead of SchedulerConfigResponse struct (#10799)" (#11433) 2021-11-02 17:42:52 -04:00
parameterized Add a test for parameterized summary counts 2021-03-25 11:27:09 -04:00
periodic Add a test for parameterized summary counts 2021-03-25 11:27:09 -04:00
podman e2e: ensure tests are constrained to Linux (#8990) 2020-09-30 09:43:30 -04:00
quotas gofmt all the files 2021-10-01 10:14:28 -04:00
remotetasks comment out unused consts to make linter happy 2021-04-30 08:31:31 -07:00
rescheduling e2e: account for new job stop CLI exit behaviour. 2022-02-01 14:16:37 +01:00
scaling e2e: account for new job stop CLI exit behaviour. 2022-02-01 14:16:37 +01:00
scalingpolicies e2e: account for new job stop CLI exit behaviour. 2022-02-01 14:16:37 +01:00
scheduler_sysbatch e2e: Run system jobs on all datacenters (#11060) 2021-08-17 11:01:47 -04:00
scheduler_system e2e: Run system jobs on all datacenters (#11060) 2021-08-17 11:01:47 -04:00
spread e2e: add reporting to flaky spread test (#9115) 2020-10-16 11:01:07 -04:00
taskevents e2e: use testify requires instead of t.Fatal 2021-01-26 09:14:47 -05:00
terraform E2E: test exercising node drain behavior for CSI volumes (#12384) 2022-03-29 11:19:23 -04:00
upgrades script e2e/upgrades: cluster upgrade scripts 2019-09-24 14:35:45 -04:00
vaultcompat gofmt all the files 2021-10-01 10:14:28 -04:00
vaultsecrets e2e: use context for executing external commands (#12185) 2022-03-04 08:55:36 -05:00
volumes e2e: moved missed volume test stop command to util helper. 2022-02-02 08:42:58 +01:00
.gitignore e2e: have TF write-out HCL for CSI volume registration (#7599) 2020-04-02 12:16:43 -04:00
e2e_test.go E2E: move example test to use golangs stdlib test runner (#12383) 2022-03-25 14:44:16 -04:00
README.md E2E: move example test to use golangs stdlib test runner (#12383) 2022-03-25 14:44:16 -04:00

End to End Tests

This package contains integration tests. Unlike tests alongside Nomad code, these tests expect there to already be a functional Nomad cluster accessible (either on localhost or via the NOMAD_ADDR env var).

See framework/doc.go for how to write tests.

The NOMAD_E2E=1 environment variable must be set for these tests to run.

Provisioning Test Infrastructure on AWS

The terraform/ folder has provisioning code to spin up a Nomad cluster on AWS. You'll need both Terraform and AWS credentials to setup AWS instances on which e2e tests will run. See the README for details. The number of servers and clients is configurable, as is the specific build of Nomad to deploy and the configuration file for each client and server.

Provisioning Local Clusters

To run tests against a local cluster, you'll need to make sure the following environment variables are set:

  • NOMAD_ADDR should point to one of the Nomad servers
  • CONSUL_HTTP_ADDR should point to one of the Consul servers
  • NOMAD_E2E=1

TODO: the scripts in ./bin currently work only with Terraform, it would be nice for us to have a way to deploy Nomad to Vagrant or local clusters.

Running

After completing the provisioning step above, you can set the client environment for NOMAD_ADDR and run the tests as shown below:

# from the ./e2e/terraform directory, set your client environment
# if you haven't already
$(terraform output environment)

cd ..
go test -v ./...

If you want to run a specific suite, you can specify the -suite flag as shown below. Only the suite with a matching Framework.TestSuite.Component will be run, and all others will be skipped.

go test -v -suite=Consul .

If you want to run a specific test, you'll need to regex-escape some of the test's name so that the test runner doesn't skip over framework struct method names in the full name of the tests:

go test -v . -run 'TestE2E/Consul/\*consul\.ScriptChecksE2ETest/TestGroup'
                              ^       ^             ^               ^
                              |       |             |               |
                          Component   |             |           Test func
                                      |             |
                                  Go Package      Struct

We're also in the process of migrating to "stdlib-style" tests that use the standard go testing package without a notion of "suite". You can run these with -run regexes the same way you would any other go test:

go test -v . -run TestExample/TestExample_Simple

I Want To...

...SSH Into One Of The Test Machines

You can use the Terraform output to find the IP address. The keys will in the ./terraform/keys/ directory.

ssh -i keys/nomad-e2e-*.pem ubuntu@${EC2_IP_ADDR}

Run terraform output for IP addresses and details.

...Deploy a Cluster of Mixed Nomad Versions

The variables.tf file describes the nomad_version, and nomad_local_binary variables that can be used for most circumstances. But if you want to deploy mixed Nomad versions, you can provide a list of versions in your terraform.tfvars file.

For example, if you want to provision 3 servers all using Nomad 0.12.1, and 2 Linux clients using 0.12.1 and 0.12.2, you can use the following variables:

# will be used for servers
nomad_version = "0.12.1"

# will override the nomad_version for Linux clients
nomad_version_client_linux = [
    "0.12.1",
    "0.12.2"
]

...Deploy Custom Configuration Files

Set the profile field to "custom" and put the configuration files in ./terraform/config/custom/ as described in the README.

...Deploy More Than 4 Linux Clients

Use the "custom" profile as described above.

...Change the Nomad Version After Provisioning

You can update the nomad_version variable, or simply rebuild the binary you have at the nomad_local_binary path so that Terraform picks up the changes. Then run terraform plan/terraform apply again. This will update Nomad in place, making the minimum amount of changes necessary.