open-nomad/e2e/README.md

# End to End Tests

This package contains integration tests. Unlike tests alongside Nomad code,
these tests expect there to already be a functional Nomad cluster accessible
(either on localhost or via the `NOMAD_ADDR` env var).

See [`framework/doc.go`](framework/doc.go) for how to write tests.

The `NOMAD_E2E=1` environment variable must be set for these tests to run.

## Provisioning Test Infrastructure on AWS

The `terraform/` folder has provisioning code to spin up a Nomad cluster on
AWS. You'll need both Terraform and AWS credentials to setup AWS instances on
which e2e tests will run. See the
[README](https://github.com/hashicorp/nomad/blob/main/e2e/terraform/README.md)
for details. The number of servers and clients is configurable, as is the
specific build of Nomad to deploy and the configuration file for each client
and server.

## Provisioning Local Clusters

To run tests against a local cluster, you'll need to make sure the following
environment variables are set:

* `NOMAD_ADDR` should point to one of the Nomad servers
* `CONSUL_HTTP_ADDR` should point to one of the Consul servers
* `NOMAD_E2E=1`

_TODO: the scripts in `./bin` currently work only with Terraform, it would be
nice for us to have a way to deploy Nomad to Vagrant or local clusters._

## Running

After completing the provisioning step above, you can set the client
environment for `NOMAD_ADDR` and run the tests as shown below:

```sh
# from the ./e2e/terraform directory, set your client environment
# if you haven't already
$(terraform output environment)

cd ..
go test -v ./...
```

If you want to run a specific suite, you can specify the `-suite` flag as
shown below. Only the suite with a matching `Framework.TestSuite.Component`
will be run, and all others will be skipped.

```sh
go test -v -suite=Consul .
```

If you want to run a specific test, you'll need to regex-escape some of the
test's name so that the test runner doesn't skip over framework struct method
names in the full name of the tests:

```sh
go test -v . -run 'TestE2E/Consul/\*consul\.ScriptChecksE2ETest/TestGroup'
                              ^       ^             ^               ^
                              |       |             |               |
                          Component   |             |           Test func
                                      |             |
                                  Go Package      Struct
```

We're also in the process of migrating to "stdlib-style" tests that
use the standard go `testing` package without a notion of "suite". You
can run these with `-run` regexes the same way you would any other go
test:

```sh
go test -v . -run TestExample/TestExample_Simple
```


## I Want To...

### ...SSH Into One Of The Test Machines

You can use the Terraform output to find the IP address. The keys will
in the `./terraform/keys/` directory.

```sh
ssh -i keys/nomad-e2e-*.pem ubuntu@${EC2_IP_ADDR}
```

Run `terraform output` for IP addresses and details.

### ...Deploy a Cluster of Mixed Nomad Versions

The `variables.tf` file describes the `nomad_version`, and
`nomad_local_binary` variables that can be used for most circumstances. But if
you want to deploy mixed Nomad versions, you can provide a list of versions in
your `terraform.tfvars` file.

For example, if you want to provision 3 servers all using Nomad 0.12.1, and 2
Linux clients using 0.12.1 and 0.12.2, you can use the following variables:

```hcl
# will be used for servers
nomad_version = "0.12.1"

# will override the nomad_version for Linux clients
nomad_version_client_linux = [
    "0.12.1",
    "0.12.2"
]
```

### ...Deploy Custom Configuration Files

Set the `profile` field to `"custom"` and put the configuration files in
`./terraform/config/custom/` as described in the
[README](https://github.com/hashicorp/nomad/blob/main/e2e/terraform/README.md#Profiles).

### ...Deploy More Than 4 Linux Clients

Use the `"custom"` profile as described above.

### ...Change the Nomad Version After Provisioning

You can update the `nomad_version` variable, or simply rebuild the binary you
have at the `nomad_local_binary` path so that Terraform picks up the
changes. Then run `terraform plan`/`terraform apply` again. This will update
Nomad in place, making the minimum amount of changes necessary.