2edbdfc8be
The e2e framework instantiates clients for Nomad/Consul but the provisioning of the actual Nomad cluster is left to Terraform. The Terraform provisioning process uses `remote-exec` to deploy specific versions of Nomad so that we don't have to bake an AMI every time we want to test a new version. But Terraform treats the resulting instances as immutable, so we can't use the same tooling to update the version of Nomad in-place. This is a prerequisite for upgrade testing. This changeset extends the e2e framework to provide the option of deploying Nomad (and, in the future, Consul/Vault) with specific versions to running infrastructure. This initial implementation is focused on deploying to a single cluster via `ssh` (because that's our current need), but provides interfaces to hook the test run at the start of the run, the start of each suite, or the start of a given test case. Terraform work includes: * provides Terraform output that written to JSON used by the framework to configure provisioning via `terraform output provisioning`. * provides Terraform output that can be used by test operators to configure their shell via `$(terraform output environment)` * drops `remote-exec` provisioning steps from Terraform * makes changes to the deployment scripts to ensure they can be run multiple times w/ different versions against the same host. |
||
|---|---|---|
| .. | ||
| packer | ||
| shared | ||
| .gitignore | ||
| compute.tf | ||
| iam.tf | ||
| main.tf | ||
| network.tf | ||
| provisioning.tf | ||
| README.md | ||
| terraform.tfvars | ||
| versions.tf | ||
Terraform provisioner for end to end tests
This folder contains terraform resources for provisioning a nomad cluster on AWS for end to end tests. It uses a Nomad binary identified by its commit SHA that's stored in a shared s3 bucket that Nomad team developers can access. The commit SHA can be from any branch that's pushed to remote.
Use envchain to store your AWS credentials.
$ cd e2e/terraform/
$ TF_VAR_nomad_sha=<nomad_sha> envchain nomadaws terraform apply
After this step, you should have a nomad client address to point the end to end tests in the e2e folder to.
SSH
Terraform will output node IPs that may be accessed via ssh:
ssh -i keys/nomad-e2e-*.pem ubuntu@${EC2_IP_ADDR}
The Windows client runs OpenSSH for conveniences, but has a different user and will drop you into a Powershell shell instead of bash:
ssh -i keys/nomad-e2e-*.pem Administrator@${EC2_IP_ADDR}
Teardown
The terraform state file stores all the info, so the nomad_sha doesn't need to be valid during teardown.
$ cd e2e/terraform/
$ envchain nomadaws TF_VAR_nomad_sha=yyyzzz terraform destroy