luxolus
/
open-nomad
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity
open-nomad/e2e/e2e_test.go

53 lines
1.8 KiB
Go
Raw Normal View History

e2e: update framework to allow deploying Nomad (#6969) The e2e framework instantiates clients for Nomad/Consul but the provisioning of the actual Nomad cluster is left to Terraform. The Terraform provisioning process uses `remote-exec` to deploy specific versions of Nomad so that we don't have to bake an AMI every time we want to test a new version. But Terraform treats the resulting instances as immutable, so we can't use the same tooling to update the version of Nomad in-place. This is a prerequisite for upgrade testing. This changeset extends the e2e framework to provide the option of deploying Nomad (and, in the future, Consul/Vault) with specific versions to running infrastructure. This initial implementation is focused on deploying to a single cluster via `ssh` (because that's our current need), but provides interfaces to hook the test run at the start of the run, the start of each suite, or the start of a given test case. Terraform work includes: * provides Terraform output that written to JSON used by the framework to configure provisioning via `terraform output provisioning`. * provides Terraform output that can be used by test operators to configure their shell via `$(terraform output environment)` * drops `remote-exec` provisioning steps from Terraform * makes changes to the deployment scripts to ensure they can be run multiple times w/ different versions against the same host.
2020-01-22 13:48:52 +00:00
// This package exists to wrap our e2e provisioning and test framework so that it
// can be run via 'go test ./e2e'. See './framework/framework.go'
e2e/framework: initial e2e framework implementation
2018-06-28 17:29:26 +00:00
package e2e
import (
e2e: update framework to allow deploying Nomad (#6969) The e2e framework instantiates clients for Nomad/Consul but the provisioning of the actual Nomad cluster is left to Terraform. The Terraform provisioning process uses `remote-exec` to deploy specific versions of Nomad so that we don't have to bake an AMI every time we want to test a new version. But Terraform treats the resulting instances as immutable, so we can't use the same tooling to update the version of Nomad in-place. This is a prerequisite for upgrade testing. This changeset extends the e2e framework to provide the option of deploying Nomad (and, in the future, Consul/Vault) with specific versions to running infrastructure. This initial implementation is focused on deploying to a single cluster via `ssh` (because that's our current need), but provides interfaces to hook the test run at the start of the run, the start of each suite, or the start of a given test case. Terraform work includes: * provides Terraform output that written to JSON used by the framework to configure provisioning via `terraform output provisioning`. * provides Terraform output that can be used by test operators to configure their shell via `$(terraform output environment)` * drops `remote-exec` provisioning steps from Terraform * makes changes to the deployment scripts to ensure they can be run multiple times w/ different versions against the same host.
2020-01-22 13:48:52 +00:00
"os"
e2e/framework: initial e2e framework implementation
2018-06-28 17:29:26 +00:00
"testing"
e2e: update framework to allow deploying Nomad (#6969) The e2e framework instantiates clients for Nomad/Consul but the provisioning of the actual Nomad cluster is left to Terraform. The Terraform provisioning process uses `remote-exec` to deploy specific versions of Nomad so that we don't have to bake an AMI every time we want to test a new version. But Terraform treats the resulting instances as immutable, so we can't use the same tooling to update the version of Nomad in-place. This is a prerequisite for upgrade testing. This changeset extends the e2e framework to provide the option of deploying Nomad (and, in the future, Consul/Vault) with specific versions to running infrastructure. This initial implementation is focused on deploying to a single cluster via `ssh` (because that's our current need), but provides interfaces to hook the test run at the start of the run, the start of each suite, or the start of a given test case. Terraform work includes: * provides Terraform output that written to JSON used by the framework to configure provisioning via `terraform output provisioning`. * provides Terraform output that can be used by test operators to configure their shell via `$(terraform output environment)` * drops `remote-exec` provisioning steps from Terraform * makes changes to the deployment scripts to ensure they can be run multiple times w/ different versions against the same host.
2020-01-22 13:48:52 +00:00
"github.com/hashicorp/nomad/e2e/framework"
Affinities e2e tests
2018-12-19 00:39:45 +00:00
_ "github.com/hashicorp/nomad/e2e/affinities"
e2e: add NomadAgent and basic client state test The e2e test code is absolutely hideous and leaks processes and files on disk. NomadAgent seems useful, but the clientstate e2e tests are very messy and slow. The last test "Corrupt" is probably the most useful as it explicitly corrupts the state file whereas the other tests attempt to reproduce steps thought to cause corruption in earlier releases of Nomad.
2019-01-26 00:51:20 +00:00
_ "github.com/hashicorp/nomad/e2e/clientstate"
e2e: Disable Connect tests The connect tests are very disruptive: restart consul/nomad agents with new tokens. The test seems particularly flaky, failing 32 times out of 73 in my sample. The tests are particularly problematic because they are disruptive and affect other tests. On failure, the nomad or consul agent on the client can get into a wedged state, so health/deployment info in subsequent tests may be wrong. In some cases, the node will be deemed as fail, and then the subsequent tests may fail when the node is deemed lost and the test allocations get migrated unexpectedly.
2021-01-26 14:44:33 +00:00
consul/connect: enable custom sidecars to use expose checks This PR enables jobs configured with a custom sidecar_task to make use of the `service.expose` feature for creating checks on services in the service mesh. Before we would check that sidecar_task had not been set (indicating that something other than envoy may be in use, which would not support envoy's expose feature). However Consul has not added support for anything other than envoy and probably never will, so having the restriction in place seems like an unnecessary hindrance. If Consul ever does support something other than Envoy, they will likely find a way to provide the expose feature anyway. Fixes #9854
2021-02-09 15:44:48 +00:00
_ "github.com/hashicorp/nomad/e2e/connect"
Basic consul registration e2e
2019-01-26 05:06:50 +00:00
_ "github.com/hashicorp/nomad/e2e/consul"
e2e: Add consultemplate test This adds a basic test for consul template, that verifies the behaviour of consul-template with task blocking and restarting of tasks
2019-01-04 15:37:11 +00:00
_ "github.com/hashicorp/nomad/e2e/consultemplate"
csi: e2e tests for EBS and EFS plugins (#7343) This changeset provides two basic e2e tests for CSI plugins targeting common AWS use cases. The EBS test launches the EBS plugin (controller + nodes) and registers an EBS volume as a Nomad CSI volume. We deploy a job that writes to the volume, stop that job, and reuse the volume for another job which should be able to read the data written by the first job. The EFS test launches the EFS plugin (nodes-only) and registers an EFS volume as a Nomad CSI volume. We deploy a job that writes to the volume, stop that job, and reuse the volume for another job which should be able to read the data written by the first job. The writer jobs mount the CSI volume at a location within the alloc dir.
2020-03-16 12:53:04 +00:00
_ "github.com/hashicorp/nomad/e2e/csi"
e2e add deployment to the list of e2e tests, minor fixes
2019-05-21 13:21:44 +00:00
_ "github.com/hashicorp/nomad/e2e/deployment"
adds two base event stream e2e tests (#9126) * adds two base event stream e2e tests test evaluation filter keys are included * Apply suggestions from code review Co-authored-by: Tim Gross <tgross@hashicorp.com> * gc aftereach Co-authored-by: Tim Gross <tgross@hashicorp.com>
2020-10-20 12:26:21 +00:00
_ "github.com/hashicorp/nomad/e2e/events"
e2e/framework: initial e2e framework implementation
2018-06-28 17:29:26 +00:00
_ "github.com/hashicorp/nomad/e2e/example"
Add PID namespacing and e2e test
2021-01-25 21:57:17 +00:00
_ "github.com/hashicorp/nomad/e2e/isolation"
task lifecycle: e2e tests
2020-08-18 17:49:50 +00:00
_ "github.com/hashicorp/nomad/e2e/lifecycle"
e2e: run client/allocs metrics tests nightly (#6842) Refactor the metrics end-to-end tests so they can be run with our e2e test framework. Runs fabio/prometheus and a collection of jobs that will cause metrics to be measured. We then query Prometheus to ensure we're publishing those allocation metrics and some metrics from the clients as well. Includes adding a placeholder for running the same tests on Windows.
2019-12-12 17:45:16 +00:00
_ "github.com/hashicorp/nomad/e2e/metrics"
e2e: ENT placeholder for namespace/quotas tests (#8973)
2020-09-28 15:23:37 +00:00
_ "github.com/hashicorp/nomad/e2e/namespaces"
client: add NetworkStatus to Allocation (#8657)
2020-10-12 17:43:04 +00:00
_ "github.com/hashicorp/nomad/e2e/networking"
e2e: node drain tests (#8906) Exercise the `nomad node drain` features, driving them via the new CLI helpers.
2020-09-21 15:52:11 +00:00
_ "github.com/hashicorp/nomad/e2e/nodedrain"
drivers: add docker upgrade path and e2e test
2019-01-23 19:44:42 +00:00
_ "github.com/hashicorp/nomad/e2e/nomad09upgrade"
add e2e tests for nomad exec
2019-04-28 21:35:42 +00:00
_ "github.com/hashicorp/nomad/e2e/nomadexec"
oversubscription: e2e tests!
2021-03-26 20:17:49 +00:00
_ "github.com/hashicorp/nomad/e2e/oversubscription"
Add a test for parameterized summary counts
2021-03-24 21:42:10 +00:00
_ "github.com/hashicorp/nomad/e2e/parameterized"
prevent double job status update (#9768) * Prevent Job Statuses from being calculated twice https://github.com/hashicorp/nomad/pull/8435 introduced atomic eval insertion iwth job (de-)registration. This change removes a now obsolete guard which checked if the index was equal to the job.CreateIndex, which would empty the status. Now that the job regisration eval insetion is atomic with the registration this check is no longer necessary to set the job statuses correctly. * test to ensure only single job event for job register * periodic e2e * separate job update summary step * fix updatejobstability to use copy instead of modified reference of job * update envoygatewaybindaddresses copy to prevent job diff on null vs empty * set ConsulGatewayBindAddress to empty map instead of nil fix nil assertions for empty map rm unnecessary guard
2021-01-22 14:18:17 +00:00
_ "github.com/hashicorp/nomad/e2e/periodic"
base podman e2e test and provisioning updates (#8104) * initial setup for terrform to install podman task driver podman * Update e2e provisioning to support root podman Excludes setup for rootless podman. updates source ami to ubuntu 18.04 Installs podman and configures podman varlink base podman test ensure client status running revert terraform directory changes * back out random go-discover go mod change * include podman varlink docs * address comments
2020-06-03 18:06:58 +00:00
_ "github.com/hashicorp/nomad/e2e/podman"
e2e: ENT placeholder for namespace/quotas tests (#8973)
2020-09-28 15:23:37 +00:00
_ "github.com/hashicorp/nomad/e2e/quotas"
core: propagate remote task handles Add a new driver capability: RemoteTasks. When a task is run by a driver with RemoteTasks set, its TaskHandle will be propagated to the server in its allocation's TaskState. If the task is replaced due to a down node or draining, its TaskHandle will be propagated to its replacement allocation. This allows tasks to be scheduled in remote systems whose lifecycles are disconnected from the Nomad node's lifecycle. See https://github.com/hashicorp/nomad-driver-ecs for an example ECS remote task driver.
2020-12-17 23:21:46 +00:00
_ "github.com/hashicorp/nomad/e2e/remotetasks"
e2e: rescheduling tests Ports the rescheduling tests (which aren't running in CI) into the current test framework so that they're run on nightly, and exercises the new CLI helpers.
2020-09-09 20:59:07 +00:00
_ "github.com/hashicorp/nomad/e2e/rescheduling"
e2e: add job scaling test suite.
2021-01-11 10:34:19 +00:00
_ "github.com/hashicorp/nomad/e2e/scaling"
_ "github.com/hashicorp/nomad/e2e/scalingpolicies"
core: implement system batch scheduler This PR implements a new "System Batch" scheduler type. Jobs can make use of this new scheduler by setting their type to 'sysbatch'. Like the name implies, sysbatch can be thought of as a hybrid between system and batch jobs - it is for running short lived jobs intended to run on every compatible node in the cluster. As with batch jobs, sysbatch jobs can also be periodic and/or parameterized dispatch jobs. A sysbatch job is considered complete when it has been run on all compatible nodes until reaching a terminal state (success or failed on retries). Feasibility and preemption are governed the same as with system jobs. In this PR, the update stanza is not yet supported. The update stanza is sill limited in functionality for the underlying system scheduler, and is not useful yet for sysbatch jobs. Further work in #4740 will improve support for the update stanza and deployments. Closes #2527
2020-10-09 21:31:38 +00:00
_ "github.com/hashicorp/nomad/e2e/scheduler_sysbatch"
_ "github.com/hashicorp/nomad/e2e/scheduler_system"
Fix spread e2e test
2018-12-17 20:39:14 +00:00
_ "github.com/hashicorp/nomad/e2e/spread"
e2e: add task events tests
2019-01-04 23:19:00 +00:00
_ "github.com/hashicorp/nomad/e2e/taskevents"
E2E: vault secrets (#9081) * rename vault API compatibility test for clarity * exercise vault secrets lease renewal
2020-10-14 12:43:28 +00:00
_ "github.com/hashicorp/nomad/e2e/vaultsecrets"
e2e: test for host volumes and Docker volumes (#8972) Exercises host volume and Docker volume functionality for the `exec` and `docker` task driver, particularly around mounting locations within the container and how this can be used with `template`.
2020-09-28 15:14:13 +00:00
_ "github.com/hashicorp/nomad/e2e/volumes"
e2e/framework: initial e2e framework implementation
2018-06-28 17:29:26 +00:00
)
func TestE2E(t *testing.T) {
e2e: update framework to allow deploying Nomad (#6969) The e2e framework instantiates clients for Nomad/Consul but the provisioning of the actual Nomad cluster is left to Terraform. The Terraform provisioning process uses `remote-exec` to deploy specific versions of Nomad so that we don't have to bake an AMI every time we want to test a new version. But Terraform treats the resulting instances as immutable, so we can't use the same tooling to update the version of Nomad in-place. This is a prerequisite for upgrade testing. This changeset extends the e2e framework to provide the option of deploying Nomad (and, in the future, Consul/Vault) with specific versions to running infrastructure. This initial implementation is focused on deploying to a single cluster via `ssh` (because that's our current need), but provides interfaces to hook the test run at the start of the run, the start of each suite, or the start of a given test case. Terraform work includes: * provides Terraform output that written to JSON used by the framework to configure provisioning via `terraform output provisioning`. * provides Terraform output that can be used by test operators to configure their shell via `$(terraform output environment)` * drops `remote-exec` provisioning steps from Terraform * makes changes to the deployment scripts to ensure they can be run multiple times w/ different versions against the same host.
2020-01-22 13:48:52 +00:00
if os.Getenv("NOMAD_E2E") == "" {
t.Skip("Skipping e2e tests, NOMAD_E2E not set")
} else {
framework.Run(t)
}
e2e/framework: initial e2e framework implementation
2018-06-28 17:29:26 +00:00
}