luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
19238 commits 1 branch 0 tags 311 MiB
Commit graph

276 commits

Author SHA1 Message Date
Drew Bailey 786989dbe3
New monitor pkg for shared monitor functionality 
Adds new package that can be used by client and server RPC endpoints to
facilitate monitoring based off of a logger

clean up old code

small comment about write

rm old comment about minsize

rename to Monitor

Removes connection logic from monitor command

Keep connection logic in endpoints, use a channel to send results from
monitoring

use new multisink logger and interfaces

small test for dropped messages

update go-hclogger and update sink/intercept logger interfaces
 2019-11-05 09:51:49 -05:00
Tim Gross 3e9ae481ce
e2e: refactor Consul configurations (#6559) 
Ensure that we're reusing the base configuration between client and
servers without the possibility of drift. Reduce the amount of `sed`
mangling of the configuration file, and make recommended changes from
`shellcheck` for this section of the provisioning script.

Fixes some rebase errors on the Nomad config as well.
 2019-10-28 09:27:40 -04:00
Tim Gross ba7e7413ef
e2e: refactor Nomad configuration (#6560) 
Share base configuration for telemetry and consul. Have the server
configurations respect the `var.server_count` config. Make changes
recommended by `shellcheck` in the provisioning scripts for this section.

Switch to OS/arch-tagged release bundles on S3 for compatibility with
adding Windows builds in the near future.
 2019-10-28 08:21:02 -04:00
Tim Gross 8be403f47b
e2e: refactor Vault configuration (#6561) 
Match the configuration directory layout we're using for Consul and
other services. Make recommended changes from `shellcheck` for this
section of the provisioning script.
 2019-10-25 15:29:01 -04:00
Tim Gross 87b3abddd3
e2e: use sockaddr for IP address configuration (#6548) 
Update the Consul and Vault configs to take advantage of their
included `go-sockaddr` library for getting the IP addresses we need in
a portable way. This particularly avoids problems with "predictable"
interface names provided by systemd.

Also adds the `sockaddr` binary to the Packer build so we can use it
in our provisioning scripts.
 2019-10-25 14:08:38 -04:00
Tim Gross efbd680d4e
e2e: split Packer build scripts from TF provisioning (#6542) 
Make a clear split between Packer and Terraform provisioning steps:
the scripts in the `packer/linux` directory are run when we build the
AMI whereas the stuff in shared are run at Terraform provisioning time.

Merging all runtime provisioning scripts into a single script for each
of server/client solves the following:

* Userdata scripts can't take arguments, they can only be templated
  and that means we have to do TF escaping in bash/powershell scripts.
* TF provisioning scripts race with userdata scripts.
 2019-10-25 08:08:24 -04:00
Tim Gross c648c4f998
e2e: upgrade terraform to 0.12.x (#6489) 2019-10-14 11:27:08 -04:00
Tim Gross 15e912ddd6
e2e: move remote-exec inline to script (#6488) 
A failing script in a `remote-exec` provisioner's `inline` stanza
won't fail the provisioning step. This lets us continue on to execute
tests against potentially broken deployments, rather than letting us
know the provisioning itself failed.
 2019-10-14 10:23:41 -04:00
Danielle Lancashire 199d24d6bf
chore: initial hclfmt 2019-10-11 14:00:05 +02:00
Lang Martin 0648402150
Merge pull request #6373 from hashicorp/b-raft-proto-upgrade 
raft protocol defaults to version 2
 2019-09-26 14:33:09 -04:00
Tim Gross d965a15490 driver/networking: don't recreate existing network namespaces 2019-09-25 14:58:17 -04:00
Tim Gross e86a476bbb failing test for #6310 2019-09-25 14:58:17 -04:00
Lang Martin 6e0ec6302b script e2e/upgrades: cluster upgrade scripts 2019-09-24 14:35:45 -04:00
Danielle 940bbcc639
Merge pull request #6342 from hashicorp/f-host-volume-e2e 
Add Host Volumes E2E test
 2019-09-18 12:59:32 -07:00
Tim Gross adde9acf57
e2e: test infra for client node restarts (#6313) 
Add a test helper that restarts a specific client node running under
systemd using a `raw_exec` job.
 2019-09-18 10:10:14 -04:00
Tim Gross 7061dcef4b
e2e: move consul status check helpers to e2eutil (#6314) 2019-09-18 08:18:19 -04:00
Danielle Lancashire 05d172ef2b
e2e: init host volumes test 2019-09-18 00:34:48 +02:00
Danielle Lancashire c50d7f2727
e2e: Add Host Volume Configuration 2019-09-17 20:06:50 +02:00
Tim Gross 55ee7a220b
e2e: fixes for race conditions in testing (#6300) 
- In script checks, ensure we're running `Exec` against the new running
  allocation and not the earlier stopped one.
- In script checks, allow `Exec` calls to error due to lack of pty when
  we use the exec to kill the task.
- In `utils.go/RegisterAllocs`, force query for allocations to wait on
  wait index returned by registration call.
 2019-09-10 13:45:16 -04:00
Tim Gross 3469c50275
e2e: tag instances with origin (#6293) 
When multiple developers are working on e2e testing, it helps to be
able to identify which infrastructure belongs to which Nomad SHA and
which developer. This adds tags to the EC2 instances.
 2019-09-06 15:49:18 -04:00
Tim Gross ede48ae19c script checks: use cat instead of ls for exit code agreement 2019-09-06 11:17:00 -04:00
Tim Gross c9c612cc70 e2e: script check testing 2019-09-06 10:18:55 -04:00
Michael Schurter 228899c32f e2e: test demo job for connect 2019-09-04 12:40:08 -07:00
Tim Gross 7ee3333a2d e2e: filter default AMI by OS 
Add an OS tag to Packer builds of our e2e test AMIs and then filters
by this in Terraform.
 2019-08-30 16:51:13 -04:00
Danielle Lancashire d454dab39b
chore: Format hcl configurations 2019-07-20 16:55:07 +02:00
Michael Schurter a3fcb8fcca e2e: debug log level for everyone! 2019-07-18 06:55:27 -07:00
Michael Schurter ea68c930fe e2e: enable_debug=true for all agents 
Enables the pprof http endpoint for debugging.
 2019-07-17 15:20:45 -07:00
Preetha 0a2e21353f
Merge pull request #5912 from hashicorp/f-systemd-nofile 
systemd: set a high but non-infinite fd limit
 2019-07-11 12:31:12 -05:00
Preetha Appan 53397722f1
add module version constraint to e2e/terraform 2019-07-05 09:18:38 -05:00
Michael Schurter 803aa62b7a systemd: set a high but non-infinite fd limit 2019-07-02 09:13:24 -07:00
Lang Martin d15d09bcc1 e2e update shell scripts argument quoting 2019-06-04 15:52:32 -04:00
Lang Martin 071dccfcce e2e/deployment DeploymentsForJob fail instead of nil, error passing 2019-06-04 14:31:42 -04:00
Lang Martin fa09e5d5f4 e2e/deployment fail if the second deployment times out 2019-06-04 14:08:30 -04:00
Lang Martin e61597a098 e2e bin/update and bin/run, README 2019-06-04 13:42:07 -04:00
Lang Martin 1635fa3c00 e2e/deployment find the second deployment, use its status 2019-06-04 13:41:52 -04:00
Lang Martin e027b9001b Update e2e/deployment/deployment.go 
Co-Authored-By: Mahmood Ali <mahmood@notnoop.com>
 2019-05-22 12:34:57 -04:00
Lang Martin 7929ef28c7 e2e/deployment comment the job files for clarity 2019-05-22 12:34:57 -04:00
Lang Martin fe69f89476 e2e add deployment to the list of e2e tests, minor fixes 2019-05-22 12:34:57 -04:00
Lang Martin 2a11d66258 e2e readme minor changes to command + env val templates and order 2019-05-22 12:34:57 -04:00
Lang Martin 97fd114535 e2e utils remove ineffectual assignment of allocs 2019-05-22 12:34:57 -04:00
Lang Martin 01276455bd e2e README typo 2019-05-22 12:34:57 -04:00
Lang Martin 824d1366dd e2e utils error format arg match 2019-05-22 12:32:08 -04:00
Lang Martin 09a6dc2054 new e2e deployment test 2019-05-22 12:32:08 -04:00
Lang Martin d73606e54e e2e util split new alloc and await placement, new WaitForDeployment 2019-05-22 12:32:08 -04:00
Preetha 2dcd4291f8
Merge pull request #5702 from hashicorp/f-filter-by-create-index 
Filter deployments by create index
 2019-05-15 21:50:41 -05:00
Michael Schurter 2b7f398726 e2e: fix nomad service for systemd<230 2019-05-14 10:53:26 -07:00
Preetha Appan 07690d6f9e
Add flag similar to --all for allocs to be able to filter deployments by latest 2019-05-13 18:33:41 -05:00
Mahmood Ali 919827f2df
Merge pull request #5632 from hashicorp/f-nomad-exec-parts-01-base 
nomad exec part 1: plumbing and docker driver
 2019-05-09 18:09:27 -04:00
Mahmood Ali 2a555a7e74 add e2e tests for nomad exec 2019-05-09 16:49:08 -04:00
Michael Schurter a1c3ce36bc
Merge pull request #5647 from hashicorp/e2e-tf 
E2E Test Terraform/Packer Improvements
 2019-05-06 15:42:52 -07:00
Mahmood Ali bfc907827c docs: update s3 urls to use virtual bucket style 
In response to https://forums.aws.amazon.com/ann.jspa?annID=6776
 2019-05-06 10:39:51 -04:00
Michael Schurter 93f3ac7a9c e2e: explain these scripts are for packer 
It took me way too long to figure out these weren't used by TF.
 2019-05-03 07:55:28 -07:00
Michael Schurter 7558747694 e2e: let the unindex clients do anything... 
...and be debugable!
 2019-05-03 07:54:55 -07:00
Michael Schurter f08dd66ffa e2e: ssh instructions + remove redundant naming 2019-05-03 07:54:34 -07:00
Michael Schurter 19889d6468 e2e: update deps and install nomad in packer 
Nomad on the packer image will be overwritten by the sha specified in
the TF var, but including a base version on the packer image makes the
image valid for independent use.
 2019-05-03 07:53:08 -07:00
Michael Schurter 13b62a68f7 e2e: enable systemd units so they start on boot 2019-05-03 07:52:03 -07:00
Chris Baker c0a7aee610
vault e2e: pass vault version into setup instead of having to infer it from test name 2019-04-10 10:34:10 -05:00
Chris Baker b290d774bc
e2e/vault: updated e2e vault tests to use version-specific policy creation endpoint (old servers are not compatible with new client) 2019-04-10 10:34:10 -05:00
Preetha Appan f0e2859c59
scripts for upgrade testing 2019-04-04 22:31:57 -05:00
Preetha Appan 17d4e80c16
small tweaks to load test jobs to make them work in Nomad 0.8.7 2019-04-02 20:38:56 -05:00
Preetha Appan 19b4bb7ec3
added cpu/disk/memory stress jobs for e2e tests 2019-04-01 22:28:18 -05:00
Preetha Appan 007d771174
Added nginx to e2e test 2019-04-01 14:52:58 -05:00
Preetha Appan a262be08e7
Remove unnecessary step in getting node client 
All allocation stats are routable from the server
 2019-04-01 10:45:41 -05:00
Preetha Appan f9019ae605
Add e2e test with raw exec job for verifying allocation resource stats 2019-03-31 09:46:23 -05:00
Preetha Appan dc370d2e6f
Use specific url prefix for metrics test 
Also changed the output to show client node IP addresses
 2019-03-27 11:04:06 -05:00
Michael Schurter 1fab175b26 test: properly skip client state in beforeall 2019-03-22 06:42:04 -07:00
Preetha fac6d8c918
Merge pull request #5405 from hashicorp/e2e_metrics 
Prometheus metrics for the e2e environment
 2019-03-21 09:30:12 -05:00
Preetha Appan bf8483c960
remove stray println 2019-03-21 09:23:37 -05:00
Michael Schurter 555d6d35ce test: skip slow state test without flag 2019-03-21 07:17:02 -07:00
Michael Schurter cd87afd15f e2e: add NomadAgent and basic client state test 
The e2e test code is absolutely hideous and leaks processes and files
on disk. NomadAgent seems useful, but the clientstate e2e tests are very
messy and slow. The last test "Corrupt" is probably the most useful as
it explicitly corrupts the state file whereas the other tests attempt to
reproduce steps thought to cause corruption in earlier releases of
Nomad.
 2019-03-21 07:14:34 -07:00
Michael Schurter 30db07cccb docs: sync systemd unit files; update deploy guide 
The systemd configs spread across our repo were fairly out of sync. This
should get them on our best practices.

The deployment guide also had some strange things like running Nomad as
a non-root user. It would be fine for servers but completely breaks
clients. For simplicity I simply removed the non-root user references.
 2019-03-19 15:18:12 -07:00
Preetha Appan 59e5ee18b0
Removed use of e2e framework 2019-03-11 09:21:04 -05:00
Preetha Appan 428f80afcc
prometheus and fabio for metrics 2019-03-11 09:21:04 -05:00
Mahmood Ali bd11a4c985 tests: disable upgrade e2e tests 
Upgrade e2e tests are failing and we haven't had bandwith to fix yet.
Having them fail makes it easy for us to miss other failures and
regressions.

As such, skip the upgrade e2e tests until we fix them.
 2019-02-27 08:40:09 -05:00
Alex Dadgar b938a71397 make nomad upgrade e2e build on non linux 2019-01-28 11:18:59 -08:00
Preetha 48b4ed91a3
Merge pull request #5249 from hashicorp/consul_e2e 
Consul e2e tests
 2019-01-28 10:46:49 -06:00
Preetha Appan 25a29cb52f
Moved in place upgrade canary test over to new e2e framework 2019-01-27 20:15:35 -06:00
Preetha Appan b4a722e08f
Basic consul registration e2e 2019-01-26 10:58:25 -06:00
Alex Dadgar d6412fd8e7 Fix double restart counting for templates 
This PR fixes an issue where template restarts would count twice since
it was emitting a restarting event.
 2019-01-25 15:38:13 -08:00
Preetha Appan 419a2682b0
update to Consul 1.4.0 for e2e tests 2019-01-24 09:52:15 -06:00
Nick Ethier e104528288
e2e/nomad09upgrade: remove test file that is not needed 2019-01-23 21:23:13 -05:00
Nick Ethier 22da77f3fa
e2e: removed unused code in nomade09upgrade suite 2019-01-23 20:54:23 -05:00
Nick Ethier ccdc1c4615
Merge branch 'f-driver-upgradepath-test' of https://github.com/hashicorp/nomad into f-driver-upgradepath-test 
* 'f-driver-upgradepath-test' of https://github.com/hashicorp/nomad:
  Apply suggestions from code review
 2019-01-23 20:26:32 -05:00
Nick Ethier 5b9013528e
drivers: add docker upgrade path and e2e test 2019-01-23 14:44:42 -05:00
Michael Schurter ce4a828fd1
Apply suggestions from code review 
Co-Authored-By: nickethier <ncethier@gmail.com>
 2019-01-23 14:09:49 -05:00
Nick Ethier a3510413ff
e2e: remove unused script 2019-01-22 23:29:15 -05:00
Nick Ethier fc370c3d5e
e2e/nomad09upgrade: add comments 2019-01-22 23:25:27 -05:00
Nick Ethier c30965eaf9
e2e: add tests for nomad driver upgrade path 2019-01-17 23:32:45 -05:00
Michael Schurter 641ba37c61 e2e: test task events for a failed sibling task 2019-01-08 14:39:37 -08:00
Michael Schurter b09c68ceaf e2e: wait for at least N nodes to be ready 
Before it was *exactly* N nodes which limited test portability between
clusters.
 2019-01-08 14:39:37 -08:00
Michael Schurter 2cf49f9121 e2e: add task events tests 2019-01-08 07:20:53 -08:00
Danielle Tomlinson d195680ec1 e2e: Add consultemplate test 
This adds a basic test for consul template, that verifies the behaviour
of consul-template with task blocking and restarting of tasks
 2019-01-07 17:53:55 +01:00
Danielle Tomlinson c13dc7f110
Merge pull request #5149 from hashicorp/dani/e2e-friendly 
e2e: Output setup instructions after terraform
 2019-01-04 22:14:03 +01:00
Danielle Tomlinson 33547c99e7 e2e: Output setup instructions after terraform 
This adds a message that provides environment setup instructions for
running e2e tests after running terraform apply.

This allows copy/pasting exports, rather than manually constructing
them.
 2019-01-04 16:55:14 +01:00
Mahmood Ali 606ab23235 goimport file 2019-01-04 08:53:50 -05:00
Preetha Appan 378dd74d2a
Added waiting on client node ready state before running e2e tests 2019-01-03 16:16:20 -06:00
Preetha 1e69a6645f
Update README.md 2019-01-03 16:15:59 -06:00
Preetha 5501ff42c9
Update README.md 2019-01-03 15:31:19 -06:00
Preetha Appan f458cb63dd
Increase alloc wait timeout in e2e test 2019-01-03 14:02:02 -06:00
Preetha 9e235f4cb6
Update e2e readme 2019-01-03 13:24:58 -06:00
Preetha 0071307414
Update README.md 2019-01-03 13:19:04 -06:00
Preetha 758ae0ca7c
Update README.md 2019-01-03 12:12:43 -06:00
Preetha Appan d182c0f5cd
Increase timeout in e2e test 2019-01-03 11:22:21 -06:00
Danielle Tomlinson d3b41a26c4 e2e: goimports e2eutil/utils.go 2019-01-03 13:31:49 +01:00
Preetha Appan 2845a556a3
Clean up map update code 2018-12-20 15:12:48 -06:00
Preetha Appan 1bebce3525
new e2e test for spread, and refactor affinity tests to share util methods 2018-12-19 21:25:32 -06:00
Preetha Appan df8b9ced10
Fix spread e2e test 2018-12-19 17:11:26 -06:00
Preetha Appan 79bd9770f2
Initial e2e tests for spread 2018-12-19 17:11:26 -06:00
Preetha Appan 9890ac7583
Make affinity e2e tests wait for leader through Before method 2018-12-19 09:56:44 -06:00
Preetha Appan 24788e9ab7
minor refactoring of affinities tests 2018-12-19 09:05:01 -06:00
Preetha Appan 29d2f8172a
Affinities e2e tests 2018-12-18 18:39:45 -06:00
Preetha 6c51232f55
Merge pull request #5012 from hashicorp/f-e2e-provisioning 
Terraform configs for e2e tests
 2018-12-18 13:45:58 -06:00
Preetha Appan fb8980b46d
added readme 2018-12-18 13:37:03 -06:00
Preetha Appan 1ba4674ce2
suggestions from code review 2018-12-17 15:06:22 -06:00
Jack Pearkes 844d981e77
Terraform configs for e2e tests 2018-12-17 11:40:09 -06:00
Alex Dadgar 86d9ad4397 fix iops bug and increase test matrix coverage 2018-12-11 15:28:21 -08:00
Alex Dadgar 99c386c076 skip e2e/vault if integration isn't set 2018-09-25 11:29:09 -07:00
Alex Dadgar f9c60c91d8 proper variable capture 2018-09-24 16:34:15 -07:00
Alex Dadgar 69cd345778 vet 2018-09-19 10:18:10 -07:00
Alex Dadgar e546215046 add a vault test matrix 2018-09-19 10:18:10 -07:00
Alex Dadgar 78c8276bc7 fix panic 2018-09-18 13:02:03 -07:00
Nick Ethier fff5ae622b
e2e/cli: comment fixups 2018-08-02 13:29:12 -04:00
Nick Ethier 2d31a59bfd
e2e/cli: add -run option to mimic go test 2018-07-31 15:21:47 -04:00
Nick Ethier 44652f455f
e2e/cli: add run command usage docs 2018-07-31 13:52:25 -04:00
Nick Ethier 69220df24a
e2e/cli: code review comments, restructing and cleanup 2018-07-31 13:52:25 -04:00
Nick Ethier 01dd1f5d65
e2e/cli: fix formatting 2018-07-31 13:52:25 -04:00
Nick Ethier 77018c6774
e2e/cli: port logging to use hclog 2018-07-31 13:51:30 -04:00
Nick Ethier 8abf4f9e25
e2e/cli: implemented run logic 2018-07-31 13:51:30 -04:00
Nick Ethier d44f5cbf4e
e2e/cli: use discover utility for nomad binary 2018-07-31 13:51:30 -04:00
Nick Ethier 33bb3d6a84
e2e/cli: initial cli implementation for nomad-e2e 2018-07-31 13:51:27 -04:00
Nick Ethier 6f55638d70
e2e/framework: fix linting 2018-07-31 13:03:52 -04:00
Nick Ethier 72d15da314
e2e: skip e2e tests during normal go test runs 2018-07-31 12:42:36 -04:00
Nick Ethier 2ad19e05f4
e2e/framework: code review fixes 2018-07-24 12:33:30 -04:00
Nick Ethier f9aadb5e52
e2e/framework: add framework.F context 2018-07-11 13:19:14 -04:00
Nick Ethier a333edca7c
e2e/framework: move *testing.T access to a context *framework.F which is scoped to each test 2018-07-11 13:12:23 -04:00
Nick Ethier c0ad8b5c69
e2e/framework: strip flag prefix and use testify/require for embedded assertions 2018-07-11 13:12:23 -04:00
Nick Ethier 03d1ec963c
e2e/framework: don't require NOMAD_ADDR and instead rely on defaults 2018-07-11 13:12:23 -04:00
Nick Ethier 8ece36a6bd
e2e/framework: remove 'step' naming and wording 2018-07-11 13:12:23 -04:00
Nick Ethier ac61ca7fd0
e2e/framework: add missing errorf var 2018-07-11 13:12:23 -04:00
Nick Ethier c3e3919637
e2e/framework: cleanup provisioner and more docs 2018-07-11 13:12:23 -04:00
Nick Ethier 5d4caa5698
e2e/framework: additional documentation 2018-07-11 13:12:23 -04:00
Nick Ethier 18b5c767e2
e2e/framework: add consul and vault client detection 2018-07-11 13:12:23 -04:00
Nick Ethier 026e86ecb9
e2e/framework: add minimal godocs 2018-07-11 13:12:23 -04:00
Nick Ethier 0f15bab3ba
e2e/framework: initial e2e framework implementation 2018-07-11 13:12:23 -04:00
Preetha Appan 27308b7c5e
Add more end to end tests for progress deadline and fix flaky test 2018-06-05 17:25:28 -05:00
Preetha Appan 6c09379bd7
Update end to end tests to use shorter progress deadlines 2018-05-07 14:55:01 -05:00
Preetha Appan b2e89d69d8
More e2e test fixes after changes to rescheduling during deployments 2018-05-07 14:55:01 -05:00
Preetha Appan 02567ade71
Some test fixes to e2e rescheduling tests 2018-05-07 14:55:01 -05:00
Michael Schurter 50e04c976e
consul: support canary tags for services 
Also refactor Consul ServiceClient to take a struct instead of a massive
set of arguments. Meant updating a lot of code but it should be far
easier to extend in the future as you will only need to update a single
struct instead of every single call site.

Adds an e2e test for canary tags.
 2018-05-07 14:55:01 -05:00
Michael Schurter d687761ebf rkt: test Stats() and always run tests 
Remove the NOMAD_TEST_RKT flag as a guard for rkt tests. Still require
Linux, root, and rkt to be installed. Only check for rkt installation
once in hopes of speeding up rkt tests a bit.
 2018-04-24 11:05:42 -07:00