Commit Graph

22955 Commits

Author SHA1 Message Date
James Rasell 24b499791d
deps: update consul-template to v0.29.0 (#12747)
* deps: update consul-template to v0.29.0

* changelog: add entry for #12747
2022-04-22 09:58:54 -07:00
Phil Renaud ab557b15e0
Adding changelog note (#12753) 2022-04-22 12:38:49 -04:00
Phil Renaud 15872cc2d4
[ui] Disconnected Clients: "Unknown" allocations in the UI (#12544)
* Unknown status for allocations accounted for

* Canary string removed

* Test cleanup

* Generate unknown in mirage

* aacidentally oovervoowled

* Update ui/app/components/allocation-status-bar.js

Co-authored-by: Derek Strickland <1111455+DerekStrickland@users.noreply.github.com>

* Disconnected state on job status in client

* Renaming Disconnected to Unknown in the job-status-in-client

* Unknown accounted for on job rows filtering and testsfix

* Adding lostAllocs as a computed dependency

* Unknown client status within acceptance test

* Swatches updated and PR comments addressed

* Unknown and disconnected added to test fixtures

Co-authored-by: Derek Strickland <1111455+DerekStrickland@users.noreply.github.com>
2022-04-22 11:25:02 -04:00
Luiz Aoqui a8cc633156
vault: revert support for entity aliases (#12723)
After a more detailed analysis of this feature, the approach taken in
PR #12449 was found to be not ideal due to poor UX (users are
responsible for setting the entity alias they would like to use) and
issues around jobs potentially masquerading itself as another Vault
entity.
2022-04-22 10:46:34 -04:00
Seth Hoenig 86d90cd906
Merge pull request #12720 from hashicorp/f-arbitrary-addresses
services: enable setting arbitrary address value in service registrations
2022-04-22 09:34:02 -05:00
Seth Hoenig ed71d5db2b services: fix imports 2022-04-22 09:15:51 -05:00
Seth Hoenig c4aab10e53 services: cr followup 2022-04-22 09:14:29 -05:00
Seth Hoenig ae6048eafa services: format ipv6 in nomad service info output
Co-authored-by: Michael Schurter <mschurter@hashicorp.com>
2022-04-22 09:14:29 -05:00
Seth Hoenig 3fcac242c6 services: enable setting arbitrary address value in service registrations
This PR introduces the `address` field in the `service` block so that Nomad
or Consul services can be registered with a custom `.Address.` to advertise.

The address can be an IP address or domain name. If the `address` field is
set, the `service.address_mode` must be set in `auto` mode.
2022-04-22 09:14:29 -05:00
Tim Gross f7d6841dd2
E2E: remove platform specific realpath code from UI run script (#12750)
We don't need the absolute path for any of the commands in this script
so long as we `cd` into the source directory path. Doing this removes
the need for weird platform-specific tricks we have to do with
realpath vs GNU realpath.
2022-04-22 10:10:18 -04:00
James Rasell b5d10bcece
docs: add upgrade note for Consul implicit constraint. (#12749) 2022-04-22 15:53:27 +02:00
Tim Gross 3fc1b67396
CSI: handle nil topologies safely in command line (#12751) 2022-04-22 09:25:04 -04:00
Tim Gross 7dd3910e51
E2E: fix debug logging on disconnected clients test (#12621) 2022-04-22 09:07:05 -04:00
James Rasell 046831466c
cli: add pagination flags to service info command. (#12730) 2022-04-22 10:32:40 +02:00
Tim Gross d200a66509
E2E: make UIs runnable from any working directory (#12739)
The E2E test runner is running from the root of the Nomad
repository. Make this run independent of the working directory for
convenience of developers and the test runner.
2022-04-21 17:00:01 -04:00
Michael Schurter 5db3a671db
cli: add -json flag to support job commands (#12591)
* cli: add -json flag to support job commands

While the CLI has always supported running JSON jobs, its support has
been via HCLv2's JSON parsing. I have no idea what format it expects the
job to be in, but it's absolutely not in the same format as the API
expects.

So I ignored that and added a new -json flag to explicitly support *API*
style JSON jobspecs.

The jobspecs can even have the wrapping {"Job": {...}} envelope or not!

* docs: fix example for `nomad job validate`

We haven't been able to validate inside driver config stanzas ever since
the move to task driver plugins. 😭
2022-04-21 13:20:36 -07:00
Tim Gross f4287c870d
cli: detect directory when applying namespace spec file (#12738)
The new `namespace apply` feature that allows for passing a namespace
specification file detects the difference between an empty namespace
and a namespace specification by checking if the file exists. For most
cases, the file will have an extension like `.hcl` and so there's
little danger that a user will apply a file spec when they intended to
apply a file name.

But because directory names typically don't include an extension,
you're much more likely to collide when trying to `namespace apply` by
name only, and then you get a confusing error message of the form:

   Failed to read file: read $namespace: is a directory

Detect the case where the namespace name collides with a directory in
the current working directory, and skip trying to load the directory.
2022-04-21 14:53:45 -04:00
Phil Renaud a5bef3ce72
[ui, bugfix] Link fix for volumes where per_alloc=true (#12713)
* Allocation page linkfix

* fix added to task page and computed prop moved to allocation model

* Fallback query added to task group when specific volume isnt knowable

* Delog

* link text reflects alloc suffix

* Helper instead of in-template conditionals

* formatVolumeName unit test

* Removing unused helper import
2022-04-21 13:57:18 -04:00
Seth Hoenig 4644bb9941
Merge pull request #12736 from hashicorp/build-update-go-1.17.9
build: update golang to 1.17.9
2022-04-21 12:13:07 -05:00
Seth Hoenig c5ce4927c5 build: update golang version script to use .go-version file 2022-04-21 12:10:14 -05:00
Seth Hoenig 382087a6df
Merge pull request #12737 from hashicorp/buid-update-ec2-instances
build: update ec2 instance profiles
2022-04-21 11:57:40 -05:00
Seth Hoenig c87bfe398f build: update ec2 instance profiles
using tools/ec2info
2022-04-21 11:47:40 -05:00
Seth Hoenig bf54ef26be build: update golang to 1.17.9 2022-04-21 11:43:01 -05:00
Tim Gross 140dbab832
docker: back out cgroup v2 OOM detection (#12735)
When shutting down an allocation that ends up needing to be
force-killed, we're getting a spurious "OOM Killed (137)" message on
the task termination event. We introduced this as part of cgroups v2
support because the Docker daemon isn't detecting the container status
correctly. Although exit code 137 is the exit code we get for
OOM-killed processes, that's because OOM kill is a `SIGKILL`. So any
sigkilled process will get that exit code.
2022-04-21 12:31:34 -04:00
Tim Gross dc013b5267
E2E: set longer timeout for CSI plugin alloc start (#12732)
The CSI plugin allocations take a while to be marked healthy,
sometimes causing E2E test flakes during the setup phase of the
tests. There's nothing CSI specific about marking plugin allocs
healthy, as the plugin supervisor hook does all the fingerprinting in
the postrun hook (the prestart hook just makes a couple of empty
directories). The timeouts we're seeing may be because of where we're
pulling the images from; most our jobs pull from a CDN-backed public
registry whereas these are pulling from ECR. Set a 1min timeout for
these to make sure we have enough time to pull the image and start the
task.
2022-04-21 11:11:43 -04:00
James Rasell 716b8e658b
api: Add support for filtering and pagination to the node list endpoint (#12727) 2022-04-21 17:04:33 +02:00
Tim Gross 79a9d788d2
docs: fix broken link from `template` to client config (#12733) 2022-04-21 11:04:04 -04:00
Derek Strickland 5e309f3f33
reconciler: Handle canaries when client disconnects (#12539)
* plan_apply: Allow node updates in disconnected node plans
* plan: Keep the job when persisting unknown allocs
* reconciler: stop unknown allocs when stopping all
* reconcile_util: reorder filtering to handle canaries; skip rescheduling unknown
* heartbeat: Fix bug in node heartbeating
2022-04-21 10:05:58 -04:00
Tim Gross 2ad9f6bc5f
E2E: playwright configuration and smoke test (#12721)
Scripts for running playwright tests in a Docker container that has
chromium and webkit preinstalled. Includes a basic smoke test for
authentication so that we can be sure the test rig is working
end-to-end. Wiring this up in CI will be in an upcoming PR.
2022-04-21 09:13:10 -04:00
James Rasell c4195c452a
docs: update HCL2 dynamic example to use block with label. (#12715) 2022-04-21 10:18:04 +02:00
James Rasell 257e1c4f96
autopilot: correctly return errors within state functions. (#12714) 2022-04-21 08:54:50 +02:00
Luiz Aoqui bf7110b4c8
ui: fix bug that prevented files streaming (#12719)
During the Ember dependecy upgrade work,
https://github.com/hashicorp/nomad/commit/ce8c039f4ce7359d60ede5dee36b9cef82
moved the `isSupported` method from using Ember's `reopenClass` to a
getter, but `reopenClass` creates a static method, so the getter must be
static as well.
2022-04-20 14:39:18 -04:00
Gowtham 1ff8b5f759
Add Concurrent Download Support for artifacts (#11531)
* add concurrent download support - resolves #11244

* format imports

* mark `wg.Done()` via `defer`

* added tests for successful and failure cases and resolved some goleak

* docs: add changelog for #11531

* test typo fixes and improvements

Co-authored-by: Michael Schurter <mschurter@hashicorp.com>
2022-04-20 10:15:56 -07:00
James Rasell 010acce59f
job_hooks: add implicit constraint when using Consul for services. (#12602) 2022-04-20 14:09:13 +02:00
James Rasell 42068f8823
client: add NOMAD_SHORT_ALLOC_ID allocation env var. (#12603) 2022-04-20 10:30:48 +02:00
Tim Gross c4d92205b4
E2E: provide options for reverse proxy for web UI (#12671)
Our E2E test environment is deployed with mTLS, but it's impractical
for us to use mTLS in headless browsers for automated testing (or even
in manual testing). Provide certificates for proxying the web UI via
Nginx. This proxy uses client certs for proxying to the HTTP endpoint
and a self-signed cert for the browser-facing endpoint. We can accept
certificate errors in the automated tests we'll be adding in the next
step of this work.
2022-04-19 16:55:05 -04:00
Tim Gross 70c262eb95
E2E: terraform provisioner upgrades (#12652)
While working on infrastructure for testing the UI in E2E, we needed
to upgrade the certificate provider. Performing a provider upgrade via
the TF `init -upgrade` brought in updates for the file and AWS
providers as well. These updates include deprecating the use of
`sensitive_content` fields, removing CA algorithm parameters that can
be inferred from keys, and removing the requirement to manually
specify AWS assume role parameters in the provider config if they're
available in the calling environment's AWS config file (as they are
via doormat or our E2E environment).
2022-04-19 14:27:14 -04:00
Seth Hoenig 8084dd29a1
Merge pull request #12604 from hashicorp/b-fixup-chroot-test
ci: fixup task runner chroot test
2022-04-19 12:58:03 -05:00
Seth Hoenig 46066fb7fb
Merge pull request #12622 from hashicorp/b-fix-docker-logger-test
ci: fix docker logger not supported test
2022-04-19 12:57:47 -05:00
Seth Hoenig d1bda4a954 ci: fixup task runner chroot test
This PR is 2 fixes for the flaky TestTaskRunner_TaskEnv_Chroot test.

And also the TestTaskRunner_Download_ChrootExec test.

- Use TinyChroot to stop copying gigabytes of junk, which causes GHA
to fail to create the environment in time.

- Pre-create cgroups on V2 systems. Normally the cgroup directory is
managed by the cpuset manager, but that is not active in taskrunner tests,
so create it by hand in the test framework.
2022-04-19 10:37:46 -05:00
Seth Hoenig 16cab10346 ci: fix docker logger not supported test
This test checks for behavior when asking for logs of a docker task
configured with a log driver that does not support streaming logs.

Previously this was using the 'gelf' log driver, but it seems that no
longer returns an error as expected. Instead we can just use the 'none'
log driver, which has the desired effect

2022-04-19T10:23:19.129-0500 [ERROR] docklog/docker_logger.go:133: log streaming ended with terminal error: error="API error (501): configured logging driver does not support reading"
2022-04-19 10:27:01 -05:00
Luiz Aoqui 8dccc48f17
changelog: fix entry for #11927 (#12577) 2022-04-19 10:46:25 -04:00
Luiz Aoqui 950a2109aa
changelog: add entry for #11944 (#12578) 2022-04-19 10:46:11 -04:00
Seth Hoenig 411158acff
Merge pull request #12586 from hashicorp/f-local-si-token
connect: create SI tokens in local scope
2022-04-19 07:53:01 -05:00
Seth Hoenig a7950e5624 cl: add missing prefix 2022-04-19 07:48:56 -05:00
Derek Strickland 7c6eb47b78
`consul-template`: revert `function_denylist` logic (#12071)
* consul-template: replace config rather than append
Co-authored-by: Seth Hoenig <seth.a.hoenig@gmail.com>
2022-04-18 13:57:56 -04:00
chavacava eb1c42e643
QueryOptions.SetTimeToBlock should take pointer receiver
Fixes a bug where blocking queries that are retried don't have their blocking 
timeout reset, resulting in them running longer than expected.
2022-04-18 10:41:27 -04:00
Tim Gross 0cf14a49cc
CI: build binaries for UI branches (#12594)
Build binaries for every code change, not just backend code
changes. This means that we'll have up-to-date compiled assets for
every commit available in CircleCI artifacts.
2022-04-18 10:29:20 -04:00
Seth Hoenig df587d8263 docs: update documentation with connect acls changes
This PR updates the changelog, adds notes the 1.3 upgrade guide, and
updates the connect integration docs with documentation about the new
requirement on Consul ACL policies of Consul agent default anonymous ACL
tokens.
2022-04-18 08:22:33 -05:00
Jorge Marey 707c7f3a11 Change consul SI tokens to be local 2022-04-18 08:22:33 -05:00