Commit Graph

23905 Commits

Author SHA1 Message Date
Seth Hoenig 87ec5fdee5
deps: update set and test (#14680)
This PR updates go-set and shoenig/test, which introduced some breaking
API changes.
2022-09-26 08:28:03 -05:00
Phil Renaud 497bd02169
[ui] Warn users when they leave an edited but unsaved variable page (#14665)
* Warning on attempt to leave

* Lintfix

* Only router.off once

* Dont warn on transition when only updating queryparams

* Remove double-push and queryparam-only issues, thanks @lgfa29

* Acceptance tests

* Changelog
2022-09-23 16:53:40 -04:00
Phil Renaud a28e1bcc1e
[ui] Service Healthchecks: styles for pseudo-timestamp axis (#14677)
* Styles for pseudo-timestamp axis

* Changelog
2022-09-23 16:53:28 -04:00
Tim Gross 17aee4d69c
fingerprint: don't clear Consul/Vault attributes on failure (#14673)
Clients periodically fingerprint Vault and Consul to ensure the server has
updated attributes in the client's fingerprint. If the client can't reach
Vault/Consul, the fingerprinter clears the attributes and requires a node
update. Although this seems like correct behavior so that we can detect
intentional removal of Vault/Consul access, it has two serious failure modes:

(1) If a local Consul agent is restarted to pick up configuration changes and the
client happens to fingerprint at that moment, the client will update its
fingerprint and result in evaluations for all its jobs and all the system jobs
in the cluster.

(2) If a client loses Vault connectivity, the same thing happens. But the
consequences are much worse in the Vault case because Vault is not run as a
local agent, so Vault connectivity failures are highly correlated across the
entire cluster. A 15 second Vault outage will cause a new `node-update`
evalution for every system job on the cluster times the number of nodes, plus
one `node-update` evaluation for every non-system job on each node. On large
clusters of 1000s of nodes, we've seen this create a large backlog of evaluations.

This changeset updates the fingerprinting behavior to keep the last fingerprint
if Consul or Vault queries fail. This prevents a storm of evaluations at the
cost of requiring a client restart if Consul or Vault is intentionally removed
from the client.
2022-09-23 14:45:12 -04:00
Charlie Voiselle 0211240a03
[vars] Update Consul Template dependency (#14667)
* Bump CT dependency
* go mod tidied
2022-09-22 16:44:48 -04:00
Michael Schurter 6161b417f3
test: add e2e for non-overlapping placements (#14646)
* test: add e2e for non-overlapping placements

Followup to #10446

Fails (as expected) against 1.3.x at the wait for blocked eval (because
the allocs are allowed to overlap).

Passes against 1.4.0-beta.1 (as expected).

* Update e2e/overlap/overlap_test.go

Co-authored-by: James Rasell <jrasell@users.noreply.github.com>
2022-09-22 13:06:17 -07:00
Derek Strickland a30fb3b58e
Update multiregion.mdx 2022-09-22 14:56:21 -04:00
Derek Strickland 78caaa2c38 multiregion: Added a section for multiregion parameterized job dispatch 2022-09-22 14:50:15 -04:00
Seth Hoenig ae5b800085
cleanup: rearrange mocks package (#14660)
This PR splits up the nomad/mock package into more files. Specific features
that have a lot of mocks get their own file (e.g. acl, variables, csi, connect, etc.).
Above that, functions that return jobs/allocs/nodes are in the job/alloc/node file. And
lastly other mocks/helpers are in mock.go
2022-09-22 13:49:58 -05:00
Derek Strickland 6874997f91
scheduler: Fix bug where the would treat multiregion jobs as paused for job types that don't use deployments (#14659)
* scheduler: Fix bug where the scheduler would treat multiregion jobs as paused for job types that don't use deployments

Co-authored-by: Tim Gross <tgross@hashicorp.com>

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-09-22 14:31:27 -04:00
Jorge Marey 92158a1c62
connect: add nomad env to envoy bootstrap (#12959)
* Add nomad env to envoy bootstrap

* Add changelog file
2022-09-22 13:18:18 -05:00
Charlie Voiselle 15bb444da8
[vars:api] Return fake QueryMeta on 403s with Peek() (#14661)
Consul-template expects that the Response Metadata is non-zero, but on a 403 we do not pass back QueryMeta headers.
2022-09-22 13:15:05 -04:00
Florian Apolloner f66d61e17f
consul: Removed unused ConsulUsage.Kinds. (#11303) 2022-09-22 10:07:14 -05:00
Phil Renaud eca0e7bf56
[ui] task logs in sidebar (#14612)
* button styles

* Further styles including global toggle adjustment

* sidebar funcs and header

* Functioning task logs in high-level sidebars

* same-lineify the show tasks toggle

* Changelog

* Full-height sidebar calc in css, plz drop soon container queries

* Active status and query params for allocations page

* Reactive shouldShowLogs getter and added to client and task group pages

* Higher order func passing, thanks @DingoEatingFuzz

* Non-service job types get allocation params passed

* Keyframe animation for task log sidebar

* Acceptance test

* A few more sub-row tests

* Lintfix
2022-09-22 10:58:52 -04:00
Tim Gross c29c4bd66c
cli: remove deprecated `eval status -json` list behavior (#14651)
In Nomad 1.2.6 we shipped `eval list`, which accepts a `-json` flag, and
deprecated the usage of `eval status` without an evaluation ID with an upgrade
note that it would be removed in Nomad 1.4.0. This changeset completes that
work.
2022-09-22 10:56:32 -04:00
Seth Hoenig 3f29e59e6e
build: update go version to go1.19.1 (#14653)
minor update, no CL needed
2022-09-22 09:40:01 -05:00
Jorge Marey 584ddfe859
Add Namespace, Job and Group to envoy stats (#14311) 2022-09-22 10:38:21 -04:00
Seth Hoenig e144d2bd00
mods: move require statement with the rest (#14652) 2022-09-22 09:36:02 -05:00
Luiz Aoqui e0ba6400a7
cli: print success message on var put (#14620) 2022-09-22 10:18:01 -04:00
Tim Gross d327a68696
operator debug: write NDJSON for large collections (#14610)
The `operator debug` command writes JSON files from API responses as a single
line containing an array of JSON objects. But some of these files can be
extremely large (GB's) for large production clusters, which makes it difficult
to parse them using typical line-oriented Unix command line tools that can
stream their inputs without consuming a lot of memory.

For collections that are typically large, instead emit newline-delimited JSON.

This changeset includes some first-pass refactoring of this command. It breaks
up monolithic methods that validate a path, create a file, serialize objects,
and write them to disk into smaller functions, some of which can now be
standalone to take advantage of generics.
2022-09-22 10:02:00 -04:00
James Rasell a25028c412
cli: fix a bug in operator API when setting HTTPS via address. (#14635)
Operators may have a setup whereby the TLS config comes from a
source other than setting Nomad specific env vars. In this case,
we should attempt to identify the scheme using the config setting
as a fallback.
2022-09-22 15:43:58 +02:00
Seth Hoenig 2088ca3345
cleanup more helper updates (#14638)
* cleanup: refactor MapStringStringSliceValueSet to be cleaner

* cleanup: replace SliceStringToSet with actual set

* cleanup: replace SliceStringSubset with real set

* cleanup: replace SliceStringContains with slices.Contains

* cleanup: remove unused function SliceStringHasPrefix

* cleanup: fixup StringHasPrefixInSlice doc string

* cleanup: refactor SliceSetDisjoint to use real set

* cleanup: replace CompareSliceSetString with SliceSetEq

* cleanup: replace CompareMapStringString with maps.Equal

* cleanup: replace CopyMapStringString with CopyMap

* cleanup: replace CopyMapStringInterface with CopyMap

* cleanup: fixup more CopyMapStringString and CopyMapStringInt

* cleanup: replace CopySliceString with slices.Clone

* cleanup: remove unused CopySliceInt

* cleanup: refactor CopyMapStringSliceString to be generic as CopyMapOfSlice

* cleanup: replace CopyMap with maps.Clone

* cleanup: run go mod tidy
2022-09-21 14:53:25 -05:00
Luiz Aoqui ad48401219
chore: move changelog file to the right folder (#14639) 2022-09-21 13:50:22 -04:00
Luiz Aoqui c3c8ae584f
api: provide more detail on ACL bootstrap request error (#14629) 2022-09-20 21:20:04 -04:00
Luiz Aoqui de2f9ca75d
Minor tweaks to the service views (#14619) 2022-09-19 17:05:21 -04:00
Luiz Aoqui 25a63195da
test: remove flaky Gate test (#14575)
The concurrent gate access test is flaky since it depends on the order
of operations of two concurrent goroutines. Despite the heavy bias
towards one of the results, it's still possible to end the execution
with a closed gate.

I believe this case was created to test an earlier implementation where
the gate state was stored and mutated internally, so the access had to
be protected by a lock. However, the final implementation changed this
approach to be only channel-based, so there is no need for this flaky
test anymore.
2022-09-19 11:31:03 -04:00
Tim Gross 38a6e7e343
remove 1.4.0 changelog entry that refers to bugfix on new code (#14611)
Bug fixes on new features in Nomad 1.4.0 don't need or want changelog entries in
the same changelog the feature appeared, so remove this one.
2022-09-16 16:14:02 -04:00
Phil Renaud d6c9676252
Added task links to various alloc tables (#14592)
* Added task links to various alloc tables

* Lintfix

* Border collapse and added to task group page

* Logs icon temporarily removed and localStorage added

* Mock task added to test

* Delog

* Two asserts in new test

* Remove commented-out code

* Changelog

* Removing args.allocation deps
2022-09-16 15:58:22 -04:00
Seth Hoenig bf0e83e3a0
deps: run all dependabot configs over the weekend (#14608) 2022-09-16 10:50:57 -05:00
Bryce Kalow a84d2de9be
website: content updates for developer (#14473)
Co-authored-by: Geoffrey Grosenbach <26+topfunky@users.noreply.github.com>
Co-authored-by: Anthony <russo555@gmail.com>
Co-authored-by: Ashlee Boyer <ashlee.boyer@hashicorp.com>
Co-authored-by: Ashlee M Boyer <43934258+ashleemboyer@users.noreply.github.com>
Co-authored-by: HashiBot <62622282+hashibot-web@users.noreply.github.com>
Co-authored-by: Kevin Wang <kwangsan@gmail.com>
2022-09-16 10:38:39 -05:00
Seth Hoenig c9bbe5230d
ci: no need to run unit tests for e2e/ui changes (#14607) 2022-09-16 09:58:06 -05:00
James Rasell 3f78a51fa5
e2e: use unique names for Connect ACL Consul policy names. (#14604)
In the event a single test fails to clear up properly after
itself, all other tests will fail as they attempt to create ACL
policies with the same names. This change ensures they use unique
ACL names, so when a single test fails, it is easy to identify it
is a problem with the test rather than the suite.
2022-09-16 13:35:40 +02:00
Derek Strickland 24af28dc30
Merge pull request #14602 from hashicorp/release/1.4.0-beta.1
Release/1.4.0 beta.1
2022-09-15 13:57:40 -04:00
Michael Schurter bd4b4b8f66
Data race fixes in tests and a new semgrep rule (#14594)
* test: don't use loop vars in goroutines

fixes a data race in the test

* test: copy objects in statestore before mutating

fixes data race in test

* test: @lgfa29's segmgrep rule for loops/goroutines

Found 2 places where we were improperly using loop variables inside
goroutines.
2022-09-15 10:35:08 -07:00
Derek Strickland 8f6406f5d0
Turn security scanning back on 2022-09-15 13:34:09 -04:00
Michael Schurter 6052aba96d
remove unused circbufwriter code (#14593)
The tests have data races and the code is unused in OSS and enterprise
codebases.

Bit more progress for #14236
2022-09-15 10:33:33 -07:00
Tim Gross 7779936e4e
testing: skip exec stream child process test (#14601)
This test is broken in CircleCI only. It works on GHA in both 20.04 and 22.04
and has been verified to work on real Nomad; temporarily commenting-out so that
we don't block unrelated CI runs.

WIP to fix in https://github.com/hashicorp/nomad/pull/14600
2022-09-15 11:53:12 -04:00
James Rasell 90d0b9157f
e2e: rewrite spread suite to use new e2e style. (#14598)
The rewrite refactors the suite to use the new style along with
other recent testing improvements. In order to ensure the spread
tests do not impact each other, there is new cleanup functionality
to ensure both the job and allocations are removed from state
before the test exits completely.
2022-09-15 17:12:20 +02:00
Tim Gross 81516db4b2
variables: fix ENT-only test failure in command tests (#14599)
The `TestVarGetCommand` test uses the wrong namespace in the autocomplete
test. The namespace only gets validated against if we have quota enforcement (or
more typically by ACL checks), so the test only fails in the ENT repo test runs.
2022-09-15 10:37:57 -04:00
hc-github-team-nomad-core 2b7ed191d7 Prepare for next release 2022-09-14 19:32:22 +00:00
hc-github-team-nomad-core a3a718e167 Generate files for 1.4.0-beta.1 release 2022-09-14 19:32:18 +00:00
Derek Strickland 280d472ca5
Update security-scan.hcl 2022-09-14 15:22:12 -04:00
hc-github-team-nomad-core 686b55cff5 Prepare for next release 2022-09-14 19:00:05 +00:00
hc-github-team-nomad-core b91437bb68 Generate files for 1.4.0-beta.1 release 2022-09-14 18:59:59 +00:00
Derek Strickland b160b74e5c Prepare release 1.4.0-beta.1 2022-09-14 14:51:38 -04:00
Tim Gross 89dfdef95d
variables: handler should catch errors before conflicts (#14591) 2022-09-14 13:14:17 -04:00
Phil Renaud cebfbb0c28
Stabilizing percy snapshots with faker (#14551)
* First attempt at stabilizing percy snapshots with faker

* Tokens seed moved to before management token generation

* Faker seed only in token test

* moving seed after storage clear

* And again, but back to no faker seeding

* Isolated seed and temporary log

* Setting seed(1) wherever we're snapshotting, or before establishing cluster scenarios

* Deliberate noop to see if percy is stable

* Changelog entry
2022-09-14 11:27:48 -04:00
Seth Hoenig 5187f92c5e
cleanup: create interface for check watcher and mock it in nsd tests (#14577)
* cleanup: create interface for check watcher and mock it in nsd tests

* cleanup: add comments for check watcher interface
2022-09-14 08:25:20 -05:00
Kyle Rarey dd361d9581
docs: Correct driver name for 'Nomad Task Group' autoscaler target (#14576) 2022-09-14 09:40:00 +02:00
Luiz Aoqui bff91bef32
build: always set osusergo (#14583)
The way this tag was previous declared would cause it to be overwritten
when the GO_TAGS env var was set.
2022-09-13 21:13:17 -04:00