Commit Graph

23736 Commits

Author SHA1 Message Date
Tim Gross d327a68696
operator debug: write NDJSON for large collections (#14610)
The `operator debug` command writes JSON files from API responses as a single
line containing an array of JSON objects. But some of these files can be
extremely large (GB's) for large production clusters, which makes it difficult
to parse them using typical line-oriented Unix command line tools that can
stream their inputs without consuming a lot of memory.

For collections that are typically large, instead emit newline-delimited JSON.

This changeset includes some first-pass refactoring of this command. It breaks
up monolithic methods that validate a path, create a file, serialize objects,
and write them to disk into smaller functions, some of which can now be
standalone to take advantage of generics.
2022-09-22 10:02:00 -04:00
James Rasell a25028c412
cli: fix a bug in operator API when setting HTTPS via address. (#14635)
Operators may have a setup whereby the TLS config comes from a
source other than setting Nomad specific env vars. In this case,
we should attempt to identify the scheme using the config setting
as a fallback.
2022-09-22 15:43:58 +02:00
Seth Hoenig 2088ca3345
cleanup more helper updates (#14638)
* cleanup: refactor MapStringStringSliceValueSet to be cleaner

* cleanup: replace SliceStringToSet with actual set

* cleanup: replace SliceStringSubset with real set

* cleanup: replace SliceStringContains with slices.Contains

* cleanup: remove unused function SliceStringHasPrefix

* cleanup: fixup StringHasPrefixInSlice doc string

* cleanup: refactor SliceSetDisjoint to use real set

* cleanup: replace CompareSliceSetString with SliceSetEq

* cleanup: replace CompareMapStringString with maps.Equal

* cleanup: replace CopyMapStringString with CopyMap

* cleanup: replace CopyMapStringInterface with CopyMap

* cleanup: fixup more CopyMapStringString and CopyMapStringInt

* cleanup: replace CopySliceString with slices.Clone

* cleanup: remove unused CopySliceInt

* cleanup: refactor CopyMapStringSliceString to be generic as CopyMapOfSlice

* cleanup: replace CopyMap with maps.Clone

* cleanup: run go mod tidy
2022-09-21 14:53:25 -05:00
Luiz Aoqui ad48401219
chore: move changelog file to the right folder (#14639) 2022-09-21 13:50:22 -04:00
Luiz Aoqui c3c8ae584f
api: provide more detail on ACL bootstrap request error (#14629) 2022-09-20 21:20:04 -04:00
Luiz Aoqui de2f9ca75d
Minor tweaks to the service views (#14619) 2022-09-19 17:05:21 -04:00
Luiz Aoqui 25a63195da
test: remove flaky Gate test (#14575)
The concurrent gate access test is flaky since it depends on the order
of operations of two concurrent goroutines. Despite the heavy bias
towards one of the results, it's still possible to end the execution
with a closed gate.

I believe this case was created to test an earlier implementation where
the gate state was stored and mutated internally, so the access had to
be protected by a lock. However, the final implementation changed this
approach to be only channel-based, so there is no need for this flaky
test anymore.
2022-09-19 11:31:03 -04:00
Tim Gross 38a6e7e343
remove 1.4.0 changelog entry that refers to bugfix on new code (#14611)
Bug fixes on new features in Nomad 1.4.0 don't need or want changelog entries in
the same changelog the feature appeared, so remove this one.
2022-09-16 16:14:02 -04:00
Phil Renaud d6c9676252
Added task links to various alloc tables (#14592)
* Added task links to various alloc tables

* Lintfix

* Border collapse and added to task group page

* Logs icon temporarily removed and localStorage added

* Mock task added to test

* Delog

* Two asserts in new test

* Remove commented-out code

* Changelog

* Removing args.allocation deps
2022-09-16 15:58:22 -04:00
Seth Hoenig bf0e83e3a0
deps: run all dependabot configs over the weekend (#14608) 2022-09-16 10:50:57 -05:00
Bryce Kalow a84d2de9be
website: content updates for developer (#14473)
Co-authored-by: Geoffrey Grosenbach <26+topfunky@users.noreply.github.com>
Co-authored-by: Anthony <russo555@gmail.com>
Co-authored-by: Ashlee Boyer <ashlee.boyer@hashicorp.com>
Co-authored-by: Ashlee M Boyer <43934258+ashleemboyer@users.noreply.github.com>
Co-authored-by: HashiBot <62622282+hashibot-web@users.noreply.github.com>
Co-authored-by: Kevin Wang <kwangsan@gmail.com>
2022-09-16 10:38:39 -05:00
Seth Hoenig c9bbe5230d
ci: no need to run unit tests for e2e/ui changes (#14607) 2022-09-16 09:58:06 -05:00
James Rasell 3f78a51fa5
e2e: use unique names for Connect ACL Consul policy names. (#14604)
In the event a single test fails to clear up properly after
itself, all other tests will fail as they attempt to create ACL
policies with the same names. This change ensures they use unique
ACL names, so when a single test fails, it is easy to identify it
is a problem with the test rather than the suite.
2022-09-16 13:35:40 +02:00
Derek Strickland 24af28dc30
Merge pull request #14602 from hashicorp/release/1.4.0-beta.1
Release/1.4.0 beta.1
2022-09-15 13:57:40 -04:00
Michael Schurter bd4b4b8f66
Data race fixes in tests and a new semgrep rule (#14594)
* test: don't use loop vars in goroutines

fixes a data race in the test

* test: copy objects in statestore before mutating

fixes data race in test

* test: @lgfa29's segmgrep rule for loops/goroutines

Found 2 places where we were improperly using loop variables inside
goroutines.
2022-09-15 10:35:08 -07:00
Derek Strickland 8f6406f5d0
Turn security scanning back on 2022-09-15 13:34:09 -04:00
Michael Schurter 6052aba96d
remove unused circbufwriter code (#14593)
The tests have data races and the code is unused in OSS and enterprise
codebases.

Bit more progress for #14236
2022-09-15 10:33:33 -07:00
Tim Gross 7779936e4e
testing: skip exec stream child process test (#14601)
This test is broken in CircleCI only. It works on GHA in both 20.04 and 22.04
and has been verified to work on real Nomad; temporarily commenting-out so that
we don't block unrelated CI runs.

WIP to fix in https://github.com/hashicorp/nomad/pull/14600
2022-09-15 11:53:12 -04:00
James Rasell 90d0b9157f
e2e: rewrite spread suite to use new e2e style. (#14598)
The rewrite refactors the suite to use the new style along with
other recent testing improvements. In order to ensure the spread
tests do not impact each other, there is new cleanup functionality
to ensure both the job and allocations are removed from state
before the test exits completely.
2022-09-15 17:12:20 +02:00
Tim Gross 81516db4b2
variables: fix ENT-only test failure in command tests (#14599)
The `TestVarGetCommand` test uses the wrong namespace in the autocomplete
test. The namespace only gets validated against if we have quota enforcement (or
more typically by ACL checks), so the test only fails in the ENT repo test runs.
2022-09-15 10:37:57 -04:00
hc-github-team-nomad-core 2b7ed191d7 Prepare for next release 2022-09-14 19:32:22 +00:00
hc-github-team-nomad-core a3a718e167 Generate files for 1.4.0-beta.1 release 2022-09-14 19:32:18 +00:00
Derek Strickland 280d472ca5
Update security-scan.hcl 2022-09-14 15:22:12 -04:00
hc-github-team-nomad-core 686b55cff5 Prepare for next release 2022-09-14 19:00:05 +00:00
hc-github-team-nomad-core b91437bb68 Generate files for 1.4.0-beta.1 release 2022-09-14 18:59:59 +00:00
Derek Strickland b160b74e5c Prepare release 1.4.0-beta.1 2022-09-14 14:51:38 -04:00
Tim Gross 89dfdef95d
variables: handler should catch errors before conflicts (#14591) 2022-09-14 13:14:17 -04:00
Phil Renaud cebfbb0c28
Stabilizing percy snapshots with faker (#14551)
* First attempt at stabilizing percy snapshots with faker

* Tokens seed moved to before management token generation

* Faker seed only in token test

* moving seed after storage clear

* And again, but back to no faker seeding

* Isolated seed and temporary log

* Setting seed(1) wherever we're snapshotting, or before establishing cluster scenarios

* Deliberate noop to see if percy is stable

* Changelog entry
2022-09-14 11:27:48 -04:00
Seth Hoenig 5187f92c5e
cleanup: create interface for check watcher and mock it in nsd tests (#14577)
* cleanup: create interface for check watcher and mock it in nsd tests

* cleanup: add comments for check watcher interface
2022-09-14 08:25:20 -05:00
Kyle Rarey dd361d9581
docs: Correct driver name for 'Nomad Task Group' autoscaler target (#14576) 2022-09-14 09:40:00 +02:00
Luiz Aoqui bff91bef32
build: always set osusergo (#14583)
The way this tag was previous declared would cause it to be overwritten
when the GO_TAGS env var was set.
2022-09-13 21:13:17 -04:00
Phil Renaud 2eb34564e8
Tooltip for node names (#14573) 2022-09-13 16:00:15 -04:00
Michael Schurter b2d22aef65
2 small data race fixes in logmon and check tests (#14538)
* logmon: fix data race around oldestLogFileIdx

* checks: fix 2 data races in tests

* logmon: move & rename lock to logically group
2022-09-13 12:54:06 -07:00
Mahmood Ali a9d5e4c510
scheduler: stopped-yet-running allocs are still running (#10446)
* scheduler: stopped-yet-running allocs are still running

* scheduler: test new stopped-but-running logic

* test: assert nonoverlapping alloc behavior

Also add a simpler Wait test helper to improve line numbers and save few
lines of code.

* docs: tried my best to describe #10446

it's not concise... feedback welcome

* scheduler: fix test that allowed overlapping allocs

* devices: only free devices when ClientStatus is terminal

* test: output nicer failure message if err==nil

Co-authored-by: Mahmood Ali <mahmood@hashicorp.com>
Co-authored-by: Michael Schurter <mschurter@hashicorp.com>
2022-09-13 12:52:47 -07:00
Phil Renaud 984e2cc4f1
Deregister keyboard blur handler (#14570) 2022-09-13 14:12:52 -04:00
Tim Gross 9636b0f837
docs: tweak some copy in the concept docs (#14566) 2022-09-13 13:21:09 -04:00
Seth Hoenig afc815c0c7
Merge pull request #14559 from hashicorp/docs-nsd-check-watcher
docs: add documentation for nomad service check restarts
2022-09-13 10:52:01 -05:00
Ashlee M Boyer fc973ebe0e
docs: Fixing heading order, adding text for links in /docs/ecosystem (#14549)
* Fixing heading order, adding text for links

* Apply suggestions from code review

Co-authored-by: Tim Gross <tgross@hashicorp.com>

* Applying more suggestions from code review

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-09-13 10:59:02 -04:00
Seth Hoenig 5b661ec84d docs: update docs for NSD check restart 2022-09-13 09:59:02 -05:00
Seth Hoenig bf4dd30919
Merge pull request #14553 from hashicorp/f-nsd-check-watcher
servicedisco: implement check_restart support for nomad service checks
2022-09-13 09:55:51 -05:00
Tim Gross eb757606f3
changelog entry for variables (#14509) 2022-09-13 10:25:26 -04:00
Tim Gross 357e7f4521
docs: include path in ACL requirements for variables (#14561)
Also add links to the ACL policy reference and variables concepts docs near the
top of the page.
2022-09-13 10:21:29 -04:00
Tim Gross 6dd79ca995
docs: variables HTTP API documentation (#14516) 2022-09-13 10:18:26 -04:00
Charlie Voiselle 6ab59d2aa6
var: Correct 0-index CAS Deletes (#14555)
* Add missing 0 case for VarDeleteCAS, more comments
* Add tests for VarDeleteCAS
2022-09-13 10:12:08 -04:00
James Rasell 41284382dd
api: update keyring comment to reflect correct feature name. (#14558) 2022-09-13 10:05:03 -04:00
Seth Hoenig 9a943107c7 servicedisco: implement check_restart for nomad service checks
This PR implements support for check_restart for checks registered
in the Nomad service provider.

Unlike Consul, Nomad service checks never report a "warning" status,
and so the check_restart.ignore_warnings configuration is not valid
for Nomad service checks.
2022-09-13 08:59:23 -05:00
Tim Gross cab787c44d
docs: keyring HTTP API documentation (#14513) 2022-09-13 09:46:54 -04:00
Seth Hoenig b960925939
Merge pull request #14546 from hashicorp/f-refactor-check-watcher
client: refactor check watcher to be reusable
2022-09-13 07:32:32 -05:00
James Rasell d65267c60c
e2e: do not assume clean cluster when checking return objects. (#14557) 2022-09-13 14:25:19 +02:00
Phil Renaud 7a1560e509
[ui] Service health check in-session history (#14515)
* Preliminary version

* Addition of a filtering helper and more styling for service check history

* Fixed-widths on table cols

* Account for new rows in test

* Explanation for magic numbers
2022-09-12 22:10:43 -04:00