Commit graph

22810 commits

Author SHA1 Message Date
Mike Nomitch 7405ebbad1
Add max client disconnect docs (#12467)
Co-authored-by: Derek Strickland <1111455+DerekStrickland@users.noreply.github.com>
2022-04-06 08:54:14 -04:00
Derek Strickland 43d20ebdbd disconnected clients: TaskGroup validation (#12418)
* TaskGroup: Validate that max_client_disconnect and stop_after_client_disconnect are mutually exclusive.
2022-04-05 17:14:50 -04:00
Derek Strickland bd719bc7b8 reconciler: 2 phase reconnects and tests (#12333)
* structs: Add alloc.Expired & alloc.Reconnected functions. Add Reconnect eval trigger by.

* node_endpoint: Emit new eval for reconnecting unknown allocs.

* filterByTainted: handle 2 phase commit filtering rules.

* reconciler: Append AllocState on disconnect. Logic updates from testing and 2 phase reconnects.

* allocs: Set reconnect timestamp. Destroy if not DesiredStatusRun. Watch for unknown status.
2022-04-05 17:13:10 -04:00
Derek Strickland bb376320a2 comments: update some stale comments referencing deprecated config name (#12271)
* comments: update some stale comments referencing deprecated config name
2022-04-05 17:12:23 -04:00
Derek Strickland cc854d842e Add description for allocs stopped due to reconnect (#12270) 2022-04-05 17:12:23 -04:00
Derek Strickland d86ab290a0 Add unknown to TaskGroupSummary (#12269) 2022-04-05 17:12:23 -04:00
Derek Strickland fd04a24ac7 disconnected clients: ensure servers meet minimum required version (#12202)
* planner: expose ServerMeetsMinimumVersion via Planner interface
* filterByTainted: add flag indicating disconnect support
* allocReconciler: accept and pass disconnect support flag
* tests: update dependent tests
2022-04-05 17:12:23 -04:00
Derek Strickland 8e9f8be511 MaxClientDisconnect Jobspec checklist (#12177)
* api: Add struct, conversion function, and tests
* TaskGroup: Add field, validation, and tests
* diff: Add diff handler and test
* docs: Update docs
2022-04-05 17:12:23 -04:00
Derek Strickland d7f44448e1 disconnected clients: Observability plumbing (#12141)
* Add disconnects/reconnect to log output and emit reschedule metrics

* TaskGroupSummary: Add Unknown, update StateStore logic, add to metrics
2022-04-05 17:12:23 -04:00
Derek Strickland c7a5b17251 Fix client test reconnect test; Remove guard test (#12173)
* Update reconnect test to new algorithm and interface; remove guard test
2022-04-05 17:12:23 -04:00
Derek Strickland 3cbd76ea9d disconnected clients: Add reconnect task event (#12133)
* Add TaskClientReconnectedEvent constant
* Add allocRunner.Reconnect function to manage task state manually
* Removes server-side push
2022-04-05 17:12:23 -04:00
DerekStrickland 9395e1878a reconciler: fix loop control bug 2022-04-05 17:12:22 -04:00
DerekStrickland da9bc350c8 evaluateNodePlan: validate plans for disconnected nodes 2022-04-05 17:12:22 -04:00
DerekStrickland 73fdf5a919 NodeStatusDisconnected: support state transitions for new node status 2022-04-05 17:12:18 -04:00
DerekStrickland c3d985047b client: reconnect unknown allocations and sync state 2022-04-05 17:10:41 -04:00
Derek Strickland b128769e19 reconciler: support disconnected clients (#12058)
* Add merge helper for string maps
* structs: add statuses, MaxClientDisconnect, and helper funcs
* taintedNodes: Include disconnected nodes
* upsertAllocsImpl: don't use existing ClientStatus when upserting unknown
* allocSet: update filterByTainted and add delayByMaxClientDisconnect
* allocReconciler: support disconnecting and reconnecting allocs
* GenericScheduler: upsert unknown and queue reconnecting

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-04-05 17:10:37 -04:00
Shishir a6801f73d1
cli: add -quiet to nomad node status command. (#12426) 2022-04-05 15:53:43 -04:00
Jai 71673f03b2
ui: eval filter (#12243)
* ui:  add triggeredBy filter

* add namespace filter

* fix:  namespace is a reserved keyword

* ui: filter by type and search

* fix:  rename closure action to

* chore:  fix data-test-attr
2022-04-05 15:30:36 -04:00
Jai 7690fda6ba
Epic: Evaluation Detail Sidebar (#12370)
* chore: prettify gutter-menu

* chore:  add portal packages

* styling:  add styles sidebar and portal behavior

* ui:  sidebar component

* ui:  create and implement statechart for evals

* ui:  actor-relationship service and provider component

* ui:  d3 hierarchy computation

* chore:  add render-modifiers and curved arrows

* ui:  create evaluation actor div

* fix related evaluations schema

* ui:  register/deregister evaluation divs

* ui:  handle resize behavior

* bug:  infinite re-render cycle

* fix:  conditional logic to prevent infinite render of flex resizing

* ui: related evaluations schema and request param

* ui: fix testing for evaluations

* refact: make related-evals a proper has-many

* chore: don't pauseTest

* temp:  debug d3 hierarchy

* ui:  move derived state logic into backing component class for detail

* ui:  deprecated related evaluations logic in statechart

* ui:  update evaluation models

* ui:  update logic to paint svg in non-viewable scroll region

* ui:  update styling

* ui:  testing for eval detail view

* ui:  delete detail from template directory

* ui:  break detail component down

* ui:  static data for /evaluation/:id endpoint

* ui:  fix styling of d3 viz

* ui:  add query parameter adapter for evals

* ui:  last minute design requests

* wip:  address browser updating detail view behavior

* refact: handle query-state change in statechart

* conditional class looking for currentEval equality (#12411)

* F UI/evaluation detail sidebar rel evals (#12415)

* ui:  remove busy id alias from statechart

* ui: edit related evaluations viz error message

* ui:  bug fixes on related evaluations view (#12423)

* ui:  remove busy id alias from statechart

* ui: edit related evaluations viz error message

* ui:  update error state

* ui:  related evaluation outline styling

* Related evaluation stylefile and non-link if it matches the active sidebar (#12428)

* Adds tabbable and keyboard pressable evaluation table rows (#12433)

* ui:  fix failing eval list tests (#12437)

* ui:  move styling into classes (#12438)

* fix test failures (#12444)

* ui:  move styling into classes

* ui:  eslint disable

* ui:  allocations have evaluations as async relationships

* ui:  fix evaluation refresh button (#12447)

* ui:  move styling into classes

* ui:  eslint disable

* ui:  allocations have evaluations as async relationships

* ui:  refresh bug

* ui:  final touches on sidebar (#12462)

* chore: turn off template linting rules

Temporarily turning off template linting because we dont have a set CSS convention and the release needs to go out ASAP.

* doc:  deprecate out of date comments and vars

* ui:  edit mirage server fetch logic

* ui:  style sidebar relative

* Modification to mocked related evals and manually set 100% height on svg (#12460)

* F UI/evaluation detail sidebar final touches (#12463)

* chore: turn off template linting rules

Temporarily turning off template linting because we dont have a set CSS convention and the release needs to go out ASAP.

* doc:  deprecate out of date comments and vars

* ui:  edit mirage server fetch logic

* ui:  style sidebar relative

* ui:  account for new related eval added to chain

Co-authored-by: Michael Klein <michael@firstiwaslike.com>
Co-authored-by: Phil Renaud <phil@riotindustries.com>
2022-04-05 14:34:37 -04:00
Luiz Aoqui ab7eb5de6e
Support Vault entity aliases (#12449)
Move some common Vault API data struct decoding out of the Vault client
so it can be reused in other situations.

Make Vault job validation its own function so it's easier to expand it.

Rename the `Job.VaultPolicies` method to just `Job.Vault` since it
returns the full Vault block, not just their policies.

Set `ChangeMode` on `Vault.Canonicalize`.

Add some missing tests.

Allows specifying an entity alias that will be used by Nomad when
deriving the task Vault token.

An entity alias assigns an indentity to a token, allowing better control
and management of Vault clients since all tokens with the same indentity
alias will now be considered the same client. This helps track Nomad
activity in Vault's audit logs and better control over Vault billing.

Add support for a new Nomad server configuration to define a default
entity alias to be used when deriving Vault tokens. This default value
will be used if the task doesn't have an entity alias defined.
2022-04-05 14:18:10 -04:00
Tim Gross b5d5393718
CSI: don't block client shutdown for node unmount (#12457)
When we unmount a volume we need to be able to recover from cases
where the plugin has been shutdown before the allocation that needs
it, so in #11892 we blocked shutting down the alloc runner hook. But
this blocks client shutdown if we're in the middle of unmounting. The
client won't be able to communicate with the plugin or send the
unpublish RPC anyways, so we should cancel the context and assume that
we'll resume the unmounting process when the client restarts.

For `-dev` mode we don't send the graceful `Shutdown()` method and
instead destroy all the allocations. In this case, we'll never be able
to communicate with the plugin but also never close the context we
need to prevent the hook from blocking. To fix this, move the retries
into their own goroutine that doesn't block the main `Postrun`.
2022-04-05 13:05:10 -04:00
James Rasell e2b730d7c9
Merge pull request #12454 from hashicorp/f-rename-service-event-stream
events: add service API logic and rename topic to service from serviceregistration
2022-04-05 16:19:14 +02:00
Grant Griffiths 18a0a2c9a4
CSI: Add secrets flag support for delete volume (#11245) 2022-04-05 08:59:11 -04:00
James Rasell ede2966c8d
events: add API helpers for service events stream topics. 2022-04-05 08:26:02 +01:00
James Rasell cc7b448d63
events: fixup service events and rename topic to service. 2022-04-05 08:25:22 +01:00
Danish Prakash e7e8ce212e
command/operator_debug: add pprof interval (#11938) 2022-04-04 15:24:12 -04:00
Michael Schurter a1f1294dc4
Merge pull request #12442 from hashicorp/f-sd-add-mixed-auth-read-endpoints
service-disco: add mixed auth to list and read RPC endpoints.
2022-04-04 12:19:29 -07:00
Tim Gross 759310d13a
CSI: volume watcher shutdown fixes (#12439)
The volume watcher design was based on deploymentwatcher and drainer,
but has an important difference: we don't want to maintain a goroutine
for the lifetime of the volume. So we stop the volumewatcher goroutine
for a volume when that volume has no more claims to free. But the
shutdown races with updates on the parent goroutine, and it's possible
to drop updates. Fortunately these updates are picked up on the next
core GC job, but we're most likely to hit this race when we're
replacing an allocation and that's the time we least want to wait.

Wait until the volume has "settled" before stopping this goroutine so
that the race between shutdown and the parent goroutine sending on
`<-updateCh` is pushed to after the window we most care about quick
freeing of claims.

* Fixes a resource leak when volumewatchers are no longer needed. The
  volume is nil and can't ever be started again, so the volume's
  `watcher` should be removed from the top-level `Watcher`.

* De-flakes the GC job test: the test throws an error because the
  claimed node doesn't exist and is unreachable. This flaked instead of
  failed because we didn't correctly wait for the first pass through the
  volumewatcher.

  Make the GC job wait for the volumewatcher to reach the quiescent
  timeout window state before running the GC eval under test, so that
  we're sure the GC job's work isn't being picked up by processing one
  of the earlier claims. Update the claims used so that we're sure the
  GC pass won't hit a node unpublish error.

* Adds trace logging to unpublish operations
2022-04-04 10:46:45 -04:00
Seth Hoenig bdc9799858
Merge pull request #12403 from hashicorp/dependabot/go_modules/github.com/creack/pty-1.1.18
build(deps): bump github.com/creack/pty from 1.1.17 to 1.1.18
2022-04-04 09:43:09 -05:00
dependabot[bot] 3c5bc49329
build(deps): bump github.com/creack/pty from 1.1.17 to 1.1.18
Bumps [github.com/creack/pty](https://github.com/creack/pty) from 1.1.17 to 1.1.18.
- [Release notes](https://github.com/creack/pty/releases)
- [Commits](https://github.com/creack/pty/compare/v1.1.17...v1.1.18)

---
updated-dependencies:
- dependency-name: github.com/creack/pty
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-04 14:25:02 +00:00
Seth Hoenig 3ce4f52740
Merge pull request #12446 from shoenig/no-pkg-err
cleanup: purge github.com/pkg/errors
2022-04-04 09:22:44 -05:00
Tim Gross 806a82dd0c
E2E: ensure that CSI EBS tests are isolated from each other (#12443)
Tear down the volume-consuming job between subtests, rather than after
all the tests are complete. For good measure, use a different ID for
the volume-consuming job as well.
2022-04-04 09:44:55 -04:00
James Rasell ea4d7366fc
service-disco: add mixed auth to list and read RPC endpoints.
In the same manner as the delete RPC, the list and read service
registration endpoints can be called either by external operators
or Nomad nodes. The latter occurs when a template is being
rendered which includes Nomad API template funcs. In this case,
the auth token is looked up as the node secret ID for auth.
2022-04-04 13:45:43 +01:00
James Rasell 19281bb2fe
Merge pull request #12304 from th0m/tlefebvre/fix-wrong-drivernetworkmanager-interface
fix: update incorrect DriverNetworkManager interface implementation
2022-04-04 11:29:22 +02:00
Seth Hoenig 9670adb6c6 cleanup: purge github.com/pkg/errors 2022-04-01 19:24:02 -05:00
Tim Gross de191e8068
Test lint touchup (#12434)
* lint: require should not be aliased in core_sched_test
* lint: require should not be aliased in volumes_watcher_test
* testing: don't alias state package in core_sched_test
2022-04-01 15:17:58 -04:00
Seth Hoenig 1ba8213e9a
Merge pull request #12432 from hashicorp/ci-gha-ignore-subpaths
ci: correctly ignore subpaths in gha
2022-04-01 09:58:23 -05:00
Seth Hoenig f9b0ffafde
Merge pull request #12431 from hashicorp/docs-sysbatch-exists-typo
docs: fix typo in system batch description
2022-04-01 09:58:06 -05:00
Seth Hoenig 4381aa122f ci: correctly ignore subpaths in gha 2022-04-01 09:49:40 -05:00
Seth Hoenig e9eacb1153 docs: fix typo in system batch description 2022-04-01 09:46:03 -05:00
Bryce Kalow 9b0d77ae78
website: redirect /api to api-docs and update internal links (#12410) 2022-03-31 11:33:27 -05:00
Tim Gross 8dccc43c2f
docs: remove deprecated client options parameters docs (#12416)
The client configuration options for drivers have been deprecated
since 0.9. We haven't torn them out completely but because they're
deprecated it's been hard to guarantee correct behavior. Remove the
documentation so that users aren't misled about their viability.
2022-03-31 11:45:51 -04:00
Seth Hoenig ebf69f93be
Merge pull request #12417 from hashicorp/tests-remove-08-groups-services
tests: remove update 08 groups services test
2022-03-31 10:40:50 -05:00
Seth Hoenig 8dcf01e94b tests: remove update 08 groups services test
This is a test around upgrading from Nomad 0.8, which is long since
no longer supported. The test is slow, flaky, and imports consul/sdk.

Remove this test as it is no longer relevant.
2022-03-31 10:14:22 -05:00
Seth Hoenig 8fe9e3c084
Merge pull request #12414 from hashicorp/tests-docker-dns-sadness
tests: create fresh harness for each docker dns test
2022-03-31 10:05:30 -05:00
Seth Hoenig 174a7532a1 tests: create fresh harness for each docker dns test
Not actually sure this fixes the flaky tests, but seems
like it could be related.
2022-03-31 08:17:34 -05:00
Seth Hoenig a897ad63aa
Merge pull request #12404 from hashicorp/tests-client-waits
tests: wait on client in a couple of tests
2022-03-30 15:23:47 -05:00
Seth Hoenig fa0dc05b7a tests: wait on client in a couple of tests
These tend to fail on GHA, where I believe the client is not
starting up fast enough before making requests. So wait on
the client agent first.

```
=== RUN   TestDebug_CapturedFiles
    operator_debug_test.go:422: serverName: TestDebug_CapturedFiles.global, clientID, 1afb00e6-13f2-d8d6-d0f9-745a3fd6e8e4
    operator_debug_test.go:492:
        	Error Trace:	operator_debug_test.go:492
        	Error:      	Should be empty, but was No node(s) with prefix "1afb00e6-13f2-d8d6-d0f9-745a3fd6e8e4" found
        	            	Failed to retrieve clients, 0 nodes found in list: 1afb00e6-13f2-d8d6-d0f9-745a3fd6e8e4
        	Test:       	TestDebug_CapturedFiles
--- FAIL: TestDebug_CapturedFiles (0.08s)
```
2022-03-30 08:48:23 -05:00
Seth Hoenig 61bf8022df
Merge pull request #12405 from hashicorp/ci-format-release-metadata-file
ci: hcl format release metadata file
2022-03-30 08:13:15 -05:00
Seth Hoenig d672bc46fd ci: add trailing newline to release metadata 2022-03-30 08:12:55 -05:00