Commit graph

24062 commits

Author SHA1 Message Date
Michele Degges 23030fa77b
[CI-only] Run builds on pushes to main and the release branches (#14630) 2022-12-12 08:48:39 -08:00
Jai 27a992f617
ui: move "add more" button in variable create and edit forms (#15434)
* refact: move conditional logic

* chore: prettify template

* test: update test for markup change

* Revert "chore: prettify template"

This reverts commit ff1e0f02eb7ea30ede2cf93e9646a339601acdbe.

* styles: add space to save button for consistency

* style:  add save selector for Submit button

* refact: conditionally render add button based on view

* Revert "test: update test for markup change"

This reverts commit 59318cde68b50aaf915be7cb9f7e332b7f0204c7.

* style: add more button wonkiness

* test: use data-test attr

* test: handle non-table view on create

* ui:  add reactive getter property to use in template

* style use grid instead of margin

Co-authored-by: Preston Bourne <preston.bourne@icloud.com>
2022-12-12 10:56:31 -05:00
Tim Gross 989d7d9fcf
csi: avoid a nil pointer when handling plugin events (#15518)
If a plugin crashes quickly enough, we can get into a situation where the
deregister function is called before it's ever registered. Safely handle the
resulting nil pointer in the dynamic registry by not emitting a plugin event,
but also update the plugin event handler to tolerate nil pointers in case we
wire it up elsewhere in the future.
2022-12-12 08:42:57 -05:00
Seth Hoenig be3f89b5f9
artifact: enable inheriting environment variables from client (#15514)
* artifact: enable inheriting environment variables from client

This PR adds client configuration for specifying environment variables that
should be inherited by the artifact sandbox process from the Nomad Client agent.

Most users should not need to set these values but the configuration is provided
to ensure backwards compatability. Configuration of go-getter should ideally be
done through the artifact block in a jobspec task.

e.g.

```hcl
client {
  artifact {
    set_environment_variables = "TMPDIR,GIT_SSH_OPTS"
  }
}
```

Closes #15498

* website: update set_environment_variables text to mention PATH
2022-12-09 15:46:07 -06:00
Michele Degges da5ee8731c
Use custom runners pinned to 20.04 for building nomad (#15490) 2022-12-09 13:15:02 -08:00
Piotr Kazmierczak db98e26375
bugfix: acl sso auth methods test failures (#15512)
This PR fixes unit test failures introduced in f4e89e2
2022-12-09 18:47:32 +01:00
Piotr Kazmierczak 08f50f7dbf
acl: make sure there is only one default Auth Method per type (#15504)
This PR adds a check that makes sure we don't insert a duplicate default ACL auth method for a given type.
2022-12-09 14:46:54 +01:00
Piotr Kazmierczak 9562662774
acl: SSO auth methods API documentation (#15475)
This PR provides documentation for the ACL Auth Methods API endpoints.

Co-authored-by: James Rasell <jrasell@users.noreply.github.com>
2022-12-09 09:47:31 +01:00
Michael Schurter c28c5ad2e8
docs: clarify rescheduling happens when tasks fail (#15485) 2022-12-08 12:58:26 -08:00
Seth Hoenig 825c5cc65e
artifact: add client toggle to disable filesystem isolation (#15503)
This PR adds the client config option for turning off filesystem isolation,
applicable on Linux systems where filesystem isolation is possible and
enabled by default.

```hcl
client{
  artifact {
    disable_filesystem_isolation = <bool:false>
  }
}
```

Closes #15496
2022-12-08 12:29:23 -06:00
Seth Hoenig 62df06c385
pointer: add Merge helper function for merging pointers (#15499)
This PR adds Merge() helper function for choosing which value of two pointers
to use during a larger merge operation.

If 'next' is not nil, use that value, otherwise use the 'previous' value.
2022-12-08 11:09:22 -06:00
Piotr Kazmierczak 1cb45630f0
acl: canonicalize ACL Auth Method object (#15492) 2022-12-08 14:05:46 +01:00
Seth Hoenig 51a2212d3d
client: sandbox go-getter subprocess with landlock (#15328)
* client: sandbox go-getter subprocess with landlock

This PR re-implements the getter package for artifact downloads as a subprocess.

Key changes include

On all platforms, run getter as a child process of the Nomad agent.
On Linux platforms running as root, run the child process as the nobody user.
On supporting Linux kernels, uses landlock for filesystem isolation (via go-landlock).
On all platforms, restrict environment variables of the child process to a static set.
notably TMP/TEMP now points within the allocation's task directory
kernel.landlock attribute is fingerprinted (version number or unavailable)
These changes make Nomad client more resilient against a faulty go-getter implementation that may panic, and more secure against bad actors attempting to use artifact downloads as a privilege escalation vector.

Adds new e2e/artifact suite for ensuring artifact downloading works.

TODO: Windows git test (need to modify the image, etc... followup PR)

* landlock: fixup items from cr

* cr: fixup tests and go.mod file
2022-12-07 16:02:25 -06:00
Piotr Kazmierczak 10f80f7d9a
bugfix: make sure streaming endpoints are only registered once (#15484)
Streaming RPCs should only be registered once, not on every RPC call, because they set keys in StreamingRpcRegistry.registry map. This PR fixes it by checking whether endpoints are already registered before calling .register() method. Fixes #15474

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2022-12-07 17:01:45 +01:00
Tim Gross e0fddee386
Pre forwarding authentication (#15417)
Upcoming work to instrument the rate of RPC requests by consumer (and eventually
rate limit) require that we authenticate a RPC request before forwarding. Add a
new top-level `Authenticate` method to the server and have it return an
`AuthenticatedIdentity` struct. RPC handlers will use the relevant fields of
this identity for performing authorization.

This changeset includes:
* The main implementation of `Authenticate`
* Provide a new RPC `ACL.WhoAmI` for debugging authentication. This endpoint
  returns the same `AuthenticatedIdentity` that will be used by RPC handlers. At
  some point we might want to give this an equivalent HTTP endpoint but I didn't
  want to add that to our public API until some of the other Workload Identity
  work is solidified, especially if we don't need it yet.
* A full coverage test of the `Authenticate` method. This sets up two server
  nodes with mTLS and ACLs, some tokens, and some allocations with workload
  identities.
* Wire up an example of using `Authenticate` in the `Namespace.Upsert` RPC and
  see how authorization happens after forwarding.
* A new semgrep rule for `Authenticate`, which we'll need to update once we're
  ready to wire up more RPC endpoints with authorization steps.
2022-12-06 14:44:03 -05:00
Phil Renaud ce0ffdd077
[ui] Policies UI (#13976)
Co-authored-by: Mike Nomitch <mail@mikenomitch.com>
2022-12-06 12:45:36 -05:00
Piotr Kazmierczak d3aac1fe08
acl: sso auth method snapshot restore test (#15482) 2022-12-06 15:15:29 +01:00
Piotr Kazmierczak 777173e8da
acl: added type to ACL Auth Method stub (#15480) 2022-12-06 14:47:05 +01:00
Piotr Kazmierczak 9c3f04b488
bugfix: corrected indentation for ACL auth method create CLI command (#15481) 2022-12-06 14:45:24 +01:00
Seth Hoenig 3ed37b0b1d
fingerprint: add fingerprinting for CNI plugins presense and version (#15452)
This PR adds a fingerprinter to set the attribute
"plugins.cni.version.<name>" => "<version>"

for each CNI plugin in <client>.cni_path (/opt/cni/bin by default).
2022-12-05 14:22:47 -06:00
Phil Renaud 541ca94576
[ui] Adding canary_tags the web UI (#15458)
* Adding canary_tags to anyplace we show service tags

* CSS moved and tabs to spaces
2022-12-05 14:50:17 -05:00
Phil Renaud df749ff54a
Add namespaces to exec window (#15454) 2022-12-02 15:38:01 -05:00
Preston Bourne cd7dcb1787
style: Remove "is-primary" class from TaskRow (#15437)
* style: Remove "is-primary" class from TaskRow

* Revert "style: Remove "is-primary" class from TaskRow"

This reverts commit 86e94c4db1d817f9f6d5b205724ce2a943987001.

* style: stop muting text deco to comply w/ WCAG

Co-authored-by: Jai Bhagat <jaybhagat841@gmail.com>
2022-12-02 11:33:01 -05:00
Tim Gross dfed1ba5bc
remove most static RPC handlers (#15451)
Nomad server components that aren't in the `nomad` package like the deployment
watcher and volume watcher need to make RPC calls but can't import the Server
struct to do so because it creates a circular reference. These components have a
"shim" object that gets populated to pass a "static" handler that has no RPC
context.

Most RPC handlers are never used in this way, but during server setup we were
constructing a set of static handlers for most RPC endpoints anyways. This is
slightly wasteful but also confusing to developers who end up being encouraged
to just copy what was being done for previous RPCs.

This changeset includes the following refactorings:
* Remove the static handlers field on the server
* Instead construct just the specific static handlers we need to pass into the
  deployment watcher and volume watcher.
* Remove the unnecessary static handler from heartbeater
* Update various tests to avoid needing the static endpoints and have them use a
  endpoint constructed on the spot.

Follow-up work will examine whether we can remove the RPCs from deployment
watcher and volume watcher entirely, falling back to raft applies like node
drainer does currently.
2022-12-02 10:12:05 -05:00
HashiBot 8dbe510d65
chore: Update Digital Team Files (#15453)
* Update generated scripts (website-start.sh)

* Update generated scripts (website-build.sh)

* Update generated scripts (should-build.sh)

* Update generated website Makefile
2022-12-02 09:55:29 -05:00
Tim Gross 7404ef46e9
docs: update plugin status docs with capabilities and topology (#15448)
The `plugin status` command supports displaying CSI capabilities and topology
accessibility, but this was missing from the documentation. Extend the
`-verbose` example to show that info.
2022-12-01 12:18:56 -05:00
Jai 87a3bd5393
bug: hitting refresh on /jobs/:jobId/services/:serviceId doesn't display data (#15431)
* ui: create id helper

* refact: handle async relationships

* chore: prettify template

* ui: helper async escape hatch

* refact: use escape hatch for tooltip
2022-12-01 10:45:31 -05:00
Tim Gross 33f32d526e
fix enterprise endpoint registration (#15446)
In #15430 we refactored the RPC endpoint configuration to make adding the RPC
context easier. But when implementing the change on the Enterprise side, I
discovered that the registration of enterprise endpoints was being done
incorrectly -- this doesn't show up on OSS because the registration is always a
no-op here.
2022-12-01 10:44:33 -05:00
Jai bb0debf2e5
refact: order search before sort (#15432) 2022-12-01 10:43:47 -05:00
Tim Gross f61f801e77
provide RPCContext to all RPC handlers (#15430)
Upcoming work to instrument the rate of RPC requests by consumer (and eventually
rate limit) requires that we thread the `RPCContext` through all RPC
handlers so that we can access the underlying connection. This changeset adds
the context to everywhere we intend to initially support it and intentionally
excludes streaming RPCs and client RPCs.

To improve the ergonomics of adding the context everywhere its needed and to
clarify the requirements of dynamic vs static handlers, I've also done a good
bit of refactoring here:

* canonicalized the RPC handler fields so they're as close to identical as
  possible without introducing unused fields (i.e. I didn't add loggers if the
  handler doesn't use them already).
* canonicalized the imports in the handler files.
* added a `NewExampleEndpoint` function for each handler that ensures we're
  constructing the handlers with the required arguments.
* reordered the registration in server.go to match the order of the files (to
  make it easier to see if we've missed one), and added a bunch of commentary
  there as to what the difference between static and dynamic handlers is.
2022-12-01 10:05:15 -05:00
Seth Hoenig 119f7b1cd1
consul: fixup expected consul tagged_addresses when using ipv6 (#15411)
This PR is a continuation of #14917, where we missed the ipv6 cases.

Consul auto-inserts tagged_addresses for keys
- lan_ipv4
- wan_ipv4
- lan_ipv6
- wan_ipv6

even though the service registration coming from Nomad does not contain such
elements. When doing the differential between services Nomad expects to be
registered vs. the services actually registered into Consul, we must first
purge these automatically inserted tagged_addresses if they do not exist in
the Nomad view of the Consul service.
2022-12-01 07:38:30 -06:00
Matus Goljer 2283c2d583
Update affinity.mdx (#15168)
Fix the comment to correspond to the code
2022-11-30 19:01:56 -05:00
Phil Renaud 0d22ea0cff
Percy-specific CSS to block out the exact token expiry time, which would mess w vis diff tests (#15422) 2022-11-30 14:33:07 -05:00
Luiz Aoqui c6ae5d95ac
docs: clarify autoscaling factor and threshold for target-value plugin (#15418) 2022-11-30 10:56:16 -05:00
dependabot[bot] 944a7dbb70
build(deps): bump google.golang.org/grpc from 1.50.1 to 1.51.0 (#15402)
* build(deps): bump google.golang.org/grpc from 1.50.1 to 1.51.0

Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.50.1 to 1.51.0.
- [Release notes](https://github.com/grpc/grpc-go/releases)
- [Commits](https://github.com/grpc/grpc-go/compare/v1.50.1...v1.51.0)

---
updated-dependencies:
- dependency-name: google.golang.org/grpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* changelog: add entry for #15402

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Luiz Aoqui <luiz@hashicorp.com>
2022-11-29 14:55:17 -05:00
dependabot[bot] 450b7383fe
build(deps): bump github.com/hashicorp/consul/sdk from 0.11.0 to 0.13.0 (#15401)
Bumps [github.com/hashicorp/consul/sdk](https://github.com/hashicorp/consul) from 0.11.0 to 0.13.0.
- [Release notes](https://github.com/hashicorp/consul/releases)
- [Changelog](https://github.com/hashicorp/consul/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/consul/compare/sdk/v0.11.0...sdk/v0.13.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/consul/sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-29 14:26:48 -05:00
dependabot[bot] 4c98934d94
build(deps): bump github.com/mitchellh/cli from 1.1.4 to 1.1.5 (#15400)
Bumps [github.com/mitchellh/cli](https://github.com/mitchellh/cli) from 1.1.4 to 1.1.5.
- [Release notes](https://github.com/mitchellh/cli/releases)
- [Commits](https://github.com/mitchellh/cli/compare/v1.1.4...v1.1.5)

---
updated-dependencies:
- dependency-name: github.com/mitchellh/cli
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-29 14:00:46 -05:00
dependabot[bot] 1bb781ae0c
build(deps): bump github.com/zclconf/go-cty from 1.12.0 to 1.12.1 (#15403)
Bumps [github.com/zclconf/go-cty](https://github.com/zclconf/go-cty) from 1.12.0 to 1.12.1.
- [Release notes](https://github.com/zclconf/go-cty/releases)
- [Changelog](https://github.com/zclconf/go-cty/blob/main/CHANGELOG.md)
- [Commits](https://github.com/zclconf/go-cty/compare/v1.12.0...v1.12.1)

---
updated-dependencies:
- dependency-name: github.com/zclconf/go-cty
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-29 13:45:58 -05:00
dependabot[bot] 572e53d12b
build(deps): bump github.com/hashicorp/vault/api from 1.8.1 to 1.8.2 (#15404)
Bumps [github.com/hashicorp/vault/api](https://github.com/hashicorp/vault) from 1.8.1 to 1.8.2.
- [Release notes](https://github.com/hashicorp/vault/releases)
- [Changelog](https://github.com/hashicorp/vault/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/vault/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/vault/api
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-11-29 13:28:46 -05:00
dependabot[bot] 866f7da8ad
build(deps): bump github.com/shoenig/test from 0.4.4 to 0.4.5 in /api (#15405)
* build(deps): bump github.com/shoenig/test from 0.4.4 to 0.4.5 in /api

Bumps [github.com/shoenig/test](https://github.com/shoenig/test) from 0.4.4 to 0.4.5.
- [Release notes](https://github.com/shoenig/test/releases)
- [Commits](https://github.com/shoenig/test/compare/v0.4.4...v0.4.5)

---
updated-dependencies:
- dependency-name: github.com/shoenig/test
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* deps: update github.com/shoenig/test v0.4.4 -> v0.4.5

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Luiz Aoqui <luiz@hashicorp.com>
2022-11-29 13:25:53 -05:00
Piotr Kazmierczak 0eccd3286c
acl: sso auth methods RPC/API/CLI should return created or updated objects (#15410)
Currently CRUD code that operates on SSO auth methods does not return created or updated object upon creation/update. This is bad UX and inconsistent behavior compared to other ACL objects like roles, policies or tokens.

This PR fixes it.

Relates to #13120
2022-11-29 07:36:36 +01:00
Seth Hoenig a65fbeb3b3
client: manually cleanup leaked iptables rules (#15407)
This PR adds a secondary path for cleaning up iptables created for an allocation
when the normal CNI library fails to do so. This typically happens when the state
of the pause container is unexpected - e.g. deleted out of band from Nomad. Before,
the iptables rules would be leaked which could lead to unexpected nat routing
behavior later on (in addition to leaked resources). With this change, we scan
for the rules created on behalf of the allocation being GC'd and delete them.

Fixes #6385
2022-11-28 11:32:16 -06:00
Jai 4c0752fc87
ui: html escaping (#15409) 2022-11-28 12:26:18 -05:00
Phil Renaud ffd16dfec6
[ui, epic] SSO and Auth improvements (#15110)
* Top nav auth dropdown (#15055)

* Basic dropdown styles

* Some cleanup

* delog

* Default nomad hover state styles

* Component separation-of-concerns and acceptance tests for auth dropdown

* lintfix

* [ui, sso] Handle token expiry 500s (#15073)

* Handle error states generally

* Dont direct, just redirect

* no longer need explicit error on controller

* Redirect on token-doesnt-exist

* Forgot to import our time lib

* Linting on _blank

* Redirect tests

* changelog

* [ui, sso] warn user about pending token expiry (#15091)

* Handle error states generally

* Dont direct, just redirect

* no longer need explicit error on controller

* Linting on _blank

* Custom notification actions and shift the template to within an else block

* Lintfix

* Make the closeAction optional

* changelog

* Add a mirage token that will always expire in 11 minutes

* Test for token expiry with ember concurrency waiters

* concurrency handling for earlier test, and button redirect test

* [ui] if ACLs are disabled, remove the Sign In link from the top of the UI (#15114)

* Remove top nav link if ACLs disabled

* Change to an enabled-by-default model since you get no agent config when ACLs are disabled but you lack a token

* PR feedback addressed; down with double negative conditionals

* lintfix

* ember getter instead of ?.prop

* [SSO] Auth Methods and Mock OIDC Flow (#15155)

* Big ol first pass at a redirect sign in flow

* dont recursively add queryparams on redirect

* Passing state and code qps

* In which I go off the deep end and embed a faux provider page in the nomad ui

* Buggy but self-contained flow

* Flow auto-delay added and a little more polish to resetting token

* secret passing turned to accessor passing

* Handle SSO Failure

* General cleanup and test fix

* Lintfix

* SSO flow acceptance tests

* Percy snapshots added

* Explicitly note the OIDC test route is mirage only

* Handling failure case for complete-auth

* Leentfeex

* Tokens page styles (#15273)

* styling and moving columns around

* autofocus and enter press handling

* Styles refined

* Split up manager and regular tests

* Standardizing to a binary status state

* Serialize auth-methods response to use "name" as primary key (#15380)

* Serializer for unique-by-name

* Use @classic because of class extension
2022-11-28 10:44:52 -05:00
Phil Renaud 752955e876
Recompute Y Axis on data change (#15381) 2022-11-28 09:56:47 -05:00
Piotr Kazmierczak db9316c4d3
acl: sso auth methods cli commands (#15322)
This PR implements CLI commands to interact with SSO auth methods.

This PR is part of the SSO work captured under ☂️ ticket #13120.
2022-11-28 10:51:45 +01:00
James Rasell 726d419da1
acl: replicate auth-methods from federated cluster leaders. (#15366) 2022-11-28 09:20:24 +01:00
Luiz Aoqui 5995ea9981
docs: improve job parse API documentation (#15387) 2022-11-25 12:46:53 -05:00
Luiz Aoqui 8f91be26ab
scheduler: create placements for non-register MRD (#15325)
* scheduler: create placements for non-register MRD

For multiregion jobs, the scheduler does not create placements on
registration because the deployment must wait for the other regions.
Once of these regions will then trigger the deployment to run.

Currently, this is done in the scheduler by considering any eval for a
multiregion job as "paused" since it's expected that another region will
eventually unpause it.

This becomes a problem where evals not triggered by a job registration
happen, such as on a node update. These types of regional changes do not
have other regions waiting to progress the deployment, and so they were
never resulting in placements.

The fix is to create a deployment at job registration time. This
additional piece of state allows the scheduler to differentiate between
a multiregion change, where there are other regions engaged in the
deployment so no placements are required, from a regional change, where
the scheduler does need to create placements.

This deployment starts in the new "initializing" status to signal to the
scheduler that it needs to compute the initial deployment state. The
multiregion deployment will wait until this deployment state is
persisted and its starts is set to "pending". Without this state
transition it's possible to hit a race condition where the plan applier
and the deployment watcher may step of each other and overwrite their
changes.

* changelog: add entry for #15325
2022-11-25 12:45:34 -05:00
Piotr Kazmierczak 9c85315bd2
bugfix: typos in acl role commands (#15382)
Co-authored-by: James Rasell <jrasell@users.noreply.github.com>
2022-11-25 10:28:33 +01:00