Commit graph

23758 commits

Author SHA1 Message Date
James Rasell 3fa8b0b270
client: fix RPC forwarding when querying checks for alloc. (#14498)
When querying the checks for an allocation, the request must be
forwarded to the agent that is running the allocation. If the
initial request is made to a server agent, the request can be made
directly to the client agent running the allocation. If the
request is made to a client agent not running the alloc, the
request needs to be forwarded to a server and then the correct
client.
2022-09-08 16:55:23 +02:00
Seth Hoenig d79ea80603
Merge pull request #14497 from hashicorp/b-guard-random-stagger
helper: guard against negative inputs into random stagger
2022-09-08 09:44:56 -05:00
Tim Gross 2a961af44c
test: fix concurrent map access in TestStatsFetcher (#14496)
The map of in-flight RPCs gets cleared by a goroutine in the test without first
locking it to make sure that it's not being accessed concurrently by the stats
fetcher itself. This can cause a panic in tests.
2022-09-08 10:41:15 -04:00
Seth Hoenig a608e7950e helper: guard against negative inputs into random stagger
This PR modifies RandomStagger to protect against negative input
values. If the given interval is negative, the value returned will
be somewhere in the stratosphere. Instead, treat negative inputs
like zero, returning zero.
2022-09-08 09:17:48 -05:00
James Rasell d14d6e051a
e2e: fixup token expiration test to account for longer forced GC. (#14491) 2022-09-08 14:43:04 +02:00
Michael Schurter 7ff0290f8b
docs: add quota panic fix changelog entry (#14485)
See https://github.com/hashicorp/nomad-enterprise/pull/839 for original
(Enterprise only)
2022-09-07 17:04:46 -07:00
Phil Renaud 7c76ca95ed Midway point on only showing alloc-specific checks 2022-09-07 15:35:58 -04:00
Phil Renaud 59cf87ca6f
Merge pull request #14408 from hashicorp/f-ui/service-discovery
Service discovery in the Nomad UI
2022-09-07 13:40:09 -04:00
Phil Renaud c5d59a8937 Remove an unused internal test comment 2022-09-07 13:39:04 -04:00
Phil Renaud 6872626ac4 Further unused test deletion 2022-09-07 10:34:09 -04:00
Phil Renaud 52bb5de25a Changelog added and unused tests removed 2022-09-07 10:31:39 -04:00
Phil Renaud b6f6f4db37 [ui] Service Discovery: Allocation Service fly-out (#14389)
* Bones of a new flyout section

* Basic sidebar behaviour and style edits

* Concept of a refID for service fragments to disambiguate task and group

* A11y audit etc

* Moves health check aggregation to serviceFragment model and retains history

* Has to be a getter

* flyout populated

* Sidebar styling

* Sidebar table and details added

* Mirage fixture

* Active status and table styles

* Unit test mock updated

* Acceptance tests for alloc services table and flyout

* Chart styles closer to mock

* Without a paused test

* Consul and Nomad icons in services table

* Alloc services test updates in light of new column changes

* without using an inherited scenario
2022-09-07 10:24:34 -04:00
Jai b4a6019e4a ui: add to collection (#14472) 2022-09-07 10:24:34 -04:00
Phil Renaud b4c5a207e8 [ui] Jobs/jobid/services table sorting (#14410)
* Sortable job service table

* sorting removed from specific service page
2022-09-07 10:24:34 -04:00
Phil Renaud e03c5a43be Job Services: fixtures and acceptance tests (#14319)
* Added to subnav and basic table implemented

* Existing services become service fragments, and services tab aggregated beneath job route

* Index page within jobs/job/services

* Watchable services

* Lintfixes

* Links to clients and individual services set up

* Child service route

* Keyboard shortcuts on service page

* Model that shows consul services as well, plus level and provider cols

* lintfix

* Level as query param

* Watch job for service name changes too

* Group level service fixtures established

* Progress at task level and job-linked services

* Task and group services on update

* Fixture side-effect cleanup

* Basic acceptance tests for job services

* Testmodel cleanup

* Disabled mirage logging

* New cluster type specifically for services

* Without explicit job-model binding

* Trying to isolate a tostring error

* Account for new tab in keyboardnav

* More test isolation attempts

* Remove skipped tests and link task to parent group by id

ui: add service health viz to table (#14369)

* ui: add service-status-bar

* test: service-status-bar

* refact: update component api for new data struct

* ui: format service health struct

* ui:  add service health viz to table

* temp: add placeholder to remind conditional watcher

* test: write tests for transformation algorithm

* refact: update transformation algo

* ui: conditionally long poll checks endpoint

* refact: add conditional logic for nomad provider

refact: update service-fragment model to include owner info

ui: differentiate between task and group-level in derived state comp

test: add test to document behavior

refact: update tests for api change

refact: update integration test for API change

chore: remove unsused vars

chore: elvis operator to protect mirage

refact: create refId instead of internalModel

refact: update algo

refact: update conditional template logic

refact: update test for api change:

chore: cant use if and not in hbs conditional
2022-09-07 10:24:33 -04:00
Jai 632ef15dd2 ui: long poll /checks endpoint (#14354)
* chore: add lodash isEqual package

* ui: fetch non ember-data records

* ui: create watcher to poll non ember-data records
2022-09-07 10:24:33 -04:00
Phil Renaud ba5ae14b6c Nomad Services: job routes, model, and serializer updates (#14226)
* Added to subnav and basic table implemented

* Existing services become service fragments, and services tab aggregated beneath job route

* Index page within jobs/job/services

* Watchable services

* Lintfixes

* Links to clients and individual services set up

* Child service route

* Keyboard shortcuts on service page

* Model that shows consul services as well, plus level and provider cols

* lintfix

* Level as query param

* Watch job for service name changes too

* Lintfix

* Testfixes

* Placeholder mirage route
2022-09-07 10:24:33 -04:00
Tim Gross 5c57a84e99
autopilot: deflake tests (#14475)
Includes:

* Remove leader upgrade raft version test, as older versions of raft are now
  incompatible with our autopilot library.

* Remove attempt to assert initial non-voter status on the `PromoteNonVoter`
  test, as this happens too quickly to reliably detect.

* Unskip some previously-skipped tests which we should make stable.

* Remove the `consul/sdk` retry helper for these tests; this uses panic recovery
  in a kind of a clever/gross way to reduce LoC but it seems to introduce some
  timing issues in the process.

* Add more test step logging and reduce logging noise from the scheduler
  goroutines to make it easier to debug failing tests.

* Be more consistent about using the `waitForStableLeadership` helper so that we
  can assert the cluster is fully stable and not just that we've added peers.
2022-09-07 09:35:01 -04:00
Luiz Aoqui 358ba279d0
ui: remove extra space in menu footer (#14457) 2022-09-06 16:53:17 -04:00
James Rasell 813c5daa96
hcl2: add strlen function and update docs. (#14463) 2022-09-06 18:42:40 +02:00
James Rasell 962b1f78e8
core: clarify ACL token expiry GC messages to show global param. (#14466) 2022-09-06 15:42:45 +02:00
Tim Gross 6ff59e71a5
cli: remove network from quota status output (#14468)
Network quotas were removed in Nomad 1.0.4. Remove the fields no longer in use
from the `quota status` output.
2022-09-06 09:37:16 -04:00
Kellen Fox 5086368a1e
Add a log line to help track node eligibility (#14125)
Co-authored-by: James Rasell <jrasell@hashicorp.com>
2022-09-06 14:03:33 +02:00
Tony Truong 4040044dc7
Removing reference to non existant test-website task in makefile 2022-09-06 01:02:21 +02:00
Yan 6e927fa125
warn destructive update only when count > 1 (#13103) 2022-09-02 15:30:06 -04:00
Giovani Avelar b5cf358212
[ui] Show a different message when there are no tasks in a job (#14071)
Different mesage when there are not tasks in a job
2022-09-02 15:20:45 -04:00
Tiernan 98022376be
Fix error handling in Client consulDiscoveryImpl (#14431)
Added a missing `continue` on non-nil error to avoid accidentally using a bad peer.
2022-09-02 15:13:03 -04:00
Luiz Aoqui 1ae26981a0
connect: interpolate task env in config values (#14445)
When configuring Consul Service Mesh, it's sometimes necessary to
provide dynamic value that are only known to Nomad at runtime. By
interpolating configuration values (in addition to configuration keys),
user are able to pass these dynamic values to Consul from their Nomad
jobs.
2022-09-02 15:00:28 -04:00
James Rasell e24de517fa
e2e: add test to exercise ACL tokens with role and policy links. (#14432) 2022-09-02 08:56:00 +02:00
Luiz Aoqui 99bddfe04d
docs: add warning about changing region config (#14443) 2022-09-01 16:47:06 -04:00
Tim Gross 7921f044e5
migrate autopilot implementation to raft-autopilot (#14441)
Nomad's original autopilot was importing from a private package in Consul. It
has been moved out to a shared library. Switch Nomad to use this library so that
we can eliminate the import of Consul, which is necessary to build Nomad ENT
with the current version of the Consul SDK. This also will let us pick up
autopilot improvements shared with Consul more easily.
2022-09-01 14:27:10 -04:00
Luiz Aoqui 94d7dddccd
cli: set -hcl2-strict to false if -hcl1 is defined (#14426)
These options are mutually exclusive but, since `-hcl2-strict` defaults
to `true` users had to explicitily set it to `false` when using `-hcl1`.

Also return `255` when job plan fails validation as this is the expected 
code in this situation.
2022-09-01 10:42:08 -04:00
Tim Gross 0ef073a669
docs: clarify CSI plugin compatibility (#14434)
Nomad is generally compliant with the CSI specification for Container
Orchestrators (CO), except for unimplemented features. However, some storage
vendors have built CSI plugins that are not compliant with the specification or
which expect that they're only deployed on Kubernetes. Nomad cannot vouch for
the compatibility of any particular plugin, so clarify this in the docs.

Co-authored-by: Derek Strickland <1111455+DerekStrickland@users.noreply.github.com>
2022-09-01 10:06:44 -04:00
Luiz Aoqui 19de803503
cli: ignore VaultToken when generating job diff (#14424) 2022-09-01 10:01:53 -04:00
James Rasell 4b9bcf94da
chore: remove use of "err" a log line context key for errors. (#14433)
Log lines which include an error should use the full term "error"
as the context key. This provides consistency across the codebase
and avoids a Go style which operators might not be aware of.
2022-09-01 15:06:10 +02:00
dependabot[bot] 9f8a3824c4
build(deps): bump github.com/hashicorp/go-version from 1.4.0 to 1.6.0 (#14364)
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: James Rasell <jrasell@hashicorp.com>
2022-09-01 11:55:42 +02:00
James Rasell 5d0cc93939
e2e: add acl test for token expiration. (#14418)
In order to add an E2E test to cover token expiration, the server
config has been updated to include a low minimum allowed TTL
value. For ease of reading, the max value is also set.
2022-09-01 09:36:09 +02:00
Brett Larson 9912dfd1e6
Update ephemeral_disk.mdx (#14356)
It is really unclear on how to use this feature. it took me a while to find this, so I thought I would purpose how to use this.
2022-08-31 20:17:41 -04:00
Derek Strickland 35e91ff376
Merge release 1.3.5 files (#14425)
* Merge release 1.3.5 files

* Generate files for 1.3.5 release

* Prepare for next release

Co-authored-by: hc-github-team-nomad-core <github-team-nomad-core@hashicorp.com>
2022-08-31 18:31:56 -04:00
Luiz Aoqui 6f5d3e724f
changelog: add entry for #14374 (#14419) 2022-08-31 10:59:19 -04:00
Luiz Aoqui 27b253bc6e
changelog: add entry for #14381 (#14416) 2022-08-31 10:41:48 -04:00
James Rasell 986355bcd9
docs: add documentation for ACL token expiration and ACL roles. (#14332)
The ACL command docs are now found within a sub-dir like the
operator command docs. Updates to the ACL token commands to
accommodate token expiry have also been added.

The ACL API docs are now found within a sub-dir like the operator
API docs. The ACL docs now include the ACL roles endpoint as well
as updated ACL token endpoints for token expiration.

The configuration section is also updated to accommodate the new
ACL and server parameters for the new ACL features.
2022-08-31 16:13:47 +02:00
James Rasell 5f3665230b
e2e: add ACL test suite with ACL Role test. (#14398)
This adds a new ACL test suite to the e2e framework which includes
an initial test for ACL roles. The ACL test includes a helper to
track and clean created Nomad resources which keeps the test
cluster clean no matter if the test fails early or not.
2022-08-31 10:11:28 +02:00
Luiz Aoqui dc6525336b
ci: fix TestNomad_BootstrapExpect_NonVoter test (#14407)
PR #12130 refactored the test to use the `wantPeers` helper, but this
function only returns the number of voting peers, which in this test
should be equal to 2.

I think the tests were passing back them because of a bug in Raft
(https://github.com/hashicorp/raft/pull/483) where a non-voting server
was able to transition to candidate state.

One possible evidence of this is that a successful test run would have
the following log line:

```
raft@v1.3.5/raft.go:1058: nomad.raft: updating configuration: command=AddVoter server-id=127.0.0.1:9101 server-addr=127.0.0.1:9101 servers="[{Suffrage:Voter ID:127.0.0.1:9107 Address:127.0.0.1:9107} {Suffrage:Voter ID:127.0.0.1:9105 Address:127.0.0.1:9105} {Suffrage:Voter ID:127.0.0.1:9103 Address:127.0.0.1:9103} {Suffrage:Voter ID:127.0.0.1:9101 Address:127.0.0.1:9101}]"
```

This commit reverts the test logic to check for peer count, regardless
of voting status.
2022-08-30 16:32:54 -04:00
Charlie Voiselle 5c0e34dd33
Vars: Update CT dependency to support variables. (#14399)
* Update Consul Template dep to support Nomad vars

* Remove `Peering` config for Consul Testservers
Upgrading to the 1.14 Consul SDK introduces and additional default
configuration—`Peering`—that is not compatible with versions of Consul
before v1.13.0. because Nomad tests against Consul v1.11.1, this
configuration has to be nil'ed out before passing it to the Consul
binary.
2022-08-30 15:26:01 -04:00
Tim Gross cc9b480996
testing: setting env var incompatible with parallel tests (#14405)
Neither the `os.Setenv` nor `t.Setenv` helper are safe to use in parallel tests
because environment variables are process-global. The stdlib panics if you try
to do this. Remove the `ci.Parallel()` call from all tests where we're setting
environment variables.
2022-08-30 14:49:03 -04:00
Tim Gross 5784fb8c58
search: enforce correct ACL for search over variables (#14397) 2022-08-30 13:27:31 -04:00
Tim Gross c9d678a91a
keyring: wrap root key in key encryption key (#14388)
Update the on-disk format for the root key so that it's wrapped with a unique
per-key/per-server key encryption key. This is a bit of security theatre for the
current implementation, but it uses `go-kms-wrapping` as the interface for
wrapping the key. This provides a shim for future support of external KMS such
as cloud provider APIs or Vault transit encryption.

* Removes the JSON serialization extension we had on the `RootKey` struct; this
  struct is now only used for key replication and not for disk serialization, so
  we don't need this helper.

* Creates a helper for generating cryptographically random slices of bytes that
  properly accounts for short reads from the source.

* No observable functional changes outside of the on-disk format, so there are
  no test updates.
2022-08-30 10:59:25 -04:00
Tim Gross 37905d94b7
docs: fixing a few more places we missed "secure" during rename (#14395) 2022-08-30 10:08:50 -04:00
Seth Hoenig 14a7872150
Merge pull request #14385 from hashicorp/f-cg-use-kill
cgroups: refactor v2 kill path to use cgroups.kill interface file
2022-08-30 09:02:02 -05:00