Commit graph

24789 commits

Author SHA1 Message Date
Luiz Aoqui aa1b33d157
node pools: add event stream support (#17412) 2023-06-06 10:14:47 -04:00
Dao Thanh Tung 7c7f2d00bb
Add check for missing path in client host_volume config (#17393) 2023-06-05 19:31:19 -04:00
Tim Gross 2d16ec6c6f
node pools: implement RPC to list jobs in a given node pool (#17396)
Implements the `NodePool.ListJobs` RPC, with pagination and filtering based on
the existing `Job.List` RPC.
2023-06-05 15:36:52 -04:00
Seth Hoenig d1d4d22f8e
test: ensure cpuset cgroup is setup before fingerprinting (#17428)
This PR fixes a racey test where we need to ensure the cpuset cgroup
is setup before trying to fingerprint it.
2023-06-05 14:15:00 -05:00
Luiz Aoqui 700168e136
node pools: fix node upsert and state mutation tests (#17430) 2023-06-05 14:58:32 -04:00
Phil Renaud f348121ec7
[ui] Remove Ember Assets Github Actions workflow (#17426)
* Remove Ember Assets gha workflow

* PR write added to permissions
2023-06-05 13:52:20 -04:00
hashicorp-copywrite[bot] 0f4532f138
[COMPLIANCE] Add Copyright and License Headers (#17429)
Co-authored-by: hashicorp-copywrite[bot] <110428419+hashicorp-copywrite[bot]@users.noreply.github.com>
2023-06-05 13:23:59 -04:00
dependabot[bot] 2f4fe019db
build(deps): bump go.etcd.io/bbolt from 1.3.6 to 1.3.7 (#16228)
* build(deps): bump go.etcd.io/bbolt from 1.3.6 to 1.3.7

Bumps [go.etcd.io/bbolt](https://github.com/etcd-io/bbolt) from 1.3.6 to 1.3.7.
- [Release notes](https://github.com/etcd-io/bbolt/releases)
- [Commits](https://github.com/etcd-io/bbolt/compare/v1.3.6...v1.3.7)

---
updated-dependencies:
- dependency-name: go.etcd.io/bbolt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* cl: update cl for bbolt

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Seth Hoenig <shoenig@duck.com>
2023-06-05 10:19:14 -05:00
dependabot[bot] b83a26a8d8
build(deps): bump github.com/dustin/go-humanize from 1.0.0 to 1.0.1 (#16227)
Bumps [github.com/dustin/go-humanize](https://github.com/dustin/go-humanize) from 1.0.0 to 1.0.1.
- [Release notes](https://github.com/dustin/go-humanize/releases)
- [Commits](https://github.com/dustin/go-humanize/compare/v1.0.0...v1.0.1)

---
updated-dependencies:
- dependency-name: github.com/dustin/go-humanize
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-05 10:17:04 -05:00
dependabot[bot] c585cc68db
build(deps): bump github.com/hashicorp/raft from 1.3.11 to 1.5.0 (#17421)
* build(deps): bump github.com/hashicorp/raft from 1.3.11 to 1.5.0

Bumps [github.com/hashicorp/raft](https://github.com/hashicorp/raft) from 1.3.11 to 1.5.0.
- [Release notes](https://github.com/hashicorp/raft/releases)
- [Changelog](https://github.com/hashicorp/raft/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/raft/compare/v1.3.11...v1.5.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/raft
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* cl: add cl for raft 1.5.0

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Seth Hoenig <shoenig@duck.com>
2023-06-05 09:03:02 -05:00
dependabot[bot] ff4c2e2ea0
build(deps): bump google.golang.org/protobuf from 1.28.1 to 1.30.0 (#17420)
Bumps google.golang.org/protobuf from 1.28.1 to 1.30.0.

---
updated-dependencies:
- dependency-name: google.golang.org/protobuf
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-05 08:57:33 -05:00
KamilCuk cc64281445
Add group_add docker option (#17313) 2023-06-02 20:26:01 -04:00
dependabot[bot] fd52020560
build(deps): bump github.com/shirou/gopsutil/v3 from 3.23.1 to 3.23.4 (#17338)
Bumps [github.com/shirou/gopsutil/v3](https://github.com/shirou/gopsutil) from 3.23.1 to 3.23.4.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v3.23.1...v3.23.4)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-02 19:30:59 -04:00
Luiz Aoqui 6039c18ab6
node pools: register a node in a node pool (#17405) 2023-06-02 17:50:50 -04:00
Luiz Aoqui b770f2b1ef
node pools: implement CLI (#17388) 2023-06-02 15:49:57 -04:00
hc-github-team-es-release-engineering 6758379e48
ci: finish migration from CCI to GHA (#17103)
namely, these workflows:
  test-e2e, test-ui, and test-windows

extra-curricularly, as part of the overall
migration effort company-wide, this also includes
some standardization such as:
 * explicit permissions:read on various workflows
 * pinned action version shas (per https://github.com/hashicorp/security-public-tsccr)
 * actionlint, which among other things runs
   shellcheck on GHA run steps

Co-authored-by: emilymianeil <eneil@hashicorp.com>
Co-authored-by: Daniel Kimsey <daniel.kimsey@hashicorp.com>
2023-06-02 14:35:55 -05:00
Daniel Bennett f7e316e9cd
tests: enable newer windows (#17401)
* "allow" (don't try to drop) linux capabilities
  in the docker test driver harness (see #15181)
* refactor to allow different busybox images
  since windows containers need to be the same
  version as the underlying OS, and we're
  moving from 2016 to 2019
* one docker test was flaky from apparently
  being a bit slower on windows, so add Wait()
2023-06-02 11:38:38 -05:00
Luiz Aoqui 3a962d07f8
np: fix node pool search permission check (#17400)
When checking if a token is allowed to query the search endpoints we
need to return an error if the search context includes `node_pool` and
the token doesn't have access to _any_ pool. This prevents returning an
empty list instead of a permission denied error.
2023-06-02 12:22:47 -04:00
Phil Renaud 03dc959c2e
UI GHA test changes re-implemented (#17399) 2023-06-02 11:59:08 -04:00
Samantha b92a782b6e
check: Add support for Consul field tls_server_name (#17334) 2023-06-02 10:19:12 -04:00
Tim Gross 56e9b944e8
node pools: validate pool exists on job registration (#17386)
Add a new job admission hook for node pools that enforces the pool exists on
registration. Also provide the skeleton function we need for Enterprise
enforcement functions we'll implement later.
2023-06-02 09:32:07 -04:00
Luiz Aoqui f755b9469f
core: refactor task validation (#17344)
Move all validations related to task fields to Task.Validate(). Prior to
this, some task validations were being done inside TaskGroup.Validate()
because they required access to some group values.

But similarly to how TaskGroup.Validate() tasks the job as parameter,
it's fair to expect the task to receive its group.
2023-06-01 19:26:42 -04:00
Luiz Aoqui 4be8d7c049
core: fix kill_timeout validation when progress_deadline is 0 (#17342) 2023-06-01 19:01:32 -04:00
Luiz Aoqui 9bb57c08e3
node pool: add search support (#17385) 2023-06-01 17:48:14 -04:00
Tim Gross 4f14fa0518
node pools: add node_pool field to job spec (#17379)
This changeset only adds the `node_pool` field to the jobspec, and ensures that
it gets picked up correctly as a change. Without the rest of the implementation
landed yet, the field will be ignored.
2023-06-01 16:08:55 -04:00
Luiz Aoqui c61e75f302
node pools: add CRUD API (#17384) 2023-06-01 15:55:49 -04:00
Bryce Kalow 9e4bb87536
Delete check-legacy-links-format.yml (#17380) 2023-06-01 13:41:17 -04:00
Luiz Aoqui 45b0391378
np: implement ACL for node pools (#17365) 2023-06-01 13:03:20 -04:00
Seth Hoenig 53ca8b9552
repo: block pushing to release branches in git hook (#17377) 2023-06-01 09:36:20 -05:00
Daniel Kimsey b703acf671
Merge pull request #17376 from hashicorp/revert-accidental-17103-commit-on-main
Revert "fixup: address review changes"
2023-06-01 09:22:30 -05:00
Daniel Kimsey ec59fd1d54 Revert "fixup: address review changes"
This reverts commit ba736e4521bb4bec14d21862895eabec14a9c2f9.

This was accidentally added by fat-fingered Admin push...
2023-06-01 08:58:07 -05:00
Daniel Kimsey 5b886165da fixup: address review changes 2023-06-01 08:48:42 -05:00
Tim Gross 06972fae0c
prioritized client updates (#17354)
The allocrunner sends several updates to the server during the early lifecycle
of an allocation and its tasks. Clients batch-up allocation updates every 200ms,
but experiments like the C2M challenge has shown that even with this batching,
servers can be overwhelmed with client updates during high volume
deployments. Benchmarking done in #9451 has shown that client updates can easily
represent ~70% of all Nomad Raft traffic.

Each allocation sends many updates during its lifetime, but only those that
change the `ClientStatus` field are critical for progressing a deployment or
kicking off a reschedule to recover from failures.

Add a priority to the client allocation sync and update the `syncTicker`
receiver so that we only send an update if there's a high priority update
waiting, or on every 5th tick. This means when there are no high priority
updates, the client will send updates at most every 1s instead of
200ms. Benchmarks have shown this can reduce overall Raft traffic by 10%, as
well as reduce client-to-server RPC traffic.

This changeset also switches from a channel-based collection of updates to a
shared buffer, so as to split batching from sending and prevent backpressure
onto the allocrunner when the RPC is slow. This doesn't have a major performance
benefit in the benchmarks but makes the implementation of the prioritized update
simpler.

Fixes: #9451
2023-05-31 15:34:16 -04:00
dependabot[bot] bdd4a55738
build(deps): bump github.com/elazarl/go-bindata-assetfs (#17339)
Bumps [github.com/elazarl/go-bindata-assetfs](https://github.com/elazarl/go-bindata-assetfs) from 1.0.1-0.20200509193318-234c15e7648f to 1.0.1.
- [Release notes](https://github.com/elazarl/go-bindata-assetfs/releases)
- [Changelog](https://github.com/elazarl/go-bindata-assetfs/blob/master/.goreleaser.yml)
- [Commits](https://github.com/elazarl/go-bindata-assetfs/commits/v1.0.1)

---
updated-dependencies:
- dependency-name: github.com/elazarl/go-bindata-assetfs
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-31 10:19:59 -04:00
Phil Renaud 52772ab0c0
Text type to password type input on profile sign-in page (#17345) 2023-05-30 16:58:34 -04:00
Phil Renaud 038d53c58f
Observe newlines when displaying variables (#17343) 2023-05-30 16:58:16 -04:00
Luiz Aoqui 6236cb8f82
cli: output errors when monitoring deployment (#17348) 2023-05-30 11:12:12 -04:00
Luiz Aoqui e236d6dedd
cli: fix panic on job restart (#17346)
When monitoring the replacement allocation, if the
`Allocations().Info()` request fails, the `alloc` variable is `nil`, so
it should not be read.
2023-05-30 11:08:49 -04:00
Luiz Aoqui bb2395031b
client: fix Consul version finterprint (#17349)
Consul v1.13.8 was released with a breaking change in the /v1/agent/self
endpoint version where a line break was being returned.

This caused the Nomad finterprint to fail because `NewVersion` errors on
parse.

This commit removes any extra space from the Consul version returned by
the API.
2023-05-30 11:07:57 -04:00
Seth Hoenig acfdf0f479
compliance: add headers with fixed copywrite tool (#17353)
Closes #17117
2023-05-30 09:20:32 -05:00
Piotr Kazmierczak cea48b24ee
fix: job canonicalization should set job priority to 50, not 0. (#17314)
Nomad API will reject jobs with priority set to 0.
2023-05-30 09:05:32 +02:00
Phil Renaud 294aa4bfe7
[ui] A few variables-ui-related bugfixes (#17319)
* A few variable-adding bugfixes

* Disable Delete button if only one KV is left, and remove entity warnings on Add More
2023-05-25 17:11:13 -04:00
Charlie Voiselle 86e04a4c6c
[core] nil check and error handling for client status in heartbeat responses (#17316)
Add a nil check to constructNodeServerInfoResponse to manage an apparent race between deregister and client heartbeats. Fixes #17310
2023-05-25 16:04:54 -04:00
Phil Renaud 98aa88c739
[ui, deployments] Show a "Latest Deployment Status" cell within the Job Status panel on steady service jobs (#17246)
* Failed or lost cell condensed

* Latest Deployment cell

* Stylistic changes and deploying state fixup

* Rewritten tooltip message and updated lost/failed tests

* failed-or-lost cell updates to job status panel acceptance tests
2023-05-25 14:20:48 -04:00
Seth Hoenig e04d8cf77b
docs: fixup example of readiness check (#17296)
A "readiness" check implies a failing healthcheck will not cause the
deployment of a service to stop - i.e. it is only used as a liveness
probe in the context of service discoverability.

Fix our docs example to reflect that a readiness check is created by
setting on_update to "ignore" (as opposed to "ignore_warnings").
2023-05-23 15:29:10 -05:00
Tim Gross b85a28b851
changelog entry for Vault SDK update (#17281) 2023-05-23 09:21:29 -04:00
dependabot[bot] deaf4c4240
build(deps): bump github.com/grpc-ecosystem/go-grpc-middleware (#16551)
Bumps [github.com/grpc-ecosystem/go-grpc-middleware](https://github.com/grpc-ecosystem/go-grpc-middleware) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/grpc-ecosystem/go-grpc-middleware/releases)
- [Commits](https://github.com/grpc-ecosystem/go-grpc-middleware/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/grpc-ecosystem/go-grpc-middleware
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-23 09:01:26 -04:00
dependabot[bot] 9ad9505ba0
build(deps): bump github.com/hashicorp/vault/sdk from 0.7.0 to 0.9.0 (#17031)
Bumps [github.com/hashicorp/vault/sdk](https://github.com/hashicorp/vault) from 0.7.0 to 0.9.0.
- [Release notes](https://github.com/hashicorp/vault/releases)
- [Changelog](https://github.com/hashicorp/vault/blob/main/CHANGELOG.md)
- [Commits](https://github.com/hashicorp/vault/compare/v0.7.0...v0.9.0)

---
updated-dependencies:
- dependency-name: github.com/hashicorp/vault/sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-23 08:40:32 -04:00
dependabot[bot] 74b55ba1ed
build(deps): bump github.com/docker/distribution (#17162)
Bumps [github.com/docker/distribution](https://github.com/docker/distribution) from 2.8.1+incompatible to 2.8.2+incompatible.
- [Release notes](https://github.com/docker/distribution/releases)
- [Commits](https://github.com/docker/distribution/compare/v2.8.1...v2.8.2)

---
updated-dependencies:
- dependency-name: github.com/docker/distribution
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-23 08:40:18 -04:00
Seth Hoenig 9e43d197f8
post 1.5.6 release (#17276)
* Generate files for 1.5.6 release

* Prepare for next release

* Merge release 1.5.6 files

* manual revert bindata_assetfs because the one on main is better

---------

Co-authored-by: hc-github-team-nomad-core <github-team-nomad-core@hashicorp.com>
2023-05-22 14:03:51 -05:00