Commit Graph

23206 Commits

Author SHA1 Message Date
Michael Schurter 2965dc6a1a
artifact: fix numerous go-getter security issues
Fix numerous go-getter security issues:

- Add timeouts to http, git, and hg operations to prevent DoS
- Add size limit to http to prevent resource exhaustion
- Disable following symlinks in both artifacts and `job run`
- Stop performing initial HEAD request to avoid file corruption on
  retries and DoS opportunities.

**Approach**

Since Nomad has no ability to differentiate a DoS-via-large-artifact vs
a legitimate workload, all of the new limits are configurable at the
client agent level.

The max size of HTTP downloads is also exposed as a node attribute so
that if some workloads have large artifacts they can specify a high
limit in their jobspecs.

In the future all of this plumbing could be extended to enable/disable
specific getters or artifact downloading entirely on a per-node basis.
2022-05-24 16:29:39 -04:00
Luiz Aoqui 0a00059f3c
core: test duplicated blocked eval stats
In the original test, the eval generator would use a random value for
the job ID, resulting in an unxercised code path for duplicate blocked
evals.
2022-05-24 15:44:06 -04:00
Seth Hoenig 83bab8ed64
Merge pull request #13058 from hashicorp/b-cgroupsv1-docker-cgparent
drivers/docker: do not set cgroup parent in v1 mode
2022-05-24 14:07:40 -05:00
Seth Hoenig c6c3ae020d drivers/docker: do not set cgroup parent in v1 mode
This PR fixes a bug where the CgroupParent on the docker
HostConfig struct was accidently being set when running in
cgroups v1 mode.
2022-05-24 11:22:50 -05:00
Seth Hoenig 27d0c0dc9f docs: add changelog 2022-05-24 09:13:15 -05:00
Seth Hoenig a5943da0c7 core: add tests for blocked evals math 2022-05-24 09:05:18 -05:00
Seth Hoenig 0c145ac1e4 core: remove correct set of resources on blocked eval 2022-05-23 15:18:55 -05:00
PinkLolicorn 83dd9e801e
docs: `mount_flags` takes a slice of strings (#13087)
The description of `mount_flags` provides incorrect example
of the accepted value format.

This fixes the issue by changing the example from a string
`ro,noatime` to a slice of strings `["ro", "noatime"]`.
2022-05-20 09:16:17 -04:00
Tim Gross cc4a1f2ec4
e2e: upgrade playwright package and container image (#13080)
The nightly playwright tests are currently failing because of a
mismatch between the expected version of Chromium and what's in the
container image. Unfortunately the previous specific tag we were using
for the container image is no longer tagged on the registry. With some
testing, I was able to find an image tag that results in a good run.
2022-05-20 08:41:07 -04:00
Jose Diaz-Gonzalez fa1077fbcd
docs: correct where task cannot be defined 2022-05-19 21:24:58 -04:00
Jose Diaz-Gonzalez ea01fe398f
Update service.check.task definition to match code
Nomad errors out when attempting to specify a task for a service that uses consul connect but does not have script or gRPC checks. See 304d0cf595/nomad/structs/structs.go (L6643) for details.
2022-05-19 20:54:49 -04:00
Will Jordan d515e5c3b0
Don't buffer json logs on agent startup (#13076)
There's no reason to buffer json logs on agent startup
since logs in this format already aren't reordered.
2022-05-19 15:40:30 -04:00
Seth Hoenig d9c10fccde
Merge pull request #13070 from hashicorp/b-vault-validator-env
cli: correctly validate job with vault token set
2022-05-19 14:31:10 -05:00
claire labry 7693818d56
Merge pull request #13068 from twunderlich-grapl/twunderlich/run-postinstall-script
[CI-only] Use the postinstall script for linux packages
2022-05-19 14:16:08 -04:00
Seth Hoenig fc58f4972c cli: correctly use and validate job with vault token set
This PR fixes `job validate` to respect '-vault-token', '$VAULT_TOKEN',
'-vault-namespace' if set.
2022-05-19 12:13:34 -05:00
Thomas Wunderlich ba6f81d843
Use the postinstall script
It appears that the postinstall script was created but never used.
This change is to actually use the post-install script.
2022-05-19 12:49:44 -04:00
Tim Gross b72ff42ada
api: include Consul token in job revert API (#13065) 2022-05-19 11:30:29 -04:00
Seth Hoenig 89c72d74d7
Merge pull request #13044 from hashicorp/b-fixup-init-redis
cli: update default redis and use nomad service discovery
2022-05-17 11:19:27 -05:00
Seth Hoenig 29d3da6dfd cl: update changelog 2022-05-17 10:35:08 -05:00
Seth Hoenig 65f7abf2f4 cli: update default redis and use nomad service discovery
Closes #12927
Closes #12958

This PR updates the version of redis used in our examples from 3.2 to 7.
The old version is very not supported anymore, and we should be setting
a good example by using a supported version.

The long-form example job is now fixed so that the service stanza uses
nomad as the service discovery provider, and so now the job runs without
a requirement of having Consul running and configured.
2022-05-17 10:24:19 -05:00
Seth Hoenig 26b5c01431
Merge pull request #12817 from twunderlich-grapl/fix-network-interpolation
Fix network.dns interpolation
2022-05-17 09:31:32 -05:00
Seth Hoenig 08becb117c cl: add changelog note for network interpolation 2022-05-17 09:14:55 -05:00
Luiz Aoqui 854209af0b
Merge pull request #13033 from hashicorp/docs-consul-upgrade-banner
docs: add Consul 1.12.0 upgrade notice
2022-05-16 19:23:08 -04:00
Luiz Aoqui fea13f39b3
docs: add Consul 1.12.0 upgrade notice 2022-05-16 18:44:26 -04:00
Luiz Aoqui 5147a3a2d4
Merge pull request #13013 from hashicorp/post-1.3.0-release
Post 1.3.0 release
2022-05-16 15:32:42 -04:00
Phil Renaud 0637eb742f
Add a forgotten comma to snapshot-specific CSS (#13030) 2022-05-16 14:13:51 -04:00
Tim Gross f4703ab8a3
docs: API package tests need a binary with your changes (#13029)
Add a note to the contributing guide pointing out that if you're
writing `api` package tests, you need to build a binary with any of
your changes.
2022-05-16 11:12:54 -04:00
Ivo 703a7954f4
[terraform/aws] Fix NVidia GPG key error (#12985)
* Fix NVidia GPG key error - NVidia rotated their repo keys, see https://forums.developer.nvidia.com/t/notice-cuda-linux-repository-key-rotation/212771
2022-05-16 06:49:01 -04:00
Karan Sharma e0be868b79
docs: Fix typo in sidecar_service (#13021) 2022-05-16 09:35:42 +02:00
Luiz Aoqui 525c0fadf4
add missing changelog entry for 1.2.7 2022-05-13 17:42:14 -04:00
Luiz Aoqui d46acb7147
Merge release 1.3.0 files 2022-05-13 17:33:09 -04:00
hc-github-team-nomad-core b28fcac665
Prepare for next release 2022-05-13 17:32:36 -04:00
hc-github-team-nomad-core 8c5dbe1a44
Generate files for 1.3.0 release 2022-05-13 17:32:20 -04:00
hc-github-team-nomad-core 214a4841b8
Prepare for next release 2022-05-13 17:32:11 -04:00
hc-github-team-nomad-core b0ec54c885
Generate files for 1.3.0-rc.1 release 2022-05-13 17:31:57 -04:00
Phil Renaud 45dc1cfd58
12986 UI fails to load job when there is an "@" in job name in nomad 130 (#13012)
* LastIndexOf and always append a namespace on job links

* Confirmed the volume equivalent and simplified idWIthNamespace logic

* Changelog added

* PR comments addressed

* Drop the redirect for the time being

* Tests updated to reflect namespace on links

* Task detail test default namespace link for test
2022-05-13 17:01:27 -04:00
Tim Gross faeb3fcd44
scheduler: volume updates should always be destructive (#13008) 2022-05-13 11:34:04 -04:00
dependabot[bot] 4ae15399bd
build(deps): bump cross-fetch from 3.1.4 to 3.1.5 in /website (#12818)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-05-13 09:31:49 -05:00
James Rasell 636b647a30
agent: fix panic when logging about protocol version config use. (#12962)
The log line comes before the agent logger has been setup,
therefore we need to use the UI logging to avoid panic.
2022-05-13 09:28:43 +02:00
Michael Schurter 7f8cf9e2dc
docs: link s/port-plan-failure to more helpful doc (#12968)
The shortlink /s/port-plan-failure is logged when a plan for a node is
rejected to help users debug and mitigate repeated `plan for node
rejected` failures.

The current link to #9506 is... less than useful. It is not clear to
users what steps they should take to either fix their cluster or
contribute to the issue.

While .../monitoring-nomad#progess isn't as comprehensive as it could
be, it's a much more gentle introduction to the class of bug than the
original issue.
2022-05-12 13:59:17 -07:00
Tim Gross 6e5d6eb3b5
docs: note that already-dispatched jobs cannot be updated (#12973) 2022-05-12 16:18:42 -04:00
Phil Renaud 916dbdcd2f
Visual diff tests seed-stabilized by default (#12965)
* Seed-stabilization by default

* Hide right-column of topology viz route

* Remove seedless run from thee test:* suite

* Related evals paths render too late

* Vis:Hidden another topo viz unstable item
2022-05-12 16:09:19 -04:00
Tim Gross ae2d7d6727
docs: remove beta tag for CSI from sidebar (#12970) 2022-05-12 14:12:40 -04:00
Eng Zer Jun 97d1bc735c
test: use `T.TempDir` to create temporary test directory (#12853)
* test: use `T.TempDir` to create temporary test directory

This commit replaces `ioutil.TempDir` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `ioutil.TempDir`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix TestLogmon_Start_restart on Windows

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* test: fix failing TestConsul_Integration

t.TempDir fails to perform the cleanup properly because the folder is
still in use

testing.go:967: TempDir RemoveAll cleanup: unlinkat /tmp/TestConsul_Integration2837567823/002/191a6f1a-5371-cf7c-da38-220fe85d10e5/web/secrets: device or resource busy

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-05-12 11:42:40 -04:00
Michael Schurter 5a43d3c675
docs: add `sysbatch` to scheduling internals (#12954) 2022-05-11 17:06:17 -07:00
Luiz Aoqui ed7798fbdf
prepare for next release (#12956) 2022-05-11 17:42:53 -04:00
Seth Hoenig 466f0c3fd2
build: use new version of hc-install (#12937)
https://github.com/shoenig/hc-install/pull/2

Uses new version of hc-install which supports the new
json content type reported by api.releases.hashicorp.com
2022-05-10 15:28:29 -04:00
Georges-Etienne Legendre 864be37c73
Fix Exec not working with reverse proxy X-Nomad-Token (#12925)
* Capture token secret on fetch

* Fix tests

* Fix lint errors
2022-05-10 13:42:12 -04:00
modrake cad8c00f9a
Merge pull request #12913 from hashicorp/mdrake/svc-acct-codeowner
add service acct to codeowners for backport merging
2022-05-06 10:44:31 -07:00
Morgan Drake 52b09953ab add service acct to codeowners for backport merging 2022-05-06 10:06:20 -07:00