Commit graph

19992 commits

Author SHA1 Message Date
Chris Baker d41d7dcad2 added changelog entry for 9311 and re-ordered changelog entries to respect secondary short-to-long ordering 2020-11-11 19:59:10 +00:00
Chris Baker 48b1674335
Merge pull request #9311 from jeromegn/allow-empty-devices
Don't ignore nil devices in plugin fingerprint
2020-11-11 13:54:03 -06:00
Tim Gross 60874ebe25
csi: Postrun hook should not change mode (#9323)
The unpublish workflow requires that we know the mode (RW vs RO) if we want to
unpublish the node. Update the hook and the Unpublish RPC so that we mark the
claim for release in a new state but leave the mode alone. This fixes a bug
where RO claims were failing node unpublish.

The core job GC doesn't know the mode, but we don't need it for that workflow,
so add a mode specifically for GC; the volumewatcher uses this as a sentinel
to check whether claims (with their specific RW vs RO modes) need to be claimed.
2020-11-11 13:06:30 -05:00
Chris Baker fbe3670b74
Merge pull request #9317 from hashicorp/f-recommendations-cli-autocomplete
recommendations CLI: autocomplete support
2020-11-11 12:04:23 -06:00
Chris Baker 26469093cd
Merge pull request #9312 from hashicorp/b-9227-scaling-policy-filtering
use both job and type query on scaling policy list endpoint
2020-11-11 12:04:06 -06:00
Tim Gross 0ef0b17b82
docker: disallow volume mounts from host by default (#9321)
The default behavior for `docker.volumes.enabled` is intended to be `false`,
but the HCL schema defaults to `true` if the value is unset. Set the default
literal value to `true`.

Additionally, Docker driver mounts of type "volume" (but not "bind") are not
being properly sandboxed with that setting. Disable Docker mounts with type
"volume" entirely whenever the `docker.volumes.enabled` flag is set to
false. Note this is unrelated to the `volume_mount` feature, which is
constrained to preconfigured host volumes or whatever is mounted by a CSI
plugin.

This changeset includes updates to unit tests that should have been failing
under the documented behavior but were not.
2020-11-11 10:03:46 -05:00
Chris Baker 4fc913eb8c
Merge branch 'master' into b-9227-scaling-policy-filtering 2020-11-11 07:50:01 -06:00
Drew Bailey 9a1fc720c8
enables audit log on full-cluster (#9315) 2020-11-11 08:33:01 -05:00
Drew Bailey 11a0dc87ed
changelog for enterprise bug (#9320) 2020-11-11 08:32:16 -05:00
Chris Baker d297083d6b changelog 2020-11-11 11:57:04 +00:00
Chris Baker 2deb77dcf5 cli: updated recommendation commands and test to remove duplication of autocompletion code 2020-11-11 11:39:26 +00:00
Chris Baker e3c0ea654d auto-complete for recommendations CLI, plus OSS components of recommendations prefix search 2020-11-11 11:13:43 +00:00
Mahmood Ali cb35c2583e
Merge pull request #9310 from hashicorp/f-warn-against-hcl2
Custom message when job file is HCL2 incompatible
2020-11-10 21:06:10 -05:00
Chris Baker 53aa5e75c9 fix #9227: use both job and type query on scaling policy list endpoint 2020-11-10 23:26:35 +00:00
Jerome Gravel-Niquet d1f1dbd203
Don't ignore nil devices in plugin fingerprint
Even if a plugin sends back an empty `[]*device.DeviceGroup`, it's transformed to `nil` during the RPC. Our custom device plugin is returning empty `FingerprintResponse.Devices` very often. Our temporary fix is to send a dummy `*DeviceGroup` if the slice is empty. This has the effect of never triggering the "first fingerprint" and therefore timing out after 50s.

In turn, this made our node exceed its hearbeat grace period when restarting it, revoking all vault tokens for its allocations, causing a restart of all our allocations because the token couldn't be renewed.

Removing the logic for `f.Devices == nil` does not appear to affect the functionality of the function.
2020-11-10 16:04:22 -05:00
Mahmood Ali f61d284fe6 Custom message when job file is HCL2 incompatible
Use a custom message when the job file is a valid HCL1 but no longer
valid under HCL 2 syntax.
2020-11-10 14:56:20 -05:00
Michael Schurter 49c92433af
Merge pull request #9305 from hashicorp/go-1.15.4
go: update from 1.15.1 to 1.15.4
2020-11-10 10:34:43 -08:00
Mahmood Ali 2d4634bcc3
Merge pull request #9304 from hashicorp/b-legacy-executors-are-executors
Legacy executors are executors after all
2020-11-10 12:54:03 -05:00
Mahmood Ali cac55f8991 update changelog
[ci skip]
2020-11-10 12:50:34 -05:00
Kris Hicks 9d03cf4c5f
protos: Update .proto files not to use Go package name (#9301)
Previously, it was required that you `go get github.com/hashicorp/nomad` to be
able to build protos, as the protoc invocation added an include directive that
pointed to `$GOPATH/src`, which is how dependent protos were discovered. As
Nomad now uses Go modules, it won't necessarily be cloned to `$GOPATH`.
(Additionally, if you _had_ go-gotten Nomad at some point, protoc compilation
would have possibly used the _wrong_ protos, as those wouldn't necessarily be
the most up-to-date ones.)

This change modifies the proto files and the `protoc` invocation to handle
discovering dependent protos via protoc plugin modifier statements that are
specific to the protoc plugin being used.

In this change, `make proto` was run to recompile the protos, which results in
changes only to the gzipped `FileDescriptorProto`.
2020-11-10 08:42:35 -08:00
Michael Schurter 1a4fe89866 docs: add go upgrade to changelog 2020-11-10 08:38:05 -08:00
Michael Schurter 447ec6389e go: update from 1.15.1 to 1.15.4
1.15.5 is due in a couple days but we may release before then.
2020-11-10 08:35:25 -08:00
Mahmood Ali ac185b41e2 Legacy executors are executors after all
This fixes a bug where pre-0.9 executors fail to recover after an
upgrade.

The bug is that legacyExecutorWrappers didn't get updated with
ExecStreaming function, and thus failed to implement the Executor
function. Sadly, this meant that all recovery attempts fail, as the
runtime check in
b312aacbc9/drivers/shared/executor/utils.go (L103-L110)
.
2020-11-10 10:20:07 -05:00
Ryan Oaks a6ad204878 Remove Netlify config 2020-11-10 09:57:14 -05:00
Seth Hoenig 7427da2a41
Merge pull request #9299 from hashicorp/b-detect-unloaded-kmod
client/fingerprint: detect unloaded dynamic bridge kernel module
2020-11-10 08:24:28 -06:00
Mahmood Ali 69849a42a5
Merge pull request #9298 from hashicorp/f-hcl2-localsvars
HCL2: Variables and Locals
2020-11-09 16:44:37 -05:00
Mahmood Ali 61650167bf use a constructor to initialize job config 2020-11-09 16:27:22 -05:00
Mahmood Ali 2e5ce5aa96 clarify test 2020-11-09 16:24:22 -05:00
Mahmood Ali a155423ba3 clarify variable references 2020-11-09 16:23:09 -05:00
Luiz Aoqui f82abfb0fc
Merge pull request #9300 from hashicorp/changelog-for-job-scale-bug
add CHANGELOG entry for #9296
2020-11-09 15:49:05 -05:00
Luiz Aoqui fb0faa657c
add CHANGELOG entry for #9296 2020-11-09 15:26:38 -05:00
Seth Hoenig bbd6ab6167 docs: add changelog entry 2020-11-09 14:16:33 -06:00
Luiz Aoqui ea81ac5d3d
Merge pull request #9296 from hashicorp/b-remove-namespace-from-scale-request
Remove Namespace field from JobScaleRequest
2020-11-09 15:13:33 -05:00
Mahmood Ali 725e715c2e Update variable interpolation tests 2020-11-09 15:03:17 -05:00
Mahmood Ali 7fcda0f01f Start accepting input variable files as CLI arguments 2020-11-09 15:02:58 -05:00
Mahmood Ali 8f5bc0d8cd Parse variable files 2020-11-09 15:02:21 -05:00
Mahmood Ali 1ae3e8a8eb Start using the new jobspec2 API 2020-11-09 15:01:31 -05:00
Mahmood Ali bbe3d315d7 Restructure Variables and Locals
This reimplements the handling of Variables and Locals in HCL2 config.

This change supports declaring variables and locals, with defaults.
2020-11-09 14:58:57 -05:00
Seth Hoenig 9960f96446 client/fingerprint: detect unloaded dynamic bridge kernel module
In Nomad v0.12.0, the client added additional fingerprinting around the
presense of the bridge kernel module. The fingerprinter only checked in
`/proc/modules` which is a list of loaded modules. In some cases, the
bridge kernel module is builtin rather than dynamically loaded. The fix
for that case is in #8721. However we were still missing the case where
the bridge module is dynamically loaded, but not yet loaded during the
startup of the Nomad agent. In this case the fingerprinter would believe
the bridge module was unavailable when really it gets loaded on demand.

This PR now has the fingerprinter scan the kernel module dependency file,
which will contain an entry for the bridge module even if it is not yet
loaded.

In summary, the client now looks for the bridge kernel module in
 - /proc/modules
 - /lib/modules/<kernel>/modules.builtin
 - /lib/modules/<kernel>/modules.dep

Closes #8423
2020-11-09 13:56:14 -06:00
Mahmood Ali 9927a6b8cd copy variables files from Packer 2020-11-09 14:35:02 -05:00
Tim Gross 08ae13d3b9
e2e: Windows provisioning improvements (#9246)
Small changes to the Windows 2016 Packer build for debuggability of
provisioning:

* improve verbosity of powershell error handling
* remove unused "tools" installation
* use ssh communicator for Packer to improve Packer build times and eliminate
  deprecated winrm remote access (unavailable from current macOS)
2020-11-09 13:29:40 -05:00
Luiz Aoqui c536286c7a
remove Namespace field from JobScaleRequest 2020-11-09 13:02:05 -05:00
Byron Wolfman 0c65d11538
Document correct default nomad port (#9290) 2020-11-06 16:37:36 -05:00
Drew Bailey 152e535ddd
re-run make sync from earlier event_stream changes (#9291) 2020-11-06 16:36:15 -05:00
Drew Bailey c181973265
append custom path to custom_config_files (#9289)
* append custom path to custom_config_files

* remove config_path variable
2020-11-06 11:16:13 -05:00
James Rasell d20e2d773c
docs: add recommendation CLI documentation. (#9288)
Co-authored-by: Chris Baker <1675087+cgbaker@users.noreply.github.com>
2020-11-06 16:30:24 +01:00
James Rasell 473fb4bb17
Merge pull request #9272 from hashicorp/f-recommendation-cli
cli: add recommendation commands.
2020-11-06 16:29:57 +01:00
Kris Hicks ecde165d79
api/testutil: Use fast fingerprint timeout in dev mode tests (#9285) 2020-11-06 07:01:27 -08:00
Buck Doyle 8b5b2116ec
Fix job detail crash when recommendations off (#9269)
Without this, visiting any job detail page on Nomad OSS would crash with
an error like this:

Error: Ember Data Request GET
/v1/recommendations?job=ping%F0%9F%A5%B3&namespace=default returned a
404 Payload (text/xml)

The problem was twofold.

1. The recommendation ability didn’t include anything about checking
whether the feature was present. This adds a request to
/v1/operator/license on application load to determine which features are
present and store them in the system service. The ability now looks for
'Dynamic Application Sizing' in that feature list.

2. Second, I didn’t check permissions at all in the job-fetching or job
detail templates.
2020-11-06 08:21:38 -06:00
James Rasell fe92d8b3cb
cli: add recommendation commands.
Adds new CLI commands for applying, dismissing and detailing Nomad
recommendations under a new top level recommendation command.
2020-11-06 11:16:24 +01:00