Commit graph

13841 commits

Author SHA1 Message Date
Danielle Tomlinson 2db5ae38d8 client: Rename drivers/shared/env => client/taskenv 2018-11-30 12:18:39 +01:00
Danielle Tomlinson f00e127d46 Revert accidental changes to vendor/ 2018-11-30 11:01:09 +01:00
Danielle Tomlinson ffc5e5d56b executors: Unify go-plugin handshake 2018-11-30 10:59:23 +01:00
Danielle Tomlinson f3a77b8084 client: Merge driver/shared/structs and client/structs 2018-11-30 10:56:45 +01:00
Danielle Tomlinson b9295f0d56 client/driver: Remove package 2018-11-30 10:47:08 +01:00
Danielle Tomlinson d4cbd608ff nomad: Remove on-submission job validation
With the introduction of driver plugins, we're temporarily relying on
_run time validation_ of driver configurations, rather than submission
time.
2018-11-30 10:47:08 +01:00
Danielle Tomlinson fdfe93aa25 fixup: executorplugin: fix rkt build 2018-11-30 10:47:08 +01:00
Danielle Tomlinson d72ecd95ec client/driver: Vendor setEnvvars into docker_test 2018-11-30 10:46:13 +01:00
Danielle Tomlinson d26a310db0 client: Move executor plugins into own package 2018-11-30 10:46:13 +01:00
Danielle Tomlinson d259c36844 driver: Flatten SetEnvvars into taskdirhook 2018-11-30 10:46:13 +01:00
Danielle Tomlinson 9b3e731f88 command: Remove Extraneous field in nodedrain test 2018-11-30 10:46:13 +01:00
Danielle Tomlinson 6b72e96eba client: Move driver/logging to logmon/logging
The logging package is used by logmon and the legacy mock_driver. Because the
legacy drivers are going away, I'm moving it here to signify its actual
ownership.
2018-11-30 10:46:13 +01:00
Danielle Tomlinson 04c8851b4c client: Migrate DriverStats optout to drivers/shared/structs 2018-11-30 10:46:13 +01:00
Danielle Tomlinson dbd82e1af4 client: Remove test dependency on client/driver 2018-11-30 10:46:13 +01:00
Danielle Tomlinson d582ea1d8b drivers: Create drivers/shared/structs
This creates a drivers/shared/structs package and moves the buffer size
checks into it.
2018-11-30 10:46:13 +01:00
Danielle Tomlinson 0544a57abe drivers: Move client/drivers/executor to drivers/shared/executor 2018-11-30 10:46:13 +01:00
Danielle Tomlinson 1a29811169 drivers: Move client/drivers/env to drivers/shared/env
As part of deprecating legacy drivers, we're moving the env package to a
new drivers/shared tree, as it is used by the modern docker and rkt
driver packages, and is useful for 3rd party plugins.
2018-11-30 10:46:13 +01:00
Nick Ethier 80ae7e34f4
Merge pull request #4906 from hashicorp/f-metric-prefix-master
Port metric prefix filtering to master
2018-11-29 22:27:47 -05:00
Nick Ethier b1484aec33
nomad: fix hclog usage 2018-11-29 22:27:39 -05:00
Danielle Tomlinson 3b4b750ce1
Merge pull request #4938 from hashicorp/b-fix-make
make: Fix invocation of vault e2e test
2018-11-29 21:15:06 +01:00
Omar Khawaja ce5ef3ab26
fix step 5 broken link (#4939) 2018-11-29 14:35:39 -05:00
Mahmood Ali ea6834e86e
Merge pull request #4937 from hashicorp/b-vault-panic
vault: protect against empty Vault secret response

Fixes #4921

Sadly, we don't have proper mechanism to mock Vault client, so not sure how to best test this.

I inspected the Vault client interactions, specially for cases where returned value is nil even if the error is also nil.  I believe we covered all correctly now:
* [`v.client.Sys().InitStatus()`](f3853f11da/nomad/vault.go (L427)) - the value is non-nil boolean
* [`v.client.Sys().CapabilitiesSelf(path)`](f3853f11da/nomad/vault.go (L812)): Capabilities handles empty bodies in [`hasCapability`](f3853f11da/vendor/github.com/hashicorp/vault/api/sys_capabilities.go (L43-L45)) - also the `nil` array is handled with proper fail-safe default.
* [`v.client.Logical().Read(fmt.Sprintf("auth/token/roles/%s", role))`](f3853f11da/nomad/vault.go (L834-L840)) handles when `rsecret` is nil
2018-11-29 10:35:58 -05:00
Danielle Tomlinson 031829c5a9 make: Fix invocation of vault e2e test 2018-11-29 15:37:26 +01:00
Mahmood Ali 0a2611e41f vault: protect against empty Vault secret response
Also, fix a case where a successful second attempt of loading token can
cause a panic.
2018-11-29 09:34:17 -05:00
Chris Baker b43090a267
Merge pull request #4932 from hashicorp/b-1172-rkt-env-vars
change to testing utilities to fix rkt tests
2018-11-29 09:18:10 -05:00
Nick Ethier bbe420718a
Merge pull request #4922 from hashicorp/f-drivermananger
add generic plugin manager interface and orchestration
2018-11-28 22:17:04 -05:00
Chris Baker da35fda145 testing: in MkAllocDir, do not update TaskConfig with All() from the task builder, just with Env() (because it pollutes environment variables with node attributes and fails the rkt tests) 2018-11-28 22:19:48 +00:00
Preetha 1f526db414
Merge pull request #4919 from hashicorp/f-fingerprint-attribute-type
Modify fingerprint interface to use typed attribute struct
2018-11-28 14:18:28 -06:00
Preetha Appan 0d90ba392e
Fix lxc test panic 2018-11-28 13:56:17 -06:00
Michael Schurter 1bd9a9f9dd
Merge pull request #4894 from hashicorp/f-device-hook
Device hook and devices affect computed node class
2018-11-28 12:10:43 -06:00
Preetha Appan 924f1b69e9
Fix failing lxc test 2018-11-28 11:05:35 -06:00
Preetha Appan bf58c65ef7
Fix LXC driver fingerprint to use typedattributes 2018-11-28 10:09:10 -06:00
Preetha Appan 9f4439243b
Fix docker driver to use new fingerprint typed attributes 2018-11-28 10:01:03 -06:00
Preetha Appan f89dbcd9cc
modify fingerprint interface to use typed attribute struct 2018-11-28 10:01:03 -06:00
Nick Ethier 60c6907ea5
client/plugin: remove println from plugin group func 2018-11-27 22:45:09 -05:00
Nick Ethier 600738e991
client/plugin: lint/spelling errors 2018-11-27 22:45:09 -05:00
Nick Ethier 45a6bf7acd
client/plugin: add generic plugin mananger interface and orchestration 2018-11-27 22:45:03 -05:00
Mahmood Ali 67d16d22b1
Merge pull request #4918 from hashicorp/f-lxc-driver
Add LXC driver
2018-11-27 22:06:42 -05:00
Mahmood Ali 9af8deabbf address review comments 2018-11-27 21:40:43 -05:00
Mahmood Ali 6d34d2fade Add Driver Plugin for LXC 2018-11-27 21:40:43 -05:00
Mahmood Ali ad1f8d8c20 Fixes in old lxc driver 2018-11-27 21:40:43 -05:00
Mahmood Ali 3f1887db8c Update go-lxc library to handle LXC 3.0 2018-11-27 21:40:43 -05:00
Michael Schurter 3e56ee005a add nil check around task resources in device hook
Looking at NewTaskRunner I'm unsure whether TaskRunner.TaskResources
(from which req.TaskResources is set) is intended to be nil at times or
if the TODO in NewTaskRunner is intended to ensure it is always non-nil.
2018-11-27 17:25:33 -08:00
Michael Schurter b75e9fce37 assume that slices contain only non-nil items 2018-11-27 17:25:33 -08:00
Michael Schurter 85073f9d29 client: properly support hook env vars
The old approach was incomplete. Hook env vars are now:

 * persisted and restored between agent restarts
 * deterministic (LWW if 2 hooks set the same key)
2018-11-27 17:25:33 -08:00
Alex Dadgar 4ee603c382 Device hook and devices affect computed node class
This PR introduces a device hook that retrieves the device mount
information for an allocation. It also updates the computed node class
computation to take into account devices.

TODO Fix the task runner unit test. The environment variable is being
lost even though it is being properly set in the prestart hook.
2018-11-27 17:25:33 -08:00
Michael Schurter e565c63eed gofmt -s -w drivers/rkt/driver_test.go 2018-11-27 17:24:23 -08:00
Michael Schurter 27e07f657e
Merge pull request #4896 from hashicorp/b-prevalloc-deadlock
Fix deadlock in previous alloc watcher by emitting last alloc update
2018-11-27 19:07:16 -06:00
Michael Schurter b75f79a793 fix test breakage caused by rebase 2018-11-27 16:34:01 -08:00
Michael Schurter 91da566935 fix mispelings 2018-11-27 16:33:55 -08:00