Commit graph

13411 commits

Author SHA1 Message Date
Mahmood Ali b55fb642f1 driver/docker: honor plugin devices 2018-12-04 21:31:28 -05:00
Mahmood Ali a580cef986 refactor device manipulation 2018-12-04 20:55:59 -05:00
Mahmood Ali adb4d69576
Merge pull request #4956 from hashicorp/b-vault-client-tweaks-followup
server/vault: Lock Vault expiration tracking
2018-12-04 19:46:59 -05:00
Mahmood Ali 366f478f8f
Merge pull request #4959 from hashicorp/fix-rkt-tests-20181204
tests: fix rkt tests
2018-12-04 19:46:41 -05:00
Mahmood Ali 8df9de6fd5 driver/rkt: use rkt environment
The rkt command itself needs an environment with PATH set to find iptables.
2018-12-04 14:00:45 -05:00
Preetha 8068d9f64e
Merge pull request #4949 from hashicorp/b-neg-running-summary
Add guards around subtracting summary count
2018-12-04 12:52:58 -06:00
Mahmood Ali f8efc40b8b tests: stop integration tests tasks explicitly
Also update the new recommended `nomad job` subcommands
2018-12-04 11:50:59 -05:00
Dan Brown 8aebe8c47d Add Reference Architecture and Deployment Guide (#4768)
* Add Nomad RA

* Add deployment guide and nav

* Deployment Guide update

* Minor typo fixes

* Update diagrams

* Fixes for review

* Link fixes and typo fix

* Edits following review

- Update image text from "zone" to "datacenter" to match Nomad terminology
- Clean up text based on Preetha's feedback

* Text updates

Based on feedback from Rob

* Update diagrams

* fixing spelling

* Add suggestions from Preetha and Omar
2018-12-04 11:49:35 -05:00
Mahmood Ali 06a5cadf35 drivers/rkt: use image isolation for rkt 2018-12-04 11:40:10 -05:00
Mahmood Ali 178365848e tests: don't assert in WaitForResult
WaitForResult expects body to fail and retries few times before giving
up.  Assertions inside the testfn body causes it to terminate abruptly
without retrying.
2018-12-04 11:40:10 -05:00
Mahmood Ali 50e38104a5 server/nomad: Lock Vault expiration tracking
`currentExpiration` field is accessed in multiple goroutines: Stats and
renewal, so needs locking.

I don't anticipate high contention, so simple mutex suffices.
2018-12-04 09:29:48 -05:00
Mahmood Ali f8ceeebf11
no t.Parallel() in excutor table driven tests (#4948)
When `t.Parallel()` is used inside a `t.Run()` sub-set, the closure
doesn't behave as expected, and some cases effectively get skipped.
More details can be found in
https://gist.github.com/posener/92a55c4cd441fc5e5e85f27bca008721
2018-12-04 09:04:04 -05:00
Mahmood Ali 216a2566c7
Update LXC with drivers/testutils changes (#4951) 2018-12-04 08:57:54 -05:00
Danielle Tomlinson 10b3e68a6d
Merge pull request #4925 from hashicorp/f-driver-plugins-dani
Third Party Driver Plugins Support
2018-12-03 20:48:19 +01:00
Michael Schurter e50f80d466
Merge pull request #4947 from githubfoam/update-demo-vagrantfile
no need to double the work
2018-12-03 11:17:08 -08:00
Preetha Appan 8656d3379f
Add guards around subtracting summary count 2018-12-03 11:16:35 -06:00
Mahmood Ali 88622b97bd
libcontainer to manage /dev and /proc (#4945)
libcontainer already manages `/dev`, overriding task_dir - so let's use it for `/proc` as well and remove deadcode.
2018-12-03 10:41:01 -05:00
githubfoam b4b79ed96a no need to double the work 2018-12-02 01:03:44 +03:00
Danielle Tomlinson 8ba0a816f3 plugins: Add support for serving driver plugins 2018-12-01 17:30:54 +01:00
Danielle Tomlinson 393b76ed7f plugins: Move driver testing support to subpackage
this allows us to drop a cyclical import, but is subobptimal as it
requires BaseDriver tests to move. This falls firmly into the realm of
being a hack. Alternatives welcome.
2018-12-01 17:29:39 +01:00
Danielle Tomlinson 9bd77e9295 testfix: Fix import cycle in allocdir tests 2018-12-01 17:25:30 +01:00
Danielle Tomlinson 66c521ca17 client: Move fingerprint structs to pkg
This removes a cyclical dependency when importing client/structs from
dependencies of the plugin_loader, specifically, drivers. Due to
client/config also depending on the plugin_loader.

It also better reflects the ownership of fingerprint structs, as they
are fairly internal to the fingerprint manager.
2018-12-01 17:10:39 +01:00
Mahmood Ali c27860ecdd
Update changelog (#4942)
Batch update for the PRs I recently merged.
2018-12-01 07:46:34 -05:00
Omar Khawaja 51bc332f8b
remove redirects for vault-integration docs (#4943) 2018-11-30 13:14:57 -05:00
Danielle Tomlinson 51a9f7369e
Merge pull request #4936 from hashicorp/f-legacy-refactor
Refactor and repackage client/driver
2018-11-30 13:38:06 +01:00
Mahmood Ali 84e04cfa40
Merge pull request #4926 from hashicorp/f-docker-image-ref
Use user provided image name to launch container
2018-11-30 07:27:39 -05:00
Mahmood Ali 94d43b8003
Merge pull request #4924 from hashicorp/f-docker-mounts
Support bind and tmpfs docker mounts
2018-11-30 07:27:17 -05:00
Michael Schurter 100e09b9ad
docs: Use valid docker bind propagation field
Co-Authored-By: notnoop <mahmood@notnoop.com>
2018-11-30 07:25:56 -05:00
Danielle Tomlinson 2db5ae38d8 client: Rename drivers/shared/env => client/taskenv 2018-11-30 12:18:39 +01:00
Danielle Tomlinson f00e127d46 Revert accidental changes to vendor/ 2018-11-30 11:01:09 +01:00
Danielle Tomlinson ffc5e5d56b executors: Unify go-plugin handshake 2018-11-30 10:59:23 +01:00
Danielle Tomlinson f3a77b8084 client: Merge driver/shared/structs and client/structs 2018-11-30 10:56:45 +01:00
Danielle Tomlinson b9295f0d56 client/driver: Remove package 2018-11-30 10:47:08 +01:00
Danielle Tomlinson d4cbd608ff nomad: Remove on-submission job validation
With the introduction of driver plugins, we're temporarily relying on
_run time validation_ of driver configurations, rather than submission
time.
2018-11-30 10:47:08 +01:00
Danielle Tomlinson fdfe93aa25 fixup: executorplugin: fix rkt build 2018-11-30 10:47:08 +01:00
Danielle Tomlinson d72ecd95ec client/driver: Vendor setEnvvars into docker_test 2018-11-30 10:46:13 +01:00
Danielle Tomlinson d26a310db0 client: Move executor plugins into own package 2018-11-30 10:46:13 +01:00
Danielle Tomlinson d259c36844 driver: Flatten SetEnvvars into taskdirhook 2018-11-30 10:46:13 +01:00
Danielle Tomlinson 9b3e731f88 command: Remove Extraneous field in nodedrain test 2018-11-30 10:46:13 +01:00
Danielle Tomlinson 6b72e96eba client: Move driver/logging to logmon/logging
The logging package is used by logmon and the legacy mock_driver. Because the
legacy drivers are going away, I'm moving it here to signify its actual
ownership.
2018-11-30 10:46:13 +01:00
Danielle Tomlinson 04c8851b4c client: Migrate DriverStats optout to drivers/shared/structs 2018-11-30 10:46:13 +01:00
Danielle Tomlinson dbd82e1af4 client: Remove test dependency on client/driver 2018-11-30 10:46:13 +01:00
Danielle Tomlinson d582ea1d8b drivers: Create drivers/shared/structs
This creates a drivers/shared/structs package and moves the buffer size
checks into it.
2018-11-30 10:46:13 +01:00
Danielle Tomlinson 0544a57abe drivers: Move client/drivers/executor to drivers/shared/executor 2018-11-30 10:46:13 +01:00
Danielle Tomlinson 1a29811169 drivers: Move client/drivers/env to drivers/shared/env
As part of deprecating legacy drivers, we're moving the env package to a
new drivers/shared tree, as it is used by the modern docker and rkt
driver packages, and is useful for 3rd party plugins.
2018-11-30 10:46:13 +01:00
Nick Ethier 80ae7e34f4
Merge pull request #4906 from hashicorp/f-metric-prefix-master
Port metric prefix filtering to master
2018-11-29 22:27:47 -05:00
Nick Ethier b1484aec33
nomad: fix hclog usage 2018-11-29 22:27:39 -05:00
Danielle Tomlinson 3b4b750ce1
Merge pull request #4938 from hashicorp/b-fix-make
make: Fix invocation of vault e2e test
2018-11-29 21:15:06 +01:00
Omar Khawaja ce5ef3ab26
fix step 5 broken link (#4939) 2018-11-29 14:35:39 -05:00
Mahmood Ali ea6834e86e
Merge pull request #4937 from hashicorp/b-vault-panic
vault: protect against empty Vault secret response

Fixes #4921

Sadly, we don't have proper mechanism to mock Vault client, so not sure how to best test this.

I inspected the Vault client interactions, specially for cases where returned value is nil even if the error is also nil.  I believe we covered all correctly now:
* [`v.client.Sys().InitStatus()`](f3853f11da/nomad/vault.go (L427)) - the value is non-nil boolean
* [`v.client.Sys().CapabilitiesSelf(path)`](f3853f11da/nomad/vault.go (L812)): Capabilities handles empty bodies in [`hasCapability`](f3853f11da/vendor/github.com/hashicorp/vault/api/sys_capabilities.go (L43-L45)) - also the `nil` array is handled with proper fail-safe default.
* [`v.client.Logical().Read(fmt.Sprintf("auth/token/roles/%s", role))`](f3853f11da/nomad/vault.go (L834-L840)) handles when `rsecret` is nil
2018-11-29 10:35:58 -05:00