94d43b8003
Merge pull request #4924 from hashicorp/f-docker-mounts
...
Support bind and tmpfs docker mounts
2018-11-30 07:27:17 -05:00
2db5ae38d8
client: Rename drivers/shared/env => client/taskenv
2018-11-30 12:18:39 +01:00
f3a77b8084
client: Merge driver/shared/structs and client/structs
2018-11-30 10:56:45 +01:00
fdfe93aa25
fixup: executorplugin: fix rkt build
2018-11-30 10:47:08 +01:00
04c8851b4c
client: Migrate DriverStats optout to drivers/shared/structs
2018-11-30 10:46:13 +01:00
d582ea1d8b
drivers: Create drivers/shared/structs
...
This creates a drivers/shared/structs package and moves the buffer size
checks into it.
2018-11-30 10:46:13 +01:00
0544a57abe
drivers: Move client/drivers/executor to drivers/shared/executor
2018-11-30 10:46:13 +01:00
1a29811169
drivers: Move client/drivers/env to drivers/shared/env
...
As part of deprecating legacy drivers, we're moving the env package to a
new drivers/shared tree, as it is used by the modern docker and rkt
driver packages, and is useful for 3rd party plugins.
2018-11-30 10:46:13 +01:00
0d90ba392e
Fix lxc test panic
2018-11-28 13:56:17 -06:00
924f1b69e9
Fix failing lxc test
2018-11-28 11:05:35 -06:00
bf58c65ef7
Fix LXC driver fingerprint to use typedattributes
2018-11-28 10:09:10 -06:00
9f4439243b
Fix docker driver to use new fingerprint typed attributes
2018-11-28 10:01:03 -06:00
f89dbcd9cc
modify fingerprint interface to use typed attribute struct
2018-11-28 10:01:03 -06:00
9af8deabbf
address review comments
2018-11-27 21:40:43 -05:00
6d34d2fade
Add Driver Plugin for LXC
2018-11-27 21:40:43 -05:00
e565c63eed
gofmt -s -w drivers/rkt/driver_test.go
2018-11-27 17:24:23 -08:00
844fd47acc
Use user provided image name to launch container
...
This allows the container to be tagged with a user friendly image name
(e.g. `redis:3.2`) rather than the image ID (e.g.
`sha256:87856cc39862cec77541d68382e4867d7ccb29a85a17221446c857ddaebca916`).
Useful for human debugging, as well as some debugging and image scanning
tools.
This risks two bad changes:
1. Discrepancy in image resolution between docker and Nomad's image
loader.
* I checked the image creation paths in Nomad, and noticed that we
either pulled the image or inspect the image with the user provided
name.
2. A race in image tagging where the tag is modified between image
loading and container creation.
* I, personally, don't think this case is cause for concern, as it is
analogous to the task running a bit later. As long as the image is
still present, creating the container should be good.
2018-11-27 16:12:15 -05:00
f6d6a50c39
add support for tmpfs
2018-11-27 07:20:17 -05:00
0a09f5521d
Support docker bind mounts
2018-11-27 07:20:17 -05:00
3dd6ba514a
drivers/rkt: updated test to include new AllocID field in TaskConfig
2018-11-26 21:37:58 +00:00
9bd4317139
modified TaskConfig to include AllocID
...
use this for volume names in drivers/rkt to address #1150
2018-11-26 18:54:26 +00:00
141092e46d
Formatting and typo fixes
2018-11-25 11:53:21 -05:00
c61d99b525
Merge pull request #4908 from hashicorp/f-docker-opts-storageopt
...
Add support for docker storage options
2018-11-20 21:08:27 -05:00
1f3fe02e62
docker: sync access to exit result within a handle
2018-11-20 20:41:32 -05:00
2275153875
Apply suggestions from code review
...
Co-Authored-By: nickethier <ncethier@gmail.com>
2018-11-20 20:33:31 -05:00
e9e415f186
Add support for storage opt
2018-11-20 16:11:02 -05:00
3ccd359735
docker: unexport new coordinator func
2018-11-19 23:07:07 -05:00
8b9b2b476e
docker: add default blocks for driver plugin config schema
2018-11-19 22:59:18 -05:00
2667f48a5d
docker: move config RPCs to config.go
2018-11-19 22:59:18 -05:00
aa9f45ae47
docker: fix tests
2018-11-19 22:59:18 -05:00
0f03e8f520
docker: remove container pointer from task handle
2018-11-19 22:59:18 -05:00
ce4b867d21
docker: move volume driver options to seperate block
2018-11-19 22:59:18 -05:00
fca2df3c79
docker: group common config into blocks
2018-11-19 22:59:17 -05:00
813341dd59
Apply suggestions from code review
...
Co-Authored-By: nickethier <ncethier@gmail.com>
2018-11-19 22:59:17 -05:00
b7bd36db30
docker: remove global pull coordinator
2018-11-19 22:59:17 -05:00
f0a86859a0
docker: remove call to global metrics instance
2018-11-19 22:59:17 -05:00
8ef73e63ce
docker: moved fingerprint code to it's own file
2018-11-19 22:59:17 -05:00
4be8a86ef9
plugins/driver: remove NodeResources from task Resources and use PercentTicks field for docker driver
2018-11-19 22:59:17 -05:00
ced5d5c445
docker: move recoverable error proto to shared structs
2018-11-19 22:59:16 -05:00
585e468085
docker: implement recover task logic
2018-11-19 22:59:16 -05:00
ee51cb6a93
docker: finished porting tests
2018-11-19 22:59:16 -05:00
3d7cdea19e
drivers/docker: more work porting tests from old driver plugin
2018-11-19 22:59:16 -05:00
8f8698b3e1
docker: started work on porting docker driver to new plugin framework
2018-11-19 22:59:15 -05:00
9e64ce7d73
docker: properly launch docker logger process
2018-11-19 22:59:12 -05:00
f8cdd561f0
client: interpolate driver configurations
...
Also add missing SetDriverNetwork calls.
2018-11-15 16:25:57 -08:00
1f4db08f42
Regenerate proto files with protoc-gen-go@v1.2.0
2018-11-14 09:01:26 -05:00
1e92161f14
Merge pull request #4858 from hashicorp/b-fix-master-20181109
...
Fix some tests in master
2018-11-13 16:08:26 -05:00
17e8446484
Merge pull request #4868 from hashicorp/b-plugin-ctx
...
Plugin client's handle plugin dying
2018-11-13 10:26:53 -08:00
356c194acc
Use materialized duration fields for driver config
2018-11-13 10:21:40 -05:00
470d20cdf3
Avoid downloading image if present locally
2018-11-13 10:21:40 -05:00
7690f389a0
Prioritize checking consumer context cancellation
...
Tests expect that as soon as eventer shuts down immediately on context
cancellations; but golang does not guarantee priority when multiple
pending channels are ready in a select statement.
2018-11-13 10:21:40 -05:00
e9067e52b4
pull alpine image needed for test
...
The test requires the image to be present locally, so importing it as
part of setup.
2018-11-13 10:21:40 -05:00
8923ea4663
Handle time.Duration in mock
...
Mock driver config uses `time.Duration` fields but we initialize them
inconsistently, as time.Duration sometimes and as duration strings other
times. Previously, `mapstructure` handles it and does the right thing.
This is no longer the case with MsgPack. I could not find a good way to
bring back old behavior without too much complexity. `MsgPack` extended
types weren't ideal here as we lose type information (e.g. int64 vs
string), and the input is a generic map and not a MsgPack serialization
of duration.
As such, I went with the simple solution of declaring the config field
as duration string, and panicing if the test doesn't pass a valid
string.
I found this to cause the smallest change in tests, but we can
alternatively force all to be int64 instead.
2018-11-13 10:21:40 -05:00
fb56dd699d
distinguish java driver tests from others
2018-11-13 10:21:40 -05:00
343df28165
Fix java driver tests
2018-11-13 10:21:40 -05:00
e506ebbc24
Merge pull request #4845 from hashicorp/r-exec-refactor
...
Update exec driver to match rawexec
2018-11-12 20:59:32 -05:00
693f244cce
Plugin client's handle plugin dying
...
This PR plumbs the plugins done ctx through the base and driver plugin
clients (device already had it). Further, it adds generic handling of
gRPC stream errors.
2018-11-12 17:09:27 -08:00
880e5015f2
rawexec: Only use cgroups when running as root.
...
If Nomad is not running as root, we should not try to use cgroups for pid
freezing.
This originally was implemented pre-driver-support in
https://github.com/hashicorp/nomad/blob/v0.8.6/client/driver/raw_exec.go#L120-L130
2018-11-10 06:45:11 -08:00
9c76a7d9b8
handle nil resources limits
...
Config may not express resource constraints on the job!
2018-11-09 15:52:44 -05:00
1ed53ca478
shorter health descriptions
2018-11-08 10:00:49 -08:00
032f86bc78
Add a helper functions for checking unix root
2018-11-08 10:00:49 -08:00
21d126cd68
Update exec driver to match rawexec
...
Update exec driver with latest patterns used by rawexec, and eliminate
accidental diffs.
2018-11-08 10:00:49 -08:00
a7ca737fb6
review comments
2018-11-07 11:31:52 -08:00
204ca8230c
Device manager
...
Introduce a device manager that manages the lifecycle of device plugins
on the client. It fingerprints, collects stats, and forwards Reserve
requests to the correct plugin. The manager, also handles device plugins
failing and validates their output.
2018-11-07 10:43:15 -08:00
32283b2a66
reconcile java with latest patterns
2018-11-06 12:49:29 -08:00
467360954a
driver/java: support non-linux targets
...
Java driver supports non-linux targets, so LinuxResources fields isn't
guaranteed to be not-nil.
2018-11-06 12:41:39 -08:00
e64ef839b8
Reconcile with latest master patterns
2018-11-06 12:41:39 -08:00
c3881bd8d1
add java driver tests
2018-11-06 12:41:39 -08:00
4f525d0a17
Implement java driver
2018-11-06 12:41:39 -08:00
392d548b85
Merge pull request #4828 from hashicorp/b-restore
...
Implement client agent restarting
2018-11-05 18:50:15 -06:00
2bbd88888c
client: first pass at implementing task restoring
...
Task restoring works but dead tasks may be restarted
2018-11-05 12:32:05 -08:00
a17521475d
Merge pull request #4826 from hashicorp/b-driver-exec-tweaks-20181031
...
Register exec driver plugin among some fixes
2018-11-02 10:11:05 -04:00
dc0fc442ab
add plugin config
2018-11-02 08:51:23 -04:00
83db3e6a50
rkt: standardize names/code
2018-11-01 15:53:07 -07:00
1a7fb51285
rawexec: standardize names/code
2018-11-01 15:53:07 -07:00
5a5d52023d
qemu: standardize names/code
2018-11-01 15:53:07 -07:00
3c5e606ef6
drivers/mock: standardize names/code
2018-10-31 11:52:51 -07:00
9a45837c4e
Register exec driver plugin among some fixes
...
Namely, remove the `enabled` configuration flag, as it's specific to
`raw_exec` driver. Also, pass resource limits to underlying call.
2018-10-31 13:55:59 -04:00
d3dafc9761
mock_driver: match other's fingerprint message
2018-10-30 17:38:23 -07:00
2a38f575c0
rkt: fix race, register by default, fix fingerprint
2018-10-30 17:37:00 -07:00
5b0b75e1ee
qemu: fix fingerprint description and handle race
2018-10-30 17:34:08 -07:00
6f4a28a7c2
rawexec: fix fingerprint results and tests
...
In tests:
* always cleanup (Kill) the harness
* don't call SetConfig more than once (a race)
* avoid cgroups to allow running without being root
2018-10-30 17:16:22 -07:00
616801417c
rawexec: fix race around h.procState
2018-10-30 17:15:47 -07:00
9fa53565ad
drivers: remove stutter from exported driver names
...
Also fix a comment on the logger that got copy/pasted around.
2018-10-30 14:05:31 -07:00
fc320906aa
qemu: fix build and register by default
...
API changes had broken the build.
2018-10-29 22:10:37 -07:00
3fcf8ba7e6
Merge pull request #4795 from hashicorp/f-plugin-config
...
Pass client configuration to plugins through loader
2018-10-29 18:42:27 -07:00
bda3b1d3b3
rename NomadConfig to ClientAgentConfig
2018-10-29 21:34:34 -04:00
68991d8acb
Merge branch 'f-driver-plugin-qemu' of github.com:hashicorp/nomad into f-driver-plugin-qemu
2018-10-25 13:51:58 -05:00
bfff771416
remove driver config struct from qemu driver since its not used
2018-10-25 13:51:06 -05:00
48d25dbef1
Fix hclog log line formatting
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:47:17 -05:00
95a1637741
Fix imports
2018-10-25 13:45:44 -05:00
ca151a7828
Fix copy pasted comment
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:43:29 -05:00
c85b90f1e4
remove newline
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:42:20 -05:00
66c15e3a67
update comment
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:41:24 -05:00
9ca6c998c7
Remove unnecessary return
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:40:28 -05:00
fc7ba5e79d
remove unnecessary return
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:40:15 -05:00
9916b9e0ce
drivers: update SetConfig to handle driver client config
2018-10-18 23:32:17 -04:00
af3e9adf2f
Update drivers/qemu/driver.go
2018-10-17 17:42:01 -05:00
93eb048b87
QEMU driver plugin
2018-10-17 11:43:15 -05:00
4b694b2dec
Merge pull request #4793 from hashicorp/b-no-cty
...
Do not use cty in drivers
2018-10-16 20:04:41 -07:00
823a6c8d6c
update rkt and exec setconfig funcs
2018-10-16 23:00:26 -04:00
65adb80ebf
plumb NomadConfig into plugins
2018-10-16 22:47:22 -04:00
b657c3909a
drivers/exec: better fingerprinting and detection on non linux
2018-10-16 22:45:33 -04:00
d94b631b6b
drivers/exec: add exec implementation
2018-10-16 22:45:28 -04:00
88c1885044
Do not use cty in drivers
2018-10-16 17:17:07 -07:00
838ddf4d4a
fix linter errors
2018-10-16 16:56:57 -07:00
448a709603
driver/docker: rename container logger
2018-10-16 16:56:56 -07:00
64605a3957
drivers/mock: fix plugin name
...
Was mock_driver before plugins, so keep the name.
2018-10-16 16:56:56 -07:00
d634157158
Address review comments around logging task properly
2018-10-16 16:56:56 -07:00
3280727c4e
Address review comments
2018-10-16 16:56:56 -07:00
02b1f26d34
Switch back to using map[string]string for port map
2018-10-16 16:56:56 -07:00
ed5641055f
fixup comments, logging, and missing method impls
...
from #4777 comments
2018-10-16 16:56:56 -07:00
1a29337e48
register drivers by default
...
Do not register mock_driver on release builds.
2018-10-16 16:56:56 -07:00
118ce7c782
drivers/mock: complete plugin impl
2018-10-16 16:56:56 -07:00
9f564781ce
drivers/mock: start mock driver implementation
2018-10-16 16:56:56 -07:00
9b3e82a4e7
make port map a slice of maps to match existing rkt driver
2018-10-16 16:56:56 -07:00
4f4777d6a6
Review comments
2018-10-16 16:56:56 -07:00
edad5e9e6d
Stats collection test
2018-10-16 16:56:56 -07:00
678072ecd1
RKT driver plugin and unit tests
2018-10-16 16:56:56 -07:00
c7ac1186c9
client: add test for driverfailure during fingerprinting
2018-10-16 16:56:56 -07:00
15e8477c4e
rkt: start rkt driver plugin
2018-10-16 16:56:56 -07:00
c2d2bfa06c
Get raw exec tests compiling and passing again
2018-10-16 16:56:56 -07:00
3183b33d24
client: review comments and fixup/skip tests
2018-10-16 16:56:56 -07:00
64e9fc3798
docklog: add go-plugin for forwarding of docker logs
2018-10-16 16:56:56 -07:00
4a4c7dbbfc
client: begin driver plugin integration
...
client: fingerprint driver plugins
2018-10-16 16:56:56 -07:00
7946a14aa8
Fix lints
2018-10-16 16:56:56 -07:00
5fb95c1a00
drivers/shared: added func comment to eventer
2018-10-16 16:56:55 -07:00
951a084198
drivers/shared: move eventer to subpackage under drivers shared package
2018-10-16 16:56:55 -07:00
969ecea6f0
drivers/utils: better handling of consumer cleanup in eventer
2018-10-16 16:56:55 -07:00
8cf32eb9d2
drivers/rawexec: PR comments and feedback
2018-10-16 16:56:55 -07:00
a95cbc38ba
drivers/raw_exec: sync access to task state
2018-10-16 16:53:31 -07:00
28e8e5852c
drivers/raw_exec: added unix specific tests
2018-10-16 16:53:31 -07:00
1f6873806e
raw_exec: move package outside of plugins dir
2018-10-16 16:53:31 -07:00
Mahmood Ali