Preetha Appan
0d90ba392e
Fix lxc test panic
2018-11-28 13:56:17 -06:00
Preetha Appan
924f1b69e9
Fix failing lxc test
2018-11-28 11:05:35 -06:00
Preetha Appan
bf58c65ef7
Fix LXC driver fingerprint to use typedattributes
2018-11-28 10:09:10 -06:00
Preetha Appan
9f4439243b
Fix docker driver to use new fingerprint typed attributes
2018-11-28 10:01:03 -06:00
Preetha Appan
f89dbcd9cc
modify fingerprint interface to use typed attribute struct
2018-11-28 10:01:03 -06:00
Mahmood Ali
9af8deabbf
address review comments
2018-11-27 21:40:43 -05:00
Mahmood Ali
6d34d2fade
Add Driver Plugin for LXC
2018-11-27 21:40:43 -05:00
Michael Schurter
e565c63eed
gofmt -s -w drivers/rkt/driver_test.go
2018-11-27 17:24:23 -08:00
Mahmood Ali
844fd47acc
Use user provided image name to launch container
...
This allows the container to be tagged with a user friendly image name
(e.g. `redis:3.2`) rather than the image ID (e.g.
`sha256:87856cc39862cec77541d68382e4867d7ccb29a85a17221446c857ddaebca916`).
Useful for human debugging, as well as some debugging and image scanning
tools.
This risks two bad changes:
1. Discrepancy in image resolution between docker and Nomad's image
loader.
* I checked the image creation paths in Nomad, and noticed that we
either pulled the image or inspect the image with the user provided
name.
2. A race in image tagging where the tag is modified between image
loading and container creation.
* I, personally, don't think this case is cause for concern, as it is
analogous to the task running a bit later. As long as the image is
still present, creating the container should be good.
2018-11-27 16:12:15 -05:00
Mahmood Ali
f6d6a50c39
add support for tmpfs
2018-11-27 07:20:17 -05:00
Mahmood Ali
0a09f5521d
Support docker bind mounts
2018-11-27 07:20:17 -05:00
Chris Baker
3dd6ba514a
drivers/rkt: updated test to include new AllocID field in TaskConfig
2018-11-26 21:37:58 +00:00
Chris Baker
9bd4317139
modified TaskConfig to include AllocID
...
use this for volume names in drivers/rkt to address #1150
2018-11-26 18:54:26 +00:00
Mahmood Ali
141092e46d
Formatting and typo fixes
2018-11-25 11:53:21 -05:00
Mahmood Ali
c61d99b525
Merge pull request #4908 from hashicorp/f-docker-opts-storageopt
...
Add support for docker storage options
2018-11-20 21:08:27 -05:00
Nick Ethier
1f3fe02e62
docker: sync access to exit result within a handle
2018-11-20 20:41:32 -05:00
Michael Schurter
2275153875
Apply suggestions from code review
...
Co-Authored-By: nickethier <ncethier@gmail.com>
2018-11-20 20:33:31 -05:00
Mahmood Ali
e9e415f186
Add support for storage opt
2018-11-20 16:11:02 -05:00
Nick Ethier
3ccd359735
docker: unexport new coordinator func
2018-11-19 23:07:07 -05:00
Nick Ethier
8b9b2b476e
docker: add default blocks for driver plugin config schema
2018-11-19 22:59:18 -05:00
Nick Ethier
2667f48a5d
docker: move config RPCs to config.go
2018-11-19 22:59:18 -05:00
Nick Ethier
aa9f45ae47
docker: fix tests
2018-11-19 22:59:18 -05:00
Nick Ethier
0f03e8f520
docker: remove container pointer from task handle
2018-11-19 22:59:18 -05:00
Nick Ethier
ce4b867d21
docker: move volume driver options to seperate block
2018-11-19 22:59:18 -05:00
Nick Ethier
fca2df3c79
docker: group common config into blocks
2018-11-19 22:59:17 -05:00
Michael Schurter
813341dd59
Apply suggestions from code review
...
Co-Authored-By: nickethier <ncethier@gmail.com>
2018-11-19 22:59:17 -05:00
Nick Ethier
b7bd36db30
docker: remove global pull coordinator
2018-11-19 22:59:17 -05:00
Nick Ethier
f0a86859a0
docker: remove call to global metrics instance
2018-11-19 22:59:17 -05:00
Nick Ethier
8ef73e63ce
docker: moved fingerprint code to it's own file
2018-11-19 22:59:17 -05:00
Nick Ethier
4be8a86ef9
plugins/driver: remove NodeResources from task Resources and use PercentTicks field for docker driver
2018-11-19 22:59:17 -05:00
Nick Ethier
ced5d5c445
docker: move recoverable error proto to shared structs
2018-11-19 22:59:16 -05:00
Nick Ethier
585e468085
docker: implement recover task logic
2018-11-19 22:59:16 -05:00
Nick Ethier
ee51cb6a93
docker: finished porting tests
2018-11-19 22:59:16 -05:00
Nick Ethier
3d7cdea19e
drivers/docker: more work porting tests from old driver plugin
2018-11-19 22:59:16 -05:00
Nick Ethier
8f8698b3e1
docker: started work on porting docker driver to new plugin framework
2018-11-19 22:59:15 -05:00
Nick Ethier
9e64ce7d73
docker: properly launch docker logger process
2018-11-19 22:59:12 -05:00
Michael Schurter
f8cdd561f0
client: interpolate driver configurations
...
Also add missing SetDriverNetwork calls.
2018-11-15 16:25:57 -08:00
Mahmood Ali
1f4db08f42
Regenerate proto files with protoc-gen-go@v1.2.0
2018-11-14 09:01:26 -05:00
Mahmood Ali
1e92161f14
Merge pull request #4858 from hashicorp/b-fix-master-20181109
...
Fix some tests in master
2018-11-13 16:08:26 -05:00
Alex Dadgar
17e8446484
Merge pull request #4868 from hashicorp/b-plugin-ctx
...
Plugin client's handle plugin dying
2018-11-13 10:26:53 -08:00
Mahmood Ali
356c194acc
Use materialized duration fields for driver config
2018-11-13 10:21:40 -05:00
Mahmood Ali
470d20cdf3
Avoid downloading image if present locally
2018-11-13 10:21:40 -05:00
Mahmood Ali
7690f389a0
Prioritize checking consumer context cancellation
...
Tests expect that as soon as eventer shuts down immediately on context
cancellations; but golang does not guarantee priority when multiple
pending channels are ready in a select statement.
2018-11-13 10:21:40 -05:00
Mahmood Ali
e9067e52b4
pull alpine image needed for test
...
The test requires the image to be present locally, so importing it as
part of setup.
2018-11-13 10:21:40 -05:00
Mahmood Ali
8923ea4663
Handle time.Duration in mock
...
Mock driver config uses `time.Duration` fields but we initialize them
inconsistently, as time.Duration sometimes and as duration strings other
times. Previously, `mapstructure` handles it and does the right thing.
This is no longer the case with MsgPack. I could not find a good way to
bring back old behavior without too much complexity. `MsgPack` extended
types weren't ideal here as we lose type information (e.g. int64 vs
string), and the input is a generic map and not a MsgPack serialization
of duration.
As such, I went with the simple solution of declaring the config field
as duration string, and panicing if the test doesn't pass a valid
string.
I found this to cause the smallest change in tests, but we can
alternatively force all to be int64 instead.
2018-11-13 10:21:40 -05:00
Mahmood Ali
fb56dd699d
distinguish java driver tests from others
2018-11-13 10:21:40 -05:00
Mahmood Ali
343df28165
Fix java driver tests
2018-11-13 10:21:40 -05:00
Mahmood Ali
e506ebbc24
Merge pull request #4845 from hashicorp/r-exec-refactor
...
Update exec driver to match rawexec
2018-11-12 20:59:32 -05:00
Alex Dadgar
693f244cce
Plugin client's handle plugin dying
...
This PR plumbs the plugins done ctx through the base and driver plugin
clients (device already had it). Further, it adds generic handling of
gRPC stream errors.
2018-11-12 17:09:27 -08:00
Danielle Tomlinson
880e5015f2
rawexec: Only use cgroups when running as root.
...
If Nomad is not running as root, we should not try to use cgroups for pid
freezing.
This originally was implemented pre-driver-support in
https://github.com/hashicorp/nomad/blob/v0.8.6/client/driver/raw_exec.go#L120-L130
2018-11-10 06:45:11 -08:00
Mahmood Ali
9c76a7d9b8
handle nil resources limits
...
Config may not express resource constraints on the job!
2018-11-09 15:52:44 -05:00
Mahmood Ali
1ed53ca478
shorter health descriptions
2018-11-08 10:00:49 -08:00
Mahmood Ali
032f86bc78
Add a helper functions for checking unix root
2018-11-08 10:00:49 -08:00
Mahmood Ali
21d126cd68
Update exec driver to match rawexec
...
Update exec driver with latest patterns used by rawexec, and eliminate
accidental diffs.
2018-11-08 10:00:49 -08:00
Alex Dadgar
a7ca737fb6
review comments
2018-11-07 11:31:52 -08:00
Alex Dadgar
204ca8230c
Device manager
...
Introduce a device manager that manages the lifecycle of device plugins
on the client. It fingerprints, collects stats, and forwards Reserve
requests to the correct plugin. The manager, also handles device plugins
failing and validates their output.
2018-11-07 10:43:15 -08:00
Mahmood Ali
32283b2a66
reconcile java with latest patterns
2018-11-06 12:49:29 -08:00
Mahmood Ali
467360954a
driver/java: support non-linux targets
...
Java driver supports non-linux targets, so LinuxResources fields isn't
guaranteed to be not-nil.
2018-11-06 12:41:39 -08:00
Mahmood Ali
e64ef839b8
Reconcile with latest master patterns
2018-11-06 12:41:39 -08:00
Mahmood Ali
c3881bd8d1
add java driver tests
2018-11-06 12:41:39 -08:00
Mahmood Ali
4f525d0a17
Implement java driver
2018-11-06 12:41:39 -08:00
Michael Schurter
392d548b85
Merge pull request #4828 from hashicorp/b-restore
...
Implement client agent restarting
2018-11-05 18:50:15 -06:00
Michael Schurter
2bbd88888c
client: first pass at implementing task restoring
...
Task restoring works but dead tasks may be restarted
2018-11-05 12:32:05 -08:00
Mahmood Ali
a17521475d
Merge pull request #4826 from hashicorp/b-driver-exec-tweaks-20181031
...
Register exec driver plugin among some fixes
2018-11-02 10:11:05 -04:00
Mahmood Ali
dc0fc442ab
add plugin config
2018-11-02 08:51:23 -04:00
Michael Schurter
83db3e6a50
rkt: standardize names/code
2018-11-01 15:53:07 -07:00
Michael Schurter
1a7fb51285
rawexec: standardize names/code
2018-11-01 15:53:07 -07:00
Michael Schurter
5a5d52023d
qemu: standardize names/code
2018-11-01 15:53:07 -07:00
Michael Schurter
3c5e606ef6
drivers/mock: standardize names/code
2018-10-31 11:52:51 -07:00
Mahmood Ali
9a45837c4e
Register exec driver plugin among some fixes
...
Namely, remove the `enabled` configuration flag, as it's specific to
`raw_exec` driver. Also, pass resource limits to underlying call.
2018-10-31 13:55:59 -04:00
Michael Schurter
d3dafc9761
mock_driver: match other's fingerprint message
2018-10-30 17:38:23 -07:00
Michael Schurter
2a38f575c0
rkt: fix race, register by default, fix fingerprint
2018-10-30 17:37:00 -07:00
Michael Schurter
5b0b75e1ee
qemu: fix fingerprint description and handle race
2018-10-30 17:34:08 -07:00
Michael Schurter
6f4a28a7c2
rawexec: fix fingerprint results and tests
...
In tests:
* always cleanup (Kill) the harness
* don't call SetConfig more than once (a race)
* avoid cgroups to allow running without being root
2018-10-30 17:16:22 -07:00
Michael Schurter
616801417c
rawexec: fix race around h.procState
2018-10-30 17:15:47 -07:00
Michael Schurter
9fa53565ad
drivers: remove stutter from exported driver names
...
Also fix a comment on the logger that got copy/pasted around.
2018-10-30 14:05:31 -07:00
Michael Schurter
fc320906aa
qemu: fix build and register by default
...
API changes had broken the build.
2018-10-29 22:10:37 -07:00
Nick Ethier
3fcf8ba7e6
Merge pull request #4795 from hashicorp/f-plugin-config
...
Pass client configuration to plugins through loader
2018-10-29 18:42:27 -07:00
Nick Ethier
bda3b1d3b3
rename NomadConfig to ClientAgentConfig
2018-10-29 21:34:34 -04:00
Preetha Appan
68991d8acb
Merge branch 'f-driver-plugin-qemu' of github.com:hashicorp/nomad into f-driver-plugin-qemu
2018-10-25 13:51:58 -05:00
Preetha Appan
bfff771416
remove driver config struct from qemu driver since its not used
2018-10-25 13:51:06 -05:00
Michael Schurter
48d25dbef1
Fix hclog log line formatting
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:47:17 -05:00
Preetha Appan
95a1637741
Fix imports
2018-10-25 13:45:44 -05:00
Michael Schurter
ca151a7828
Fix copy pasted comment
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:43:29 -05:00
Michael Schurter
c85b90f1e4
remove newline
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:42:20 -05:00
Michael Schurter
66c15e3a67
update comment
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:41:24 -05:00
Michael Schurter
9ca6c998c7
Remove unnecessary return
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:40:28 -05:00
Michael Schurter
fc7ba5e79d
remove unnecessary return
...
Co-Authored-By: preetapan <preetha@hashicorp.com>
2018-10-25 13:40:15 -05:00
Nick Ethier
9916b9e0ce
drivers: update SetConfig to handle driver client config
2018-10-18 23:32:17 -04:00
Michael Schurter
af3e9adf2f
Update drivers/qemu/driver.go
2018-10-17 17:42:01 -05:00
Preetha Appan
93eb048b87
QEMU driver plugin
2018-10-17 11:43:15 -05:00
Alex Dadgar
4b694b2dec
Merge pull request #4793 from hashicorp/b-no-cty
...
Do not use cty in drivers
2018-10-16 20:04:41 -07:00
Nick Ethier
823a6c8d6c
update rkt and exec setconfig funcs
2018-10-16 23:00:26 -04:00
Nick Ethier
65adb80ebf
plumb NomadConfig into plugins
2018-10-16 22:47:22 -04:00
Nick Ethier
b657c3909a
drivers/exec: better fingerprinting and detection on non linux
2018-10-16 22:45:33 -04:00
Nick Ethier
d94b631b6b
drivers/exec: add exec implementation
2018-10-16 22:45:28 -04:00
Alex Dadgar
88c1885044
Do not use cty in drivers
2018-10-16 17:17:07 -07:00
Michael Schurter
838ddf4d4a
fix linter errors
2018-10-16 16:56:57 -07:00
Nick Ethier
448a709603
driver/docker: rename container logger
2018-10-16 16:56:56 -07:00
Michael Schurter
64605a3957
drivers/mock: fix plugin name
...
Was mock_driver before plugins, so keep the name.
2018-10-16 16:56:56 -07:00
Preetha Appan
d634157158
Address review comments around logging task properly
2018-10-16 16:56:56 -07:00
Preetha Appan
3280727c4e
Address review comments
2018-10-16 16:56:56 -07:00
Preetha Appan
02b1f26d34
Switch back to using map[string]string for port map
2018-10-16 16:56:56 -07:00
Michael Schurter
ed5641055f
fixup comments, logging, and missing method impls
...
from #4777 comments
2018-10-16 16:56:56 -07:00
Michael Schurter
1a29337e48
register drivers by default
...
Do not register mock_driver on release builds.
2018-10-16 16:56:56 -07:00
Michael Schurter
118ce7c782
drivers/mock: complete plugin impl
2018-10-16 16:56:56 -07:00
Nick Ethier
9f564781ce
drivers/mock: start mock driver implementation
2018-10-16 16:56:56 -07:00
Preetha Appan
9b3e82a4e7
make port map a slice of maps to match existing rkt driver
2018-10-16 16:56:56 -07:00
Preetha Appan
4f4777d6a6
Review comments
2018-10-16 16:56:56 -07:00
Preetha Appan
edad5e9e6d
Stats collection test
2018-10-16 16:56:56 -07:00
Preetha Appan
678072ecd1
RKT driver plugin and unit tests
2018-10-16 16:56:56 -07:00
Nick Ethier
c7ac1186c9
client: add test for driverfailure during fingerprinting
2018-10-16 16:56:56 -07:00
Nick Ethier
15e8477c4e
rkt: start rkt driver plugin
2018-10-16 16:56:56 -07:00
Preetha Appan
c2d2bfa06c
Get raw exec tests compiling and passing again
2018-10-16 16:56:56 -07:00
Nick Ethier
3183b33d24
client: review comments and fixup/skip tests
2018-10-16 16:56:56 -07:00
Nick Ethier
64e9fc3798
docklog: add go-plugin for forwarding of docker logs
2018-10-16 16:56:56 -07:00
Nick Ethier
4a4c7dbbfc
client: begin driver plugin integration
...
client: fingerprint driver plugins
2018-10-16 16:56:56 -07:00
Alex Dadgar
7946a14aa8
Fix lints
2018-10-16 16:56:56 -07:00
Nick Ethier
5fb95c1a00
drivers/shared: added func comment to eventer
2018-10-16 16:56:55 -07:00
Nick Ethier
951a084198
drivers/shared: move eventer to subpackage under drivers shared package
2018-10-16 16:56:55 -07:00
Nick Ethier
969ecea6f0
drivers/utils: better handling of consumer cleanup in eventer
2018-10-16 16:56:55 -07:00
Nick Ethier
8cf32eb9d2
drivers/rawexec: PR comments and feedback
2018-10-16 16:56:55 -07:00
Nick Ethier
a95cbc38ba
drivers/raw_exec: sync access to task state
2018-10-16 16:53:31 -07:00
Nick Ethier
28e8e5852c
drivers/raw_exec: added unix specific tests
2018-10-16 16:53:31 -07:00
Nick Ethier
1f6873806e
raw_exec: move package outside of plugins dir
2018-10-16 16:53:31 -07:00