204ca8230c
Device manager
...
Introduce a device manager that manages the lifecycle of device plugins
on the client. It fingerprints, collects stats, and forwards Reserve
requests to the correct plugin. The manager, also handles device plugins
failing and validates their output.
2018-11-07 10:43:15 -08:00
feb83a2be3
assign devices
2018-11-07 10:32:03 -08:00
53543b3e32
register the java plugin
2018-11-06 12:41:39 -08:00
392d548b85
Merge pull request #4828 from hashicorp/b-restore
...
Implement client agent restarting
2018-11-05 18:50:15 -06:00
d29d09023e
client: do not run terminal allocs
2018-11-05 12:32:05 -08:00
2bbd88888c
client: first pass at implementing task restoring
...
Task restoring works but dead tasks may be restarted
2018-11-05 12:32:05 -08:00
a17521475d
Merge pull request #4826 from hashicorp/b-driver-exec-tweaks-20181031
...
Register exec driver plugin among some fixes
2018-11-02 10:11:05 -04:00
9a45837c4e
Register exec driver plugin among some fixes
...
Namely, remove the `enabled` configuration flag, as it's specific to
`raw_exec` driver. Also, pass resource limits to underlying call.
2018-10-31 13:55:59 -04:00
2a38f575c0
rkt: fix race, register by default, fix fingerprint
2018-10-30 17:37:00 -07:00
ac12ae1729
plugins: SetConfig shouldn't panic on nil driver configs
2018-10-30 17:14:27 -07:00
eb67e1db36
drivers: only log non-cancellation errors
2018-10-30 17:13:35 -07:00
fc320906aa
qemu: fix build and register by default
...
API changes had broken the build.
2018-10-29 22:10:37 -07:00
3fcf8ba7e6
Merge pull request #4795 from hashicorp/f-plugin-config
...
Pass client configuration to plugins through loader
2018-10-29 18:42:27 -07:00
87ee46214f
plugins/drivers: nil gaurd driver config when building executor config
2018-10-29 21:43:19 -04:00
bda3b1d3b3
rename NomadConfig to ClientAgentConfig
2018-10-29 21:34:34 -04:00
5d71e880a4
plugins/drivers: fix panic when copying a nil TaskHandle
2018-10-19 09:45:45 -07:00
58b430edae
added driver specific client config struct to plugin configuration
2018-10-18 23:31:01 -04:00
4b694b2dec
Merge pull request #4793 from hashicorp/b-no-cty
...
Do not use cty in drivers
2018-10-16 20:04:41 -07:00
823a6c8d6c
update rkt and exec setconfig funcs
2018-10-16 23:00:26 -04:00
cbf81d3564
test for setting NomadConfig in plugin loader
2018-10-16 22:47:26 -04:00
65adb80ebf
plumb NomadConfig into plugins
2018-10-16 22:47:22 -04:00
b657c3909a
drivers/exec: better fingerprinting and detection on non linux
2018-10-16 22:45:33 -04:00
d94b631b6b
drivers/exec: add exec implementation
2018-10-16 22:45:28 -04:00
88c1885044
Do not use cty in drivers
2018-10-16 17:17:07 -07:00
2b1b3d7e1e
tests: get tests building if not yet passing
2018-10-16 16:56:57 -07:00
fd2db90569
plugins: msgpack codec should use cty tags
2018-10-16 16:56:56 -07:00
1a29337e48
register drivers by default
...
Do not register mock_driver on release builds.
2018-10-16 16:56:56 -07:00
4f4777d6a6
Review comments
2018-10-16 16:56:56 -07:00
678072ecd1
RKT driver plugin and unit tests
2018-10-16 16:56:56 -07:00
c7ac1186c9
client: add test for driverfailure during fingerprinting
2018-10-16 16:56:56 -07:00
c2d2bfa06c
Get raw exec tests compiling and passing again
2018-10-16 16:56:56 -07:00
ed3cdaf3d1
plugin/driver: add Copy funcs
2018-10-16 16:56:56 -07:00
4a4c7dbbfc
client: begin driver plugin integration
...
client: fingerprint driver plugins
2018-10-16 16:56:56 -07:00
7946a14aa8
Fix lints
2018-10-16 16:56:56 -07:00
951a084198
drivers/shared: move eventer to subpackage under drivers shared package
2018-10-16 16:56:55 -07:00
969ecea6f0
drivers/utils: better handling of consumer cleanup in eventer
2018-10-16 16:56:55 -07:00
8d04ae860d
plugins/drivers: remove bool to track if eventLoop shutdown and use context instead
2018-10-16 16:56:55 -07:00
8cf32eb9d2
drivers/rawexec: PR comments and feedback
2018-10-16 16:56:55 -07:00
5742a6b932
plugin/drivers: rework eventer and change naming stream -> consumer
2018-10-16 16:56:55 -07:00
84ce8c3487
extra logging
2018-10-16 16:56:55 -07:00
6f0ed6184b
Fix client reloading and pass the plugin loaders to server and client
2018-10-16 16:56:55 -07:00
183561cf82
Plugin loader initialization
2018-10-16 16:54:12 -07:00
cc76555814
Internal plugin catalog
2018-10-16 16:53:31 -07:00
352c05cdf4
plugin/drivers: plumb in stdout/stderr paths
2018-10-16 16:53:31 -07:00
1f6873806e
raw_exec: move package outside of plugins dir
2018-10-16 16:53:31 -07:00
8b876e1cce
fix package references after drivers/base subpackage removed
2018-10-16 16:53:31 -07:00
0e3f85222a
driver/raw_exec: port existing raw_exec tests and add some testing utilities
2018-10-16 16:53:31 -07:00
8644e8508c
driver/raw_exec: export driver config fields so they are encoded
2018-10-16 16:53:31 -07:00
3c17f50b29
lint: remove unused code and fix spelling
2018-10-16 16:53:31 -07:00
d9628ff394
driver/raw_exec: more tests and bug fixes
...
added wrapper struct for plugin.ReattachConfig to better handle serialization
2018-10-16 16:53:31 -07:00
5617f3615b
driver/raw_exec: initial raw_exec implementation
2018-10-16 16:53:31 -07:00
bcc5c4a8bd
clientv2: base driver plugin ( #4671 )
...
Driver plugin framework to facilitate development of driver plugins.
Implementing plugins only need to implement the DriverPlugin interface.
The framework proxies this interface to the go-plugin GRPC interface generated
from the driver.proto spec.
A testing harness is provided to allow implementing drivers to test the full
lifecycle of the driver plugin. An example use:
func TestMyDriver(t *testing.T) {
harness := NewDriverHarness(t, &MyDiverPlugin{})
// The harness implements the DriverPlugin interface and can be used as such
taskHandle, err := harness.StartTask(...)
}
2018-10-16 16:53:31 -07:00
3e7008c2dc
Add BlockAttrs
2018-10-15 20:47:38 -07:00
06719d0b5a
Merge pull request #4783 from hashicorp/f-device-scheduling
...
Device constraint checking
2018-10-15 15:31:32 -07:00
b94505b256
Merge pull request #4780 from hashicorp/f-device-attributes
...
Device fingerprinting uses Attribute object
2018-10-15 13:37:49 -07:00
4098d2e99c
fix typos
2018-10-15 13:15:58 -07:00
7ecd65109a
Check constraints on devices
2018-10-14 13:35:47 -07:00
f9b056e1d1
Replace attributes map with new Attribute object
2018-10-13 14:08:58 -07:00
5fc9a95201
Use Attribute in device fingerprinting
2018-10-13 11:43:06 -07:00
c31a234fb3
fmt -s
2018-10-12 15:34:21 -07:00
cbb5f21112
New parser and comparison
2018-10-12 15:25:34 -07:00
364180b396
Units defined and parsing
2018-10-11 23:30:28 -07:00
0183fb4e5c
nvidia package restructue + build non-linux
2018-10-05 13:56:04 -07:00
118e3fe7e9
Introduce nvidia-plugin reserve
...
- added reserve functionality that returns OCI compliant env variables
specifying GPU IDs to be injected inside the container
2018-10-04 14:55:34 -07:00
0ea1dc1776
Introduce Nvidia-plugin stats
...
- created go-nvml wrapper for stats
- added stats feature to nvidia-plugin
2018-10-03 15:12:05 -07:00
b4a4b395e3
Introduce nvidia-plugin fingerprinting
...
- created go-nvml wrapper for fingerprinting
- added fingerprinting feature to nvidia-plugin
2018-10-03 15:11:56 -07:00
ceb0fa8d13
fix documentation of reattach and use testlog
2018-09-18 10:48:37 -07:00
8236478815
singleton wrapper
2018-09-18 10:08:46 -07:00
ca28afa3b2
small fixes
2018-09-15 16:42:38 -07:00
7552112a0d
review comments
2018-09-10 17:29:28 -07:00
643b9e8ced
Plugin loader
2018-09-09 16:41:47 -07:00
c6576ddac1
Fix make check errors
2018-09-04 16:03:52 -07:00
2c2a8322b5
Fix device launcher ctx cleanup
2018-09-04 15:48:38 -07:00
6a1db58593
Update README.md
2018-08-28 15:08:59 -07:00
b7f3e24391
launcher readme
2018-08-28 15:00:48 -07:00
e14fa23961
Add stats to launcher
2018-08-28 13:59:57 -07:00
3e3ca1ac04
Add stats to example plugin
2018-08-28 13:59:57 -07:00
83ad5afb69
Example device plugin and helpers
...
This PR introduces:
* An example device plugin that exposes files as devices for testing.
* Helpers for serving a device plugin
* A launcher binary that allows interacting with a device plugin without
needing a Nomad client.
2018-08-28 13:59:57 -07:00
cefa806519
gofmt -s
2018-08-28 10:32:03 -07:00
c394798106
add stats to device interface
2018-08-27 16:11:07 -07:00
1eb043a46c
statistics protos
2018-08-23 11:18:35 -07:00
6f1b5e2921
lint: fix comment spelling
2018-08-22 22:47:38 -04:00
f14c8a223e
driver/base: fix proto pr comments
2018-08-22 22:20:46 -04:00
b3b80b380c
driver/base: add size on disk field for task
2018-08-22 22:20:46 -04:00
a005792f98
driver/base: fixup comments from code review
2018-08-22 22:20:46 -04:00
3a817dc7c9
driver/base: add task event emitting to proto
2018-08-22 22:20:46 -04:00
a2c32dda57
driver/base: Add initial protobuf spec
2018-08-22 22:20:46 -04:00
9ac7dcd1ee
Describe public interface of Fingerprint
2018-08-20 15:19:08 -07:00
98a67938a0
initial device go-plugin
2018-08-16 16:49:09 -07:00
c49beed26e
vet
2018-08-13 11:10:33 -07:00
19cffc9a3a
Review comments
2018-08-13 11:01:44 -07:00
c324e98a25
Initial go-plugin
2018-08-12 15:58:39 -07:00
bb20dbde43
move protos
2018-08-10 11:16:55 -07:00
42b432d18d
block attr spec
2018-08-10 11:14:12 -07:00
d6b291b00d
hcl interface to cty function
2018-08-10 10:51:23 -07:00
720cc4d90c
Fix protos
2018-08-09 13:29:05 -07:00
a355b903a6
SetConfig
2018-08-07 16:16:23 -07:00
ca121ce110
review feedback
2018-08-07 13:03:21 -07:00
97ccf42c64
Base and device plugin and hclspec parser
2018-08-07 13:03:21 -07:00
Alex Dadgar