Commit graph

41 commits

Author SHA1 Message Date
Mahmood Ali 64f80343fc drivers: re-export ResourceUsage structs
Re-export the ResourceUsage structs in drivers package to avoid drivers
directly depending on the internal client/structs package directly.

I attempted moving the structs to drivers, but that caused some import
cycles that was a bit hard to disentagle.  Alternatively, I added an
alias here that's sufficient for our purposes of avoiding external
drivers depend on internal packages, while allowing us to restructure
packages in future without breaking source compatibility.
2019-01-08 09:11:47 -05:00
Mahmood Ali 916a40bb9e move cstructs.DeviceNetwork to drivers pkg 2019-01-08 09:11:47 -05:00
Mahmood Ali 9369b123de use drivers.FSIsolation 2019-01-08 09:11:47 -05:00
Danielle Tomlinson a9b9ad34dc drivers: Implement InternalPluginDriver interface
This implements the InternalPluginDriver interface in each driver, and
calls the cancellation fn for their respective eventers.

This fixes a per task goroutine leak during test suite execution.
2019-01-08 13:49:31 +01:00
Alex Dadgar c9825a9c36 recover 2019-01-07 14:49:40 -08:00
Preetha Appan 2fb2de3cef
Standardize driver health description messages for all drivers 2019-01-06 22:06:38 -06:00
Mahmood Ali d23f47736c drivers/exec: run as nobody by default
libcontainer based drivers (e.g. exec, java) should default to running
processes as `nobody` [1]; but libcontainer treats empty user as `root`
in our case (either because of default or due to `root` being current
user).

[1] 94c28a4c6c/website/source/docs/job-specification/task.html.md (task-parameters)
2019-01-03 14:29:18 -05:00
Danielle Tomlinson 7b31027ea3 exec: Return undetected when not running as root
This is an unrecoverable error, so we should only do this check once,
rather than returning unhealthy constantly.
2018-12-20 12:54:19 +01:00
Alex Dadgar 4c57d2ec4d Add plugin API versioning to plugin loader and plugins 2018-12-18 16:48:00 -08:00
Alex Dadgar 327b551b39 Drivers 2018-12-18 15:50:11 -08:00
Nick Ethier 09dadf0a23
Merge branch 'master' into f-grpc-executor
* master: (71 commits)
  Fix output of 'nomad deployment fail' with no arg
  Always create a running allocation when testing task state
  tests: ensure exec tests pass valid task resources (#4992)
  some changes for more idiomatic code
  fix iops related tests
  fixed bug in loop delay
  gofmt
  improved code for readability
  client: updateAlloc release lock after read
  fixup! device attributes in `nomad node status -verbose`
  drivers/exec: support device binds and mounts
  fix iops bug and increase test matrix coverage
  tests: tag image explicitly
  changelog
  ci: install lxc-templates explicitly
  tests: skip checking rdma cgroup
  ci: use Ubuntu 16.04 (Xenial) in TravisCI
  client: update driver info on new fingerprint
  drivers/docker: enforce volumes.enabled (#4983)
  client: Style: use fluent style for building loggers
  ...
2018-12-13 14:41:09 -05:00
Mahmood Ali f0ec27da3c
tests: ensure exec tests pass valid task resources (#4992)
Prior to 97f33bb1537d04905cb84199672bcdf46ebb4e65, executor cgroup validation errors were
silently ignored.  Enforcing them reveals test cases that missed them.

This doesn't change customer facing contract, as resource struct is
is either configured or we default to 100 (much higher than 2).
2018-12-12 20:40:38 -05:00
Mahmood Ali 74bd0be6ea drivers/exec: support device binds and mounts 2018-12-11 18:35:21 -05:00
Mahmood Ali e6e71fb47a tests: skip checking rdma cgroup
rdma was added in most recent kernels and libcontainer/docker
don't isolate them by default.
2018-12-11 15:49:11 -05:00
Mahmood Ali 97829a3f02 fix dtestutil.NewDriverHarness ref 2018-12-08 09:58:23 -05:00
Mahmood Ali 021d3720b5
Merge pull request #4950 from hashicorp/b-exc-libcontainer-kill
executor: kill all container processes
2018-12-08 09:52:42 -05:00
Nick Ethier 86e9c11ec2
executor: don't drop errors when configuring libcontainer cfg, add nil check on resources 2018-12-07 14:03:42 -05:00
Nick Ethier 19a695308f
executor: fix tests 2018-12-06 21:39:53 -05:00
Nick Ethier 29ef54c0ee
executor: merge plugin shim with executor package 2018-12-06 21:13:45 -05:00
Nick Ethier 71353a88d4
executor: remove structs package 2018-12-06 20:54:14 -05:00
Nick Ethier 02f4b0fac5
executor: update driver references 2018-12-05 11:04:18 -05:00
Mahmood Ali 3a18105d06 drivers/exec: refactor stop/kill tests
Simplify the tests to do all assertions within the main goroutine and
account for status propagation delay.
2018-12-04 20:34:43 -05:00
Mahmood Ali bd8e4f1c15 Test Stopping a multi-process exec
Ensure that exec children processes get killed as well.
2018-12-03 20:40:19 -05:00
Danielle Tomlinson 393b76ed7f plugins: Move driver testing support to subpackage
this allows us to drop a cyclical import, but is subobptimal as it
requires BaseDriver tests to move. This falls firmly into the realm of
being a hack. Alternatives welcome.
2018-12-01 17:29:39 +01:00
Danielle Tomlinson fdfe93aa25 fixup: executorplugin: fix rkt build 2018-11-30 10:47:08 +01:00
Danielle Tomlinson 0544a57abe drivers: Move client/drivers/executor to drivers/shared/executor 2018-11-30 10:46:13 +01:00
Preetha Appan f89dbcd9cc
modify fingerprint interface to use typed attribute struct 2018-11-28 10:01:03 -06:00
Alex Dadgar 17e8446484
Merge pull request #4868 from hashicorp/b-plugin-ctx
Plugin client's handle plugin dying
2018-11-13 10:26:53 -08:00
Alex Dadgar 693f244cce Plugin client's handle plugin dying
This PR plumbs the plugins done ctx through the base and driver plugin
clients (device already had it). Further, it adds generic handling of
gRPC stream errors.
2018-11-12 17:09:27 -08:00
Mahmood Ali 9c76a7d9b8 handle nil resources limits
Config may not express resource constraints on the job!
2018-11-09 15:52:44 -05:00
Mahmood Ali 1ed53ca478 shorter health descriptions 2018-11-08 10:00:49 -08:00
Mahmood Ali 032f86bc78 Add a helper functions for checking unix root 2018-11-08 10:00:49 -08:00
Mahmood Ali 21d126cd68 Update exec driver to match rawexec
Update exec driver with latest patterns used by rawexec, and eliminate
accidental diffs.
2018-11-08 10:00:49 -08:00
Alex Dadgar 204ca8230c Device manager
Introduce a device manager that manages the lifecycle of device plugins
on the client. It fingerprints, collects stats, and forwards Reserve
requests to the correct plugin. The manager, also handles device plugins
failing and validates their output.
2018-11-07 10:43:15 -08:00
Mahmood Ali dc0fc442ab add plugin config 2018-11-02 08:51:23 -04:00
Mahmood Ali 9a45837c4e Register exec driver plugin among some fixes
Namely, remove the `enabled` configuration flag, as it's specific to
`raw_exec` driver.  Also, pass resource limits to underlying call.
2018-10-31 13:55:59 -04:00
Nick Ethier bda3b1d3b3
rename NomadConfig to ClientAgentConfig 2018-10-29 21:34:34 -04:00
Nick Ethier 9916b9e0ce
drivers: update SetConfig to handle driver client config 2018-10-18 23:32:17 -04:00
Nick Ethier 823a6c8d6c
update rkt and exec setconfig funcs 2018-10-16 23:00:26 -04:00
Nick Ethier b657c3909a
drivers/exec: better fingerprinting and detection on non linux 2018-10-16 22:45:33 -04:00
Nick Ethier d94b631b6b
drivers/exec: add exec implementation 2018-10-16 22:45:28 -04:00