luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
15959 commits 1 branch 0 tags 311 MiB
Commit graph

142 commits

Author SHA1 Message Date
Alex Dadgar c9825a9c36 recover 2019-01-07 14:49:40 -08:00
Alex Dadgar a6b36df4de remove nil logger 2019-01-07 14:48:01 -08:00
Preetha Appan 2fb2de3cef
Standardize driver health description messages for all drivers 2019-01-06 22:06:38 -06:00
Danielle Tomlinson 43f2dc0c36 chore: Fix environement->environment typo 2019-01-03 13:31:26 +01:00
Danielle Tomlinson 45174ac3e9
Merge pull request #5041 from hashicorp/dani/b-driver-healt 
drivers: Cleanup root user fingerprinting
 2019-01-03 13:16:28 +01:00
Danielle Tomlinson 63b5e1a9e9 plugins: Add consistent message for requires root 2018-12-20 12:54:01 +01:00
Alex Dadgar 9d34802f7a Store device envs separately and pass to drivers 2018-12-19 14:23:09 -08:00
Alex Dadgar fff09162aa proto 2018-12-19 13:54:19 -08:00
Nick Ethier ce1a5cba0e
drivermanager: use allocID and task name to route task events 2018-12-18 23:01:51 -05:00
Alex Dadgar 4c57d2ec4d Add plugin API versioning to plugin loader and plugins 2018-12-18 16:48:00 -08:00
Alex Dadgar b653ae2af7 utilities 2018-12-18 15:48:52 -08:00
Alex Dadgar b9ee03b2c1 protos 2018-12-18 15:48:52 -08:00
Nick Ethier 0c50a51c19
executor: encode mounts and devices correctly when using grpc 2018-12-15 00:08:23 -05:00
Nick Ethier 09dadf0a23
Merge branch 'master' into f-grpc-executor 
* master: (71 commits)
  Fix output of 'nomad deployment fail' with no arg
  Always create a running allocation when testing task state
  tests: ensure exec tests pass valid task resources (#4992)
  some changes for more idiomatic code
  fix iops related tests
  fixed bug in loop delay
  gofmt
  improved code for readability
  client: updateAlloc release lock after read
  fixup! device attributes in `nomad node status -verbose`
  drivers/exec: support device binds and mounts
  fix iops bug and increase test matrix coverage
  tests: tag image explicitly
  changelog
  ci: install lxc-templates explicitly
  tests: skip checking rdma cgroup
  ci: use Ubuntu 16.04 (Xenial) in TravisCI
  client: update driver info on new fingerprint
  drivers/docker: enforce volumes.enabled (#4983)
  client: Style: use fluent style for building loggers
  ...
 2018-12-13 14:41:09 -05:00
Nick Ethier 86e9c11ec2
executor: don't drop errors when configuring libcontainer cfg, add nil check on resources 2018-12-07 14:03:42 -05:00
Nick Ethier 2283cb2c39
executor: use drivers.Resources as resource model 2018-12-06 21:22:02 -05:00
Nick Ethier 29ef54c0ee
executor: merge plugin shim with executor package 2018-12-06 21:13:45 -05:00
Nick Ethier 71353a88d4
executor: remove structs package 2018-12-06 20:54:14 -05:00
Alex Dadgar 1e3c3cb287 Deprecate IOPS 
IOPS have been modelled as a resource since Nomad 0.1 but has never
actually been detected and there is no plan in the short term to add
detection. This is because IOPS is a bit simplistic of a unit to define
the performance requirements from the underlying storage system. In its
current state it adds unnecessary confusion and can be removed without
impacting any users. This PR leaves IOPS defined at the jobspec parsing
level and in the api/ resources since these are the two public uses of
the field. These should be considered deprecated and only exist to allow
users to stop using them during the Nomad 0.9.x release. In the future,
there should be no expectation that the field will exist.
 2018-12-06 15:09:26 -08:00
Nick Ethier 8b20de4801
executor: use grpc instead of netrpc as plugin protocol 
* Added protobuf spec for executor
 * Seperated executor structs into their own package
 2018-12-05 11:03:56 -05:00
Danielle Tomlinson 8ba0a816f3 plugins: Add support for serving driver plugins 2018-12-01 17:30:54 +01:00
Danielle Tomlinson 393b76ed7f plugins: Move driver testing support to subpackage 
this allows us to drop a cyclical import, but is subobptimal as it
requires BaseDriver tests to move. This falls firmly into the realm of
being a hack. Alternatives welcome.
 2018-12-01 17:29:39 +01:00
Danielle Tomlinson 2db5ae38d8 client: Rename drivers/shared/env => client/taskenv 2018-11-30 12:18:39 +01:00
Danielle Tomlinson ffc5e5d56b executors: Unify go-plugin handshake 2018-11-30 10:59:23 +01:00
Danielle Tomlinson fdfe93aa25 fixup: executorplugin: fix rkt build 2018-11-30 10:47:08 +01:00
Danielle Tomlinson d26a310db0 client: Move executor plugins into own package 2018-11-30 10:46:13 +01:00
Danielle Tomlinson d582ea1d8b drivers: Create drivers/shared/structs 
This creates a drivers/shared/structs package and moves the buffer size
checks into it.
 2018-11-30 10:46:13 +01:00
Danielle Tomlinson 0544a57abe drivers: Move client/drivers/executor to drivers/shared/executor 2018-11-30 10:46:13 +01:00
Danielle Tomlinson 1a29811169 drivers: Move client/drivers/env to drivers/shared/env 
As part of deprecating legacy drivers, we're moving the env package to a
new drivers/shared tree, as it is used by the modern docker and rkt
driver packages, and is useful for 3rd party plugins.
 2018-11-30 10:46:13 +01:00
Chris Baker b43090a267
Merge pull request #4932 from hashicorp/b-1172-rkt-env-vars 
change to testing utilities to fix rkt tests
 2018-11-29 09:18:10 -05:00
Chris Baker da35fda145 testing: in MkAllocDir, do not update TaskConfig with All() from the task builder, just with Env() (because it pollutes environment variables with node attributes and fails the rkt tests) 2018-11-28 22:19:48 +00:00
Preetha 1f526db414
Merge pull request #4919 from hashicorp/f-fingerprint-attribute-type 
Modify fingerprint interface to use typed attribute struct
 2018-11-28 14:18:28 -06:00
Preetha Appan f89dbcd9cc
modify fingerprint interface to use typed attribute struct 2018-11-28 10:01:03 -06:00
Alex Dadgar 4ee603c382 Device hook and devices affect computed node class 
This PR introduces a device hook that retrieves the device mount
information for an allocation. It also updates the computed node class
computation to take into account devices.

TODO Fix the task runner unit test. The environment variable is being
lost even though it is being properly set in the prestart hook.
 2018-11-27 17:25:33 -08:00
Chris Baker a1fb1f3830
Merge pull request #4891 from hashicorp/b-1150-rkt-volume-names 
drivers/rkt: fix invalid volumes
 2018-11-27 18:55:00 -05:00
Chris Baker c0bc9d069d change to docs in the driver proto to reflect standard pattern 2018-11-27 23:52:24 +00:00
Preetha Appan b9a22f8047
Fix panic in test setup when task does not have resources 
This affects exec/rawexec drivers
 2018-11-26 21:42:45 -06:00
Preetha Appan 125869686b
Fix nil dereference in copy method 2018-11-26 15:53:15 -06:00
Chris Baker 9bd4317139 modified TaskConfig to include AllocID 
use this for volume names in drivers/rkt to address #1150
 2018-11-26 18:54:26 +00:00
Mahmood Ali 141092e46d Formatting and typo fixes 2018-11-25 11:53:21 -05:00
Nick Ethier 1f3fe02e62
docker: sync access to exit result within a handle 2018-11-20 20:41:32 -05:00
Nick Ethier aa9f45ae47
docker: fix tests 2018-11-19 22:59:18 -05:00
Nick Ethier 4be8a86ef9
plugins/driver: remove NodeResources from task Resources and use PercentTicks field for docker driver 2018-11-19 22:59:17 -05:00
Nick Ethier ced5d5c445
docker: move recoverable error proto to shared structs 2018-11-19 22:59:16 -05:00
Nick Ethier 69049d37f5
drivers: added NodeResources to drivers.TaskConfig 2018-11-19 22:59:16 -05:00
Nick Ethier 3d7cdea19e
drivers/docker: more work porting tests from old driver plugin 2018-11-19 22:59:16 -05:00
Nick Ethier 117b9e6584
drivers: support recoverable errors in the plugin RPC layer 2018-11-19 22:59:15 -05:00
Nick Ethier 8f8698b3e1
docker: started work on porting docker driver to new plugin framework 2018-11-19 22:59:15 -05:00
Mahmood Ali 1f4db08f42 Regenerate proto files with protoc-gen-go@v1.2.0 2018-11-14 09:01:26 -05:00
Alex Dadgar 17e8446484
Merge pull request #4868 from hashicorp/b-plugin-ctx 
Plugin client's handle plugin dying
 2018-11-13 10:26:53 -08:00
Alex Dadgar 693f244cce Plugin client's handle plugin dying 
This PR plumbs the plugins done ctx through the base and driver plugin
clients (device already had it). Further, it adds generic handling of
gRPC stream errors.
 2018-11-12 17:09:27 -08:00
Mahmood Ali 032f86bc78 Add a helper functions for checking unix root 2018-11-08 10:00:49 -08:00
Alex Dadgar 204ca8230c Device manager 
Introduce a device manager that manages the lifecycle of device plugins
on the client. It fingerprints, collects stats, and forwards Reserve
requests to the correct plugin. The manager, also handles device plugins
failing and validates their output.
 2018-11-07 10:43:15 -08:00
Michael Schurter d29d09023e client: do not run terminal allocs 2018-11-05 12:32:05 -08:00
Michael Schurter 2bbd88888c client: first pass at implementing task restoring 
Task restoring works but dead tasks may be restarted
 2018-11-05 12:32:05 -08:00
Michael Schurter eb67e1db36 drivers: only log non-cancellation errors 2018-10-30 17:13:35 -07:00
Nick Ethier 3fcf8ba7e6
Merge pull request #4795 from hashicorp/f-plugin-config 
Pass client configuration to plugins through loader
 2018-10-29 18:42:27 -07:00
Nick Ethier 87ee46214f
plugins/drivers: nil gaurd driver config when building executor config 2018-10-29 21:43:19 -04:00
Nick Ethier bda3b1d3b3
rename NomadConfig to ClientAgentConfig 2018-10-29 21:34:34 -04:00
Michael Schurter 5d71e880a4 plugins/drivers: fix panic when copying a nil TaskHandle 2018-10-19 09:45:45 -07:00
Nick Ethier 58b430edae
added driver specific client config struct to plugin configuration 2018-10-18 23:31:01 -04:00
Nick Ethier 823a6c8d6c
update rkt and exec setconfig funcs 2018-10-16 23:00:26 -04:00
Nick Ethier b657c3909a
drivers/exec: better fingerprinting and detection on non linux 2018-10-16 22:45:33 -04:00
Nick Ethier d94b631b6b
drivers/exec: add exec implementation 2018-10-16 22:45:28 -04:00
Michael Schurter 2b1b3d7e1e tests: get tests building if not yet passing 2018-10-16 16:56:57 -07:00
Preetha Appan 4f4777d6a6 Review comments 2018-10-16 16:56:56 -07:00
Preetha Appan 678072ecd1 RKT driver plugin and unit tests 2018-10-16 16:56:56 -07:00
Nick Ethier c7ac1186c9 client: add test for driverfailure during fingerprinting 2018-10-16 16:56:56 -07:00
Preetha Appan c2d2bfa06c Get raw exec tests compiling and passing again 2018-10-16 16:56:56 -07:00
Nick Ethier ed3cdaf3d1 plugin/driver: add Copy funcs 2018-10-16 16:56:56 -07:00
Nick Ethier 4a4c7dbbfc client: begin driver plugin integration 
client: fingerprint driver plugins
 2018-10-16 16:56:56 -07:00
Alex Dadgar 7946a14aa8 Fix lints 2018-10-16 16:56:56 -07:00
Nick Ethier 951a084198 drivers/shared: move eventer to subpackage under drivers shared package 2018-10-16 16:56:55 -07:00
Nick Ethier 969ecea6f0 drivers/utils: better handling of consumer cleanup in eventer 2018-10-16 16:56:55 -07:00
Nick Ethier 8d04ae860d plugins/drivers: remove bool to track if eventLoop shutdown and use context instead 2018-10-16 16:56:55 -07:00
Nick Ethier 8cf32eb9d2 drivers/rawexec: PR comments and feedback 2018-10-16 16:56:55 -07:00
Nick Ethier 5742a6b932 plugin/drivers: rework eventer and change naming stream -> consumer 2018-10-16 16:56:55 -07:00
Nick Ethier 352c05cdf4 plugin/drivers: plumb in stdout/stderr paths 2018-10-16 16:53:31 -07:00
Nick Ethier 1f6873806e raw_exec: move package outside of plugins dir 2018-10-16 16:53:31 -07:00
Nick Ethier 8b876e1cce fix package references after drivers/base subpackage removed 2018-10-16 16:53:31 -07:00
Nick Ethier 0e3f85222a driver/raw_exec: port existing raw_exec tests and add some testing utilities 2018-10-16 16:53:31 -07:00
Nick Ethier 8644e8508c driver/raw_exec: export driver config fields so they are encoded 2018-10-16 16:53:31 -07:00
Nick Ethier 3c17f50b29 lint: remove unused code and fix spelling 2018-10-16 16:53:31 -07:00
Nick Ethier d9628ff394 driver/raw_exec: more tests and bug fixes 
added wrapper struct for plugin.ReattachConfig to better handle serialization
 2018-10-16 16:53:31 -07:00
Nick Ethier 5617f3615b driver/raw_exec: initial raw_exec implementation 2018-10-16 16:53:31 -07:00
Nick Ethier bcc5c4a8bd clientv2: base driver plugin (#4671) 
Driver plugin framework to facilitate development of driver plugins.

Implementing plugins only need to implement the DriverPlugin interface.
The framework proxies this interface to the go-plugin GRPC interface generated
from the driver.proto spec.

A testing harness is provided to allow implementing drivers to test the full
lifecycle of the driver plugin. An example use:

func TestMyDriver(t *testing.T) {
    harness := NewDriverHarness(t, &MyDiverPlugin{})
    // The harness implements the DriverPlugin interface and can be used as such
    taskHandle, err := harness.StartTask(...)
}
 2018-10-16 16:53:31 -07:00
Nick Ethier 6f1b5e2921
lint: fix comment spelling 2018-08-22 22:47:38 -04:00
Nick Ethier f14c8a223e driver/base: fix proto pr comments 2018-08-22 22:20:46 -04:00
Nick Ethier b3b80b380c driver/base: add size on disk field for task 2018-08-22 22:20:46 -04:00
Nick Ethier a005792f98 driver/base: fixup comments from code review 2018-08-22 22:20:46 -04:00
Nick Ethier 3a817dc7c9 driver/base: add task event emitting to proto 2018-08-22 22:20:46 -04:00
Nick Ethier a2c32dda57 driver/base: Add initial protobuf spec 2018-08-22 22:20:46 -04:00