Commit graph

1563 commits

Author SHA1 Message Date
Michael Schurter 1bd9a9f9dd
Merge pull request #4894 from hashicorp/f-device-hook
Device hook and devices affect computed node class
2018-11-28 12:10:43 -06:00
Mahmood Ali ad1f8d8c20 Fixes in old lxc driver 2018-11-27 21:40:43 -05:00
Michael Schurter 85073f9d29 client: properly support hook env vars
The old approach was incomplete. Hook env vars are now:

 * persisted and restored between agent restarts
 * deterministic (LWW if 2 hooks set the same key)
2018-11-27 17:25:33 -08:00
Michael Schurter 2d0b44c3b4 client: test more env key variations 2018-11-15 16:26:25 -08:00
Michael Schurter 5e51e2c2d5 client: turn env into nested objects for task configs 2018-11-15 16:25:57 -08:00
Michael Schurter f8cdd561f0 client: interpolate driver configurations
Also add missing SetDriverNetwork calls.
2018-11-15 16:25:57 -08:00
Danielle Tomlinson 0917e93537
Merge pull request #4869 from hashicorp/b-executor-stdout
executor: Fix stdout stderr copy/paste
2018-11-13 19:22:37 -08:00
Mahmood Ali ac3b4571eb Address review comments 2018-11-13 10:21:40 -05:00
Mahmood Ali 69f26783e4 avoid setting resource limit on rkt command
Was accidentally modified in 5b14d24bf4626bab420d00783d92bcf25e0b641e .
2018-11-13 10:21:40 -05:00
Mahmood Ali 8fa26f5521 Fix docker log fetching in tests
We no longer use syslog for tracking logs so tracking them explicitly
here
2018-11-13 10:21:40 -05:00
Mahmood Ali 88fa968623 killing should be done with wait client
Incidentally changed in 5b14d24bf4626bab420d00783d92bcf25e0b641e
2018-11-13 10:21:40 -05:00
Mahmood Ali e5e6f9a785 Update Docker name parsing lookup
`ParseNamed` function changed in e9f3f2cfee9d729a8642344c4fa4ea70b2d49468
where became `ParsedNormalizedName` with extra checks.
2018-11-13 10:21:40 -05:00
Danielle Tomlinson bfeded1f30 executor: Fix stdout stderr copy/paste 2018-11-12 22:08:04 -08:00
Nick Ethier b0ddc03409
Merge pull request #4765 from jippi/increase-line-scan-limit
fix: increase log rotator line scan limit
2018-10-29 18:46:30 -07:00
Preetha Appan b85cc38f3d
Stat path to binary to handle raw exec driver interpolated binary path 2018-10-26 17:24:05 -05:00
Preetha Appan 55ac8d3d12
Fix test linting 2018-10-26 10:30:12 -05:00
Nick Ethier d94b631b6b
drivers/exec: add exec implementation 2018-10-16 22:45:28 -04:00
Alex Dadgar 89dafaaea9 compile on windows 2018-10-16 16:56:56 -07:00
Alex Dadgar ad4fac526c more test fixes 2018-10-16 16:56:56 -07:00
Michael Schurter f279b1d1b1 tests: test logs endpoint against pending task
Although the really exciting change is making WaitForRunning return the
allocations that it started. This should cut down test boilerplate
significantly.
2018-10-16 16:56:55 -07:00
Michael Schurter 2325348053 mock_driver: close waitCh after exiting
mock_driver wasn't behaving like other driver handles.
2018-10-16 16:56:55 -07:00
Nick Ethier 0e3f85222a driver/raw_exec: port existing raw_exec tests and add some testing utilities 2018-10-16 16:53:31 -07:00
Nick Ethier d9628ff394 driver/raw_exec: more tests and bug fixes
added wrapper struct for plugin.ReattachConfig to better handle serialization
2018-10-16 16:53:31 -07:00
Nick Ethier bcc5c4a8bd clientv2: base driver plugin (#4671)
Driver plugin framework to facilitate development of driver plugins.

Implementing plugins only need to implement the DriverPlugin interface.
The framework proxies this interface to the go-plugin GRPC interface generated
from the driver.proto spec.

A testing harness is provided to allow implementing drivers to test the full
lifecycle of the driver plugin. An example use:

func TestMyDriver(t *testing.T) {
    harness := NewDriverHarness(t, &MyDiverPlugin{})
    // The harness implements the DriverPlugin interface and can be used as such
    taskHandle, err := harness.StartTask(...)
}
2018-10-16 16:53:31 -07:00
Nick Ethier 5dee1141d1 executor v2 (#4656)
* client/executor: refactor client to remove interpolation

* executor: POC libcontainer based executor

* vendor: use hashicorp libcontainer fork

* vendor: add libcontainer/nsenter dep

* executor: updated executor interface to simplify operations

* executor: implement logging pipe

* logmon: new logmon plugin to manage task logs

* driver/executor: use logmon for log management

* executor: fix tests and windows build

* executor: fix logging key names

* executor: fix test failures

* executor: add config field to toggle between using libcontainer and standard executors

* logmon: use discover utility to discover nomad executable

* executor: only call libcontainer-shim on main in linux

* logmon: use seperate path configs for stdout/stderr fifos

* executor: windows fixes

* executor: created reusable pid stats collection utility that can be used in an executor

* executor: update fifo.Open calls

* executor: fix build

* remove executor from docker driver

* executor: Shutdown func to kill and cleanup executor and its children

* executor: move linux specific universal executor funcs to seperate file

* move logmon initialization to a task runner hook

* client: doc fixes and renaming from code review


* taskrunner: use shared config struct for logmon fifo fields

* taskrunner: logmon only needs to be started once per task
2018-10-16 16:53:31 -07:00
Nick Ethier 66ff12e5f7 Update runc/libcontainer and friends (#4655)
* vendor: bump libcontainer and docker to remove Sirupsen imports

* vendor: fix bad vendoring of archive package

* vendor: fix api changes to cgroups in executor

* vendor: fix docker api changes

* vendor: update github.com/Azure/go-ansiterm to use non capitalized logrus import
2018-10-16 16:53:30 -07:00
Michael Schurter 5749ede04e keep forgetting lxc 2018-10-16 16:53:30 -07:00
Michael Schurter 7132b67c1e Add Network method to Handle interface
Should probably be moved to an Inspect method in the Driver Plugin world
2018-10-16 16:53:30 -07:00
Michael Schurter 9a63d6103d tr: add validate task hook 2018-10-16 16:53:29 -07:00
Alex Dadgar b9bed81e6e Initial V2 alloc runner 2018-10-16 16:53:28 -07:00
Alex Dadgar a78cefec18 use int64 2018-10-16 15:34:32 -07:00
Preetha Appan 7c0d8c646c
Change CPU/Disk/MemoryMB to int everywhere in new resource structs 2018-10-16 16:21:42 -05:00
Christian Winther 0c5154100c
fix: increase log rotator line scan limit
In case where gelf/json logging is used, its fairly easy to exceed the 16k limit, resulting in json output being cut up into multiple strings

the result is invalid json lines which can create all kind of badness in the logging server

This fixes https://github.com/hashicorp/nomad/issues/4699

Signed-off-by: Christian Winther <jippignu@gmail.com>
2018-10-09 18:57:18 +02:00
Alex Dadgar 52f9cd7637 fixing tests 2018-10-04 14:26:19 -07:00
Alex Dadgar c6576ddac1 Fix make check errors 2018-09-04 16:03:52 -07:00
Alex Dadgar 089b533047 Fix kill timeout exceeding 5m on Docker driver
Fixes an issue where the Docker API client would timeout before the kill
timeout was hit.
2018-08-17 16:01:09 -07:00
Alex Dadgar 49a1ba9297
Merge pull request #4535 from hashicorp/f-keep-docker-container-0.8.4
Option to prevent removal of container on exit
2018-07-26 11:11:22 -07:00
Charlie Voiselle f319a149cd Option to prevent removal of container on exit 2018-07-26 11:10:48 -07:00
Michael Schurter ddf948001e
Merge pull request #4462 from omame/omame/cpu_cfs_period
Add support for specifying cpu_cfs_period in the Docker driver
2018-07-25 09:34:38 -07:00
Daniele Valeriani b0a14caca2 Add test for cpu_cfs_period 2018-07-16 22:43:34 +02:00
Michael Schurter 91588cb861 rkt: revert to redis 3.2 to favor stability 2018-07-09 16:15:32 -07:00
Michael Schurter c56f899ee9 rkt: speed up tests
Disable networking when it's not needed and improve failure message for
UserGroup test by including the full ps output on failure.
2018-07-09 14:02:27 -07:00
Michael Schurter a1d4f77ce0 rkt: skip retrieving network information when net=none
Even when net=none we would attempt to retrieve network information from
rkt which would spew useless log lines such as:

```
testlog.go:30: 20:37:31.409209 [DEBUG] driver.rkt: failed getting network info for pod UUID 8303cfe6-0c10-4288-84f5-cb79ad6dbf1c attempt 2: no networks found. Sleeping for 970ms
```

It would also delay tests for ~60s during the network information retry
period.

So skip this when net=none. It's unlikely anyone actually uses net=none
outside of tests, so I doubt anyone will notice this change.

Official docs:
https://coreos.com/rkt/docs/latest/networking/overview.html#no-loopback-only-networking
2018-07-09 13:44:43 -07:00
Michael Schurter 0fbc84b81d tests: make alloc id consistent in helper
It worked, but the old code used a different alloc id for the path than
the actual alloc! Use the same alloc id everywhere to prevent confusing
test output.
2018-07-09 13:37:35 -07:00
Michael Schurter f3b8815c96 rkt: fix failing TestRktDriver_UserGroup test
Started failing due to the docker redis image switching from Debian
jessie to stretch:
53f8680550 (diff-acff46b161a3b7d6ed01ba79a032acc9)

Switched from Debian based image to Alpine to get a working `ps` command
again (albeit busybox's stripped down implementation)
2018-07-09 12:19:02 -07:00
Daniele Valeriani 748f6afd89 Validate the value of cpu_cfs_period 2018-07-02 22:30:22 +02:00
Daniele Valeriani 9364446a03 Remove an unnecessary conversion 2018-07-02 17:47:23 +02:00
Daniele Valeriani 906952a2c8 Add support for specifying cpu_cfs_period in the Docker driver 2018-07-02 16:37:04 +02:00
Alex Dadgar 300b1a7a15 Tests only use testlog package logger 2018-06-13 15:40:56 -07:00
Nick Ethier f36eb14360
Merge pull request #4403 from hashicorp/b-fix-dispatched-optional-meta
Fix dispatched optional meta correctly
2018-06-11 16:17:14 -04:00