Commit Graph

13841 Commits

Author SHA1 Message Date
Preetha Appan 469a286b1b
Refactor logging in drivers to use a tri-state boolean
Changes logging warnings/errors only if the state changes
from healthy to unhealthy
2019-01-16 10:19:31 -06:00
Preetha Appan 0c2c0a2d43
Make docker driver logging less redundant 2019-01-16 10:16:57 -06:00
Michael Schurter 282cdde382
Merge pull request #5199 from hashicorp/test-dispatch
test: porting TestTaskRunner_SimpleRun_Dispatch
2019-01-16 06:33:51 -08:00
Michael Schurter daa7d029a1 test: porting TestTaskRunner_SimpleRun_Dispatch
Porting test from 0.8 to 0.9.
2019-01-15 15:22:13 -08:00
Alex Dadgar 5de91cbfb7
Merge pull request #5046 from oleksii-shyman/docker_runtimes
Nvidia runtime
2019-01-15 14:34:14 -08:00
Chris Baker d355650026
Merge pull request #5194 from hashicorp/f-minor-update-docs-tense
docs: updated tense around fixed bug
2019-01-15 14:50:57 -05:00
oleksii.shyman e41fbf7577 Add support for docker runtimes
- docker fingerprint issues a docker api system info call to get the
  list of supported OCI runtimes.
  - OCI runtimes are reported as comma separated list of names
  - docker driver is aware of GPU runtime presence
  - docker driver throws an error when user tries to run container with
  GPU, when GPU runtime is not present
  - docker GPU runtime name is configurable
2019-01-15 11:34:47 -08:00
oleksii.shyman 7393e91145 Update go-dockerclient and docker dependencies
- go-dockerclient is updated because of lack of "runtime" API field
 - docker/docker is updated because go-dockerclient depends on docker
 master branch
2019-01-15 11:28:57 -08:00
Alex Dadgar f349e3f2e5
Merge pull request #5195 from hashicorp/d-update-087-date
make change to the 0.8.7 release date
2019-01-15 09:53:07 -08:00
Cathy Chang 5d1e4b5650 make change to the 0.8.7 release date 2019-01-15 09:47:13 -08:00
Chris Baker 8b15543a14 docs: updated tense around fixed bug 2019-01-15 17:43:03 +00:00
Danielle Tomlinson b918d25e62
Merge pull request #5192 from hashicorp/dani/executor-close
executor: Always close stdout/stderr fifos
2019-01-15 17:49:04 +01:00
Danielle Tomlinson f120c8f8f6
Merge pull request #5184 from hashicorp/dani/b-logmon-reattach
docker: Terminate dockerlogger
2019-01-15 16:48:40 +01:00
Danielle Tomlinson 7f1ff3fab6 executor: Always close stdout/stderr fifos 2019-01-15 16:47:27 +01:00
Michael Schurter 48afda786b
Merge pull request #5187 from hashicorp/test-consul
Port a bunch of pre-0.9 Consul tests to 0.9
2019-01-15 07:41:50 -08:00
Mahmood Ali ce5a4e7b94
Merge pull request #5189 from hashicorp/b-fix-executor-logs
propagate logs to executor plugin
2019-01-15 10:26:01 -05:00
Danielle Tomlinson 272a8726d7 docker: Terminate dockerlogger
Previously, we did not attempt to stop Docker Logger processes until
DestroyTask, which means that under many circumstances, we will never
successfully close the plugin client.

This commit terminates the plugin process when `run` terminates, or when
`DestroyTask` is called.

Steps to repro:

```
$ nomad agent -dev
$ nomad init
$ nomad run example.nomad
$ nomad stop example
$ ps aux | grep nomad # See docker logger process running
$ signal the dev agent
$ ps aux | grep nomad # See docker logger process running
```
2019-01-15 14:58:05 +01:00
Mahmood Ali 5649f72d27 propogate logs to executor plugin 2019-01-15 08:25:03 -05:00
Alex Dadgar 471fdb3ccf
Merge pull request #5173 from hashicorp/b-log-levels
Plugins use parent loggers
2019-01-14 16:14:30 -08:00
Nick Ethier c619e70d39
Merge pull request #5018 from hashicorp/f-executor-stats
executor: streaming stats api
2019-01-14 15:02:35 -05:00
Michael Schurter 4e7ea460e8 test: port some pre-0.9 DeploymentHealth tests
Skipping a failing one as I need to move to some other work and don't
want to leave this work orphaned on my machine.
2019-01-14 09:56:53 -08:00
Michael Schurter ff2f23f5f9 test: assert service interpolation behavior
Ported from pre-0.9 tests.
2019-01-14 09:56:53 -08:00
Michael Schurter 5746be5844 test: add some extra logging 2019-01-14 09:56:53 -08:00
Michael Schurter fc1bb95ef8 Remove old comment; it's been fixed! 2019-01-14 09:56:53 -08:00
Michael Schurter e877bb6370 test: assert shutdown delay deregs first
Restore a pre-0.9 test that asserts Consul services are deregistered
before a task's shutdown delay.
2019-01-14 09:56:53 -08:00
Michael Schurter 1ca858fa92
Update client/allocrunner/taskrunner/stats_hook.go
Co-Authored-By: nickethier <ncethier@gmail.com>
2019-01-14 12:31:27 -05:00
Nick Ethier fbd403df96
tr: stop stats collection on Exited hook 2019-01-14 12:30:14 -05:00
Nick Ethier a4534779d3
qemu: missing gofmt 2019-01-13 16:06:56 -05:00
Michael Schurter ff034ffbc9
Update drivers/qemu/driver_test.go
use t.Logf instead of fmt.Printf

Co-Authored-By: nickethier <ncethier@gmail.com>
2019-01-12 21:33:55 -05:00
Nick Ethier 597b7b751d
tr: add retry /w backoff to stats_hook failure 2019-01-12 12:18:24 -05:00
Nick Ethier 3b395d7100
drivers: plumb grpc client logger 2019-01-12 12:18:23 -05:00
Nick Ethier 7e306afde3
executor: fix failing stats related test 2019-01-12 12:18:23 -05:00
Nick Ethier b0d9440474
docker: add test for stats collection 2019-01-12 12:18:22 -05:00
Nick Ethier 9fea54e0dc
executor: implement streaming stats API
plugins/driver: update driver interface to support streaming stats

client/tr: use streaming stats api

TODO:
 * how to handle errors and closed channel during stats streaming
 * prevent tight loop if Stats(ctx) returns an error

drivers: update drivers TaskStats RPC to handle streaming results

executor: better error handling in stats rpc

docker: better control and error handling of stats rpc

driver: allow stats to return a recoverable error
2019-01-12 12:18:22 -05:00
Preetha f562a1d169
Merge pull request #5164 from hashicorp/b-client-addalloc-errhandle
Handle client initialization errors when adding/restoring allocs
2019-01-12 10:40:41 -06:00
Preetha Appan 9e8dbf6a4b
linting fixes 2019-01-12 10:38:20 -06:00
Preetha Appan c94179578d
Make unit test for allocrunner failure much nicer 2019-01-12 10:38:20 -06:00
Preetha Appan da0d083b03
Add unit test to simulate alloc runner creation failure 2019-01-12 10:38:20 -06:00
Preetha Appan e7b59ac08c
Only set deployment health if not already set 2019-01-12 10:38:20 -06:00
Michael Schurter dbf4c3a3c8
Apply suggestions from code review
Co-Authored-By: preetapan <preetha@hashicorp.com>
2019-01-12 10:38:20 -06:00
Preetha Appan 7bd1440710
REfactor statedb factory config to set it directly in client config 2019-01-12 10:38:20 -06:00
Preetha Appan e237f19b38
Remove invalid allocs 2019-01-12 10:38:20 -06:00
Preetha Appan f059ef8a47
Modified destroy failure handling to rely on allocrunner's destroy method
Added a unit test with custom statedb implementation that errors, to
use to verify destroy errors
2019-01-12 10:37:12 -06:00
Preetha Appan 6c95da8f67
Add back code to mark alloc as failed when restore fails
Also modify restore such that any handled errors don't propagate
back to the client
2019-01-12 10:37:12 -06:00
Preetha Appan 5fde0b0f5c
Revert code that made an alloc update when restore fails
Restore currently shuts down the client so the alloc update cant
always make it to the server
2019-01-12 10:37:12 -06:00
Preetha Appan 41bfdd764b
Handle client initialization errors when adding allocs or restoring allocs
We mark the alloc as failed and track failed allocs so that we don't send
updates after the first time
2019-01-12 10:37:12 -06:00
Danielle Tomlinson 6c0d30caa0
Merge pull request #5025 from hashicorp/dani/allocLockCleanup
client: Cleanup allocrunner access
2019-01-11 23:35:02 +01:00
Alex Dadgar a573816c85 changelog 2019-01-11 13:09:44 -08:00
Alex Dadgar 5621086f50 Enable json logs 2019-01-11 11:36:37 -08:00
Alex Dadgar 6f64cfab7e Update client1.hcl 2019-01-11 11:36:37 -08:00