Commit Graph

13906 Commits

Author SHA1 Message Date
Danielle Tomlinson 5d54a0408f fingerprint: Limit vault shutdown waiting
When vault is installed through chocolatey, it also installs a shim that
will not pass kill signals to the child. This means the process will
never actually terminate, and we lose the process handle.

Here, rather than waiting forever, we timeout fast.
2019-01-17 18:43:13 +01:00
Danielle Tomlinson dc55d3e353 vaultclient: Update tests for vault 1.0 2019-01-17 18:43:13 +01:00
Danielle Tomlinson 7a5d511349 fingerprinter: Use HCLogger for windows 2019-01-17 18:43:13 +01:00
Danielle Tomlinson 96b1d5d20c testtask: Build on windows 2019-01-17 18:43:13 +01:00
Danielle Tomlinson 2656f28f08 chore: Setup appveyor for windows test execution 2019-01-17 18:43:13 +01:00
Mahmood Ali 061c38e4b5
Merge pull request #5202 from hashicorp/b-no-gpl-gopsutil
Update shirou/gopsutil dep to use non-GPL code
2019-01-17 11:40:04 -05:00
Mahmood Ali 21e193ecf4 Update shirou/gopsutil dep to use non-GPL code
Previously used `github.com/shirou/gopsutil`[1], used some GPL code [2].
This was somewhat unintentional, and was addressed later [3].

Due to being late in the cycle of Nomad release when this is noticed,
and time elapsed since we updated the dependency, we want to be
conservative in our package updates.

As such, we opted to go with forking the repo to use the previously used
version with the GPL removal code commit, done in [4].

[1] 5776ff9c7c
[2] 5776ff9c7c/host/include/smc.c
[3] c95755e4bc
[4] 62d5761ddb
2019-01-17 11:08:30 -05:00
Mahmood Ali 677e11da86 fix imports 2019-01-16 19:53:48 -05:00
Michael Schurter bea4d297b9
Merge pull request #5197 from hashicorp/b-rkt-cpu
rkt: revert to pre-0.9 --cpu flag
2019-01-16 15:05:21 -08:00
Preetha 71aa7c2b37
Merge pull request #5182 from hashicorp/b-docker-fingerprint-log-once
Make driver logging less redundant
2019-01-16 11:35:33 -06:00
Preetha Appan 55319b05d1
clean up read access 2019-01-16 11:04:11 -06:00
Danielle Tomlinson a695b3562c
Merge pull request #5193 from hashicorp/dani/logmon-reattach
logmon: Reattach to existing loggers
2019-01-16 17:34:13 +01:00
Preetha Appan 469a286b1b
Refactor logging in drivers to use a tri-state boolean
Changes logging warnings/errors only if the state changes
from healthy to unhealthy
2019-01-16 10:19:31 -06:00
Preetha Appan 0c2c0a2d43
Make docker driver logging less redundant 2019-01-16 10:16:57 -06:00
Michael Schurter 282cdde382
Merge pull request #5199 from hashicorp/test-dispatch
test: porting TestTaskRunner_SimpleRun_Dispatch
2019-01-16 06:33:51 -08:00
Danielle Tomlinson 99da4c780d logmon: Reattach to existing loggers
This commit prevents us from creating duplicate logmon hooks when
restoring allocations by persisting the logmon reattach config using
HookData.
2019-01-16 14:56:10 +01:00
Michael Schurter daa7d029a1 test: porting TestTaskRunner_SimpleRun_Dispatch
Porting test from 0.8 to 0.9.
2019-01-15 15:22:13 -08:00
Alex Dadgar 5de91cbfb7
Merge pull request #5046 from oleksii-shyman/docker_runtimes
Nvidia runtime
2019-01-15 14:34:14 -08:00
Michael Schurter 1879a7f788 rkt: revert to pre-0.9 --cpu flag
See
https://github.com/hashicorp/nomad/issues/3394#issuecomment-453296121
for details. During 0.9 development we switched to shares, but we'd
prefer to maintain backward compat.
2019-01-15 13:15:28 -08:00
Chris Baker d355650026
Merge pull request #5194 from hashicorp/f-minor-update-docs-tense
docs: updated tense around fixed bug
2019-01-15 14:50:57 -05:00
oleksii.shyman e41fbf7577 Add support for docker runtimes
- docker fingerprint issues a docker api system info call to get the
  list of supported OCI runtimes.
  - OCI runtimes are reported as comma separated list of names
  - docker driver is aware of GPU runtime presence
  - docker driver throws an error when user tries to run container with
  GPU, when GPU runtime is not present
  - docker GPU runtime name is configurable
2019-01-15 11:34:47 -08:00
oleksii.shyman 7393e91145 Update go-dockerclient and docker dependencies
- go-dockerclient is updated because of lack of "runtime" API field
 - docker/docker is updated because go-dockerclient depends on docker
 master branch
2019-01-15 11:28:57 -08:00
Alex Dadgar f349e3f2e5
Merge pull request #5195 from hashicorp/d-update-087-date
make change to the 0.8.7 release date
2019-01-15 09:53:07 -08:00
Cathy Chang 5d1e4b5650 make change to the 0.8.7 release date 2019-01-15 09:47:13 -08:00
Chris Baker 8b15543a14 docs: updated tense around fixed bug 2019-01-15 17:43:03 +00:00
Danielle Tomlinson b918d25e62
Merge pull request #5192 from hashicorp/dani/executor-close
executor: Always close stdout/stderr fifos
2019-01-15 17:49:04 +01:00
Danielle Tomlinson f120c8f8f6
Merge pull request #5184 from hashicorp/dani/b-logmon-reattach
docker: Terminate dockerlogger
2019-01-15 16:48:40 +01:00
Danielle Tomlinson 7f1ff3fab6 executor: Always close stdout/stderr fifos 2019-01-15 16:47:27 +01:00
Michael Schurter 48afda786b
Merge pull request #5187 from hashicorp/test-consul
Port a bunch of pre-0.9 Consul tests to 0.9
2019-01-15 07:41:50 -08:00
Mahmood Ali ce5a4e7b94
Merge pull request #5189 from hashicorp/b-fix-executor-logs
propagate logs to executor plugin
2019-01-15 10:26:01 -05:00
Danielle Tomlinson 272a8726d7 docker: Terminate dockerlogger
Previously, we did not attempt to stop Docker Logger processes until
DestroyTask, which means that under many circumstances, we will never
successfully close the plugin client.

This commit terminates the plugin process when `run` terminates, or when
`DestroyTask` is called.

Steps to repro:

```
$ nomad agent -dev
$ nomad init
$ nomad run example.nomad
$ nomad stop example
$ ps aux | grep nomad # See docker logger process running
$ signal the dev agent
$ ps aux | grep nomad # See docker logger process running
```
2019-01-15 14:58:05 +01:00
Mahmood Ali 5649f72d27 propogate logs to executor plugin 2019-01-15 08:25:03 -05:00
Alex Dadgar 471fdb3ccf
Merge pull request #5173 from hashicorp/b-log-levels
Plugins use parent loggers
2019-01-14 16:14:30 -08:00
Mahmood Ali 9909d98bee Track Basic Memory Usage as reported by cgroups
Track current memory usage, `memory.usage_in_bytes`, in addition to
`memory.max_memory_usage_in_bytes` and friends.  This number is closer
what Docker reports.

Related to https://github.com/hashicorp/nomad/issues/5165 .
2019-01-14 18:47:52 -05:00
Nick Ethier c619e70d39
Merge pull request #5018 from hashicorp/f-executor-stats
executor: streaming stats api
2019-01-14 15:02:35 -05:00
Michael Schurter 4e7ea460e8 test: port some pre-0.9 DeploymentHealth tests
Skipping a failing one as I need to move to some other work and don't
want to leave this work orphaned on my machine.
2019-01-14 09:56:53 -08:00
Michael Schurter ff2f23f5f9 test: assert service interpolation behavior
Ported from pre-0.9 tests.
2019-01-14 09:56:53 -08:00
Michael Schurter 5746be5844 test: add some extra logging 2019-01-14 09:56:53 -08:00
Michael Schurter fc1bb95ef8 Remove old comment; it's been fixed! 2019-01-14 09:56:53 -08:00
Michael Schurter e877bb6370 test: assert shutdown delay deregs first
Restore a pre-0.9 test that asserts Consul services are deregistered
before a task's shutdown delay.
2019-01-14 09:56:53 -08:00
Michael Schurter 1ca858fa92
Update client/allocrunner/taskrunner/stats_hook.go
Co-Authored-By: nickethier <ncethier@gmail.com>
2019-01-14 12:31:27 -05:00
Nick Ethier fbd403df96
tr: stop stats collection on Exited hook 2019-01-14 12:30:14 -05:00
Nick Ethier a4534779d3
qemu: missing gofmt 2019-01-13 16:06:56 -05:00
Michael Schurter ff034ffbc9
Update drivers/qemu/driver_test.go
use t.Logf instead of fmt.Printf

Co-Authored-By: nickethier <ncethier@gmail.com>
2019-01-12 21:33:55 -05:00
Nick Ethier 597b7b751d
tr: add retry /w backoff to stats_hook failure 2019-01-12 12:18:24 -05:00
Nick Ethier 3b395d7100
drivers: plumb grpc client logger 2019-01-12 12:18:23 -05:00
Nick Ethier 7e306afde3
executor: fix failing stats related test 2019-01-12 12:18:23 -05:00
Nick Ethier b0d9440474
docker: add test for stats collection 2019-01-12 12:18:22 -05:00
Nick Ethier 9fea54e0dc
executor: implement streaming stats API
plugins/driver: update driver interface to support streaming stats

client/tr: use streaming stats api

TODO:
 * how to handle errors and closed channel during stats streaming
 * prevent tight loop if Stats(ctx) returns an error

drivers: update drivers TaskStats RPC to handle streaming results

executor: better error handling in stats rpc

docker: better control and error handling of stats rpc

driver: allow stats to return a recoverable error
2019-01-12 12:18:22 -05:00
Preetha f562a1d169
Merge pull request #5164 from hashicorp/b-client-addalloc-errhandle
Handle client initialization errors when adding/restoring allocs
2019-01-12 10:40:41 -06:00