Commit graph

140 commits

Author SHA1 Message Date
Mahmood Ali 67e2a0ac05
docker: report unhealthy in unsupported Windows (#5356)
On Windows, Nomad only supports Windows containers, so report as
unhealthy otherwise.
2019-02-27 08:10:23 -05:00
Danielle Tomlinson e250aad31b
Merge pull request #5355 from hashicorp/dani/windows-dockerstats
docker: Support Stats on Windows
2019-02-26 16:39:48 +01:00
Danielle Tomlinson e3dc80bea3 docker: Return undetected before first detection
This commit causes the docker driver to return undetected before it
first establishes a connection to the docker daemon.

This fixes a bug where hosts without docker installed would return as
unhealthy, rather than undetected.
2019-02-25 11:02:42 +01:00
Danielle Tomlinson 8aff115fca docker: Support stats on Windows 2019-02-22 14:19:58 +01:00
Mahmood Ali 6d30284ec9
Merge pull request #5341 from hashicorp/ci-windows-docker
Run Docker tests in Windows AppVeyor CI
2019-02-21 13:17:33 -05:00
Danielle Tomlinson 2610e2d9ef docker: Avoid leaking containers during Reattach
Currently if a docker_logger cannot be reattached to, we will leak the
container that was being used. This is problematic if e.g using static
ports as it means you can never recover your task, or if a service is
expensive to run and will then be running without supervision.
2019-02-20 17:47:06 +01:00
Danielle Tomlinson 953755ce24
Merge pull request #5335 from hashicorp/dani/docker-logger-spawn
Increase resiliency of docker driver logging
2019-02-20 17:16:05 +01:00
Michael Schurter a1645edb0b Update drivers/docker/docklog/docker_logger.go
Co-Authored-By: dantoml <dani@tomlinson.io>
2019-02-20 17:12:56 +01:00
Danielle Tomlinson 2f18441a47 docker: Respawn docker logger during recovery
Sometimes the nomad docker_logger may be killed by a service manager
when restarting the client for upgrades or reliability reasons.

Currently if this happens, we leak the users container and try to
reschedule over it.

This commit adds a new step to the recovery process that will spawn a
new docker logger process that will fetch logs from _the current
timestamp_. This is to avoid restarting users tasks because our logging
sidecar has failed.
2019-02-20 17:12:56 +01:00
Mahmood Ali 8c82c19831 tests: IsTravis() -> IsCI()
Replace IsTravis() references that is intended for more CI environments
rather than for Travis environment specifically.
2019-02-20 08:21:03 -05:00
Mahmood Ali fedab3d7b0 driver/docker: Skip failing Windows tests
Skip currently Docker tests that fail on Windows for further
investigation.
2019-02-20 07:48:02 -05:00
Mahmood Ali 4def8529db driver/docker: use BlockAttrs for storage_opts
storage_opts is a new field in 0.9 cycle and doesn't have backward
compatibility constraints.
2019-02-19 20:35:28 -05:00
Danielle Tomlinson 3cf3ac7eac dlogger: Increase resilience to docker api failure
This commit adds some extra resiliency to the docker logger in the case
of API failure from the docker daemon, by restarting the stream from the
current point in time if the stream returns and the container is still
running.
2019-02-19 15:17:54 +01:00
Mahmood Ali 46cd3c3f55 drivers: restore port_map old json support
This ensures that `port_map` along with other block like attribute
declarations (e.g. ulimit, labels, etc) can handle various hcl and json
syntax that was supported in 0.8.

In 0.8.7, the following declarations are effectively equivalent:

```
// hcl block
port_map {
  http = 80
  https = 443
}

// hcl assignment
port_map = {
  http  = 80
  https = 443
}

// json single element array of map (default in API response)
{"port_map": [{"http": 80, "https": 443}]}

// json array of individual maps (supported accidentally iiuc)
{"port_map: [{"http": 80}, {"https": 443}]}
```

We achieve compatbility by using `NewAttr("...", "list(map(string))",
false)` to be serialized to a `map[string]string` wrapper, instead of using
`BlockAttrs` declaration.  The wrapper merges the list of maps
automatically, to ease driver development.

This approach is closer to how v0.8.7 implemented the fields [1][2], and
despite its verbosity, seems to perserve 0.8.7 behavior in hcl2.

This is only required for built-in types that have backward
compatibility constraints.  External drivers should use `BlockAttrs`
instead, as they see fit.

[1] https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/docker.go#L216
[2] https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/docker.go#L698-L700
2019-02-16 11:37:33 -05:00
Danielle Tomlinson be431cb83d
Merge pull request #5326 from hashicorp/dani/json-submission
api: Fix compatibility with pre 0.9 API jobs
2019-02-14 18:56:13 +01:00
Mahmood Ali 1430f94b2a
Update drivers/docker/config_test.go
Co-Authored-By: dantoml <dani@tomlinson.io>
2019-02-14 18:55:10 +01:00
Danielle Tomlinson 3f696be06b Add regression test for parsing null mounts 2019-02-14 18:03:35 +01:00
Danielle Tomlinson a3a1491958 drivers/docker: SIGTERM to stop containers
Windows Docker daemon does not support SIGINT, SIGTERM is the semantic
equivalent that allows for graceful shutdown before being followed up by
a SIGKILL.
2019-02-14 15:38:54 +00:00
Mahmood Ali f7102cd01d
tests: add hcl task driver config parsing tests (#5314)
* drivers: add config parsing tests

Add basic tests for parsing and encoding task config.

* drivers/docker: fix some config declarations

* refactor and document config parse helpers
2019-02-12 14:46:37 -05:00
Michael Schurter 3b84e08fa4
Merge pull request #5297 from hashicorp/b-docker-logging
Docker: Fix logging config parsing
2019-02-11 06:57:52 -08:00
Gertjan Roggemans 94ca78354b docker: Fix volume driver_config options spec (#5309)
Fixes #5308
2019-02-11 09:18:44 -05:00
Michael Schurter e1e4b10884 docker: fix logging config parsing
Fixes
https://groups.google.com/d/topic/nomad-tool/B3Uo6Kns2BI/discussion
2019-02-04 11:07:57 -08:00
Nick Ethier e7ea26449e
client: fix bug during 0.8 state up grade that causes external drivers to fail 2019-01-30 14:22:29 -05:00
Alex Dadgar bc804dda2e Nomad 0.9.0-beta1 generated code 2019-01-30 10:49:44 -08:00
Alex Dadgar 991bcc3ef1 Don't fall through 2019-01-28 09:53:19 -08:00
Alex Dadgar 403faa0d7c comment 2019-01-28 09:47:53 -08:00
Alex Dadgar 68ced492fb Fix killing non-existant container with a kill timeout 2019-01-25 16:21:51 -08:00
Nick Ethier 448f93f1e9
Merge pull request #5237 from hashicorp/b-docker-09upgrade-shutdown
drivers/docker: handle shutdown of upgraded tasks correctly
2019-01-25 12:13:03 -05:00
Nick Ethier ac693fe6ac
docker: only log missing bridge_ip on initial fingerprint 2019-01-24 22:34:57 -05:00
Nick Ethier c7cc81924d
drivers/docker: handle shutdown of upgraded tasks correctly 2019-01-24 14:21:59 -05:00
Nick Ethier a5823a270c
Merge pull request #5209 from hashicorp/f-driver-upgradepath-test
e2e tests for upgrade path to driver plugins
2019-01-23 21:03:29 -05:00
Nick Ethier be976d9c9a
Merge branch 'master' into f-driver-upgradepath-test
* master: (23 commits)
  tests: avoid assertion in goroutine
  spell check
  ci: run checkscripts
  tests: deflake TestRktDriver_StartWaitRecoverWaitStop
  drivers/rkt: Remove unused github.com/rkt/rkt
  drivers/rkt: allow development on non-linux
  cli: Hide `nomad docker_logger` from help output
  api: test api and structs are in sync
  goimports until make check is happy
  nil check node resources to prevent panic
  tr: use context in as select statement
  move pluginutils -> helper/pluginutils
  vet
  goimports
  gofmt
  Split hclspec
  move hclutils
  Driver tests do not use hcl2/hcl, hclspec, or hclutils
  move reattach config
  loader and singleton
  ...
2019-01-23 21:01:24 -05:00
Nick Ethier 5b9013528e
drivers: add docker upgrade path and e2e test 2019-01-23 14:44:42 -05:00
Alex Dadgar 48dfbbcad9 fix docker runtime attribute 2019-01-23 10:58:46 -08:00
Michael Schurter 32daa7b47b goimports until make check is happy 2019-01-23 06:27:14 -08:00
Michael Schurter be0bab7c3f move pluginutils -> helper/pluginutils
I wanted a different color bikeshed, so I get to paint it
2019-01-22 15:50:08 -08:00
Alex Dadgar b2c7268843 move reattach config 2019-01-22 15:11:58 -08:00
Alex Dadgar cdcd3c929c loader and singleton 2019-01-22 15:11:57 -08:00
Nick Ethier e3c6f89b9a
drivers: use consts for task handle version 2019-01-18 18:31:01 -05:00
Nick Ethier 6804450c69
cleanup code comments and small fixes from refactor 2019-01-18 18:31:01 -05:00
Mahmood Ali 5df63fda7c
Merge pull request #5190 from hashicorp/f-memory-usage
Track Basic Memory Usage as reported by cgroups
2019-01-18 16:46:02 -05:00
Danielle Tomlinson b65bf78513 docker: Fix missing import 2019-01-17 18:44:27 +01:00
Danielle Tomlinson 7fca934509 chore: General Cleanup 2019-01-17 18:43:14 +01:00
Danielle Tomlinson e73962d8d6 docker: Only run Cleanup test on unix os' 2019-01-17 18:43:14 +01:00
Danielle Tomlinson 30a5e25d94 fixup: Typo in docker test 2019-01-17 18:43:14 +01:00
Danielle Tomlinson 82018cd030 chore: Fix docklog linting 2019-01-17 18:43:14 +01:00
Danielle Tomlinson 3b2ff2005b chore: Fix docker test linting
Due to https://github.com/tsenart/deadcode/issues/3 we can't specify
these consts on their own. This moves them into the _platform_test.go
files to avoid creating a package that only exposes a couple of values.
2019-01-17 18:43:14 +01:00
Danielle Tomlinson de86435cf8 docker: Test cleanup for windows
* Docker for Windows does not support ulimits
* Use filepath.ToSlash to test workdir
* Convert expected mount paths to system style
* Skip security-opt test on windows
  - Windows does not support seccomp, and it's unclear which options are
    available.
* Skip StartN due to lack of sigint
* docker: Use api to get image info on windows
* No bridge on windows
* Stop hardcoding /bin/
2019-01-17 18:43:14 +01:00
Danielle Tomlinson 3d2f0bb7da docker: ExpandPath tests validate slashpath 2019-01-17 18:43:13 +01:00
Danielle Tomlinson 37190a5595 dockerlogger: Fix tests on windows
Uses the home directory and windows path expansion, as c:\tmp doesn't
necessarily exist, and mktemp would involve unnecessarily complicating
the commands.
2019-01-17 18:43:13 +01:00