Commit graph

14303 commits

Author SHA1 Message Date
Michael Schurter 79ccf00b72 tests: add new task runner test helper
Adds a new helper and removes a duplicated test.
2019-02-20 07:21:27 -08:00
Danielle Tomlinson 16d9440794
Merge pull request #5342 from hashicorp/dani/powercolors
Fix colored terminal output on Windows
2019-02-20 15:13:58 +01:00
Mahmood Ali 8c82c19831 tests: IsTravis() -> IsCI()
Replace IsTravis() references that is intended for more CI environments
rather than for Travis environment specifically.
2019-02-20 08:21:03 -05:00
Mahmood Ali 1ca2bae429 ci: Run Docker tests on AppVeyor Windows 2019-02-20 08:21:03 -05:00
Danielle Tomlinson 91c300c310 ui: Support colored output on Windows
This commit uses the go-colorable library to enable support for coloured
UI output on Windows. This acts as a compatibility layer that takes
standard unix-y terminal codes and translates them into the requisite
windows calls as required.
2019-02-20 14:01:35 +01:00
Danielle Tomlinson 3ae58eba84 vendor: Update to latest go-colorable 2019-02-20 13:56:15 +01:00
Mahmood Ali fedab3d7b0 driver/docker: Skip failing Windows tests
Skip currently Docker tests that fail on Windows for further
investigation.
2019-02-20 07:48:02 -05:00
Mahmood Ali 33ff8c3e8d tests: expect Docker on AppVeyor
Prepare to run docker on AppVeyor Windows environment
2019-02-20 07:41:47 -05:00
Mahmood Ali dd8a5c862a
Merge pull request #5321 from hashicorp/b-portmap-regression
drivers: restore port_map old json support
2019-02-19 20:58:37 -05:00
Mahmood Ali 4def8529db driver/docker: use BlockAttrs for storage_opts
storage_opts is a new field in 0.9 cycle and doesn't have backward
compatibility constraints.
2019-02-19 20:35:28 -05:00
Mahmood Ali a394cd63f4
CVE-2019-5736: Update libcontainer depedencies (#5334)
* CVE-2019-5736: Update libcontainer depedencies

Libcontainer is vulnerable to a runc container breakout, that was
reported as CVE-2019-5736[1].  Upgrading vendored libcontainer with the fix.

The runc changes are captured in 369b920277 .

[1] https://seclists.org/oss-sec/2019/q1/119
2019-02-19 20:21:18 -05:00
Preetha Appan 0149bbc608
cli Ui implementation that logs to a hclogger
This makes it so any messages output to the UI *after* the agent has started
will be logged in json format correctly
2019-02-19 17:53:14 -06:00
Omar Khawaja 2bd79d757a
adding dropdown nav to external plugins section (#5338) 2019-02-19 17:00:03 -05:00
Michael Schurter 554bf5be02
Merge pull request #5329 from hashicorp/b-health-deadline
client: fix setting alloc unhealthy at deadline
2019-02-19 08:40:53 -08:00
Michael Schurter 159042a1a3 client: fix setting alloc unhealthy at deadline
During the 0.9 client refactor the code to fail a deployment when the
deadline was reached was broken. This restores and tests that behavior.
2019-02-19 07:44:14 -08:00
Danielle Tomlinson 3cf3ac7eac dlogger: Increase resilience to docker api failure
This commit adds some extra resiliency to the docker logger in the case
of API failure from the docker daemon, by restarting the stream from the
current point in time if the stream returns and the container is still
running.
2019-02-19 15:17:54 +01:00
Charlie Voiselle 094e2a28d5 Warn that data_dir can't be in the jail (#4618) 2019-02-18 20:40:45 -05:00
Mahmood Ali 24d510290a
Fix device.attr typo (#5331)
Fix https://github.com/hashicorp/nomad/issues/5315
2019-02-16 13:07:02 -05:00
Mahmood Ali 46cd3c3f55 drivers: restore port_map old json support
This ensures that `port_map` along with other block like attribute
declarations (e.g. ulimit, labels, etc) can handle various hcl and json
syntax that was supported in 0.8.

In 0.8.7, the following declarations are effectively equivalent:

```
// hcl block
port_map {
  http = 80
  https = 443
}

// hcl assignment
port_map = {
  http  = 80
  https = 443
}

// json single element array of map (default in API response)
{"port_map": [{"http": 80, "https": 443}]}

// json array of individual maps (supported accidentally iiuc)
{"port_map: [{"http": 80}, {"https": 443}]}
```

We achieve compatbility by using `NewAttr("...", "list(map(string))",
false)` to be serialized to a `map[string]string` wrapper, instead of using
`BlockAttrs` declaration.  The wrapper merges the list of maps
automatically, to ease driver development.

This approach is closer to how v0.8.7 implemented the fields [1][2], and
despite its verbosity, seems to perserve 0.8.7 behavior in hcl2.

This is only required for built-in types that have backward
compatibility constraints.  External drivers should use `BlockAttrs`
instead, as they see fit.

[1] https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/docker.go#L216
[2] https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/docker.go#L698-L700
2019-02-16 11:37:33 -05:00
Preetha 95edd4a97b
Merge pull request #5324 from hashicorp/f-nomad-init
expand job init example with spread and affinity
2019-02-14 14:58:10 -06:00
Preetha Appan 35f6db47b8
fix indentation 2019-02-14 12:49:26 -06:00
Preetha Appan f443c7d321
Fix whitespace 2019-02-14 12:49:26 -06:00
Chris Baker ab02f4588e
Update job_init.go
minor
2019-02-14 12:49:26 -06:00
Preetha Appan d405881e34
expand job init example with spread and affinity 2019-02-14 12:49:26 -06:00
Michael Schurter b5d9d6a32d
Merge pull request #5319 from hashicorp/test-port-08
Fix Consul Service PreKill Hook
2019-02-14 10:18:41 -08:00
Danielle Tomlinson be431cb83d
Merge pull request #5326 from hashicorp/dani/json-submission
api: Fix compatibility with pre 0.9 API jobs
2019-02-14 18:56:13 +01:00
Mahmood Ali 1430f94b2a
Update drivers/docker/config_test.go
Co-Authored-By: dantoml <dani@tomlinson.io>
2019-02-14 18:55:10 +01:00
Danielle Tomlinson 42b0edffd2
Merge pull request #5327 from hashicorp/dani/windows-term
drivers/docker: SIGTERM to stop containers
2019-02-14 18:05:28 +01:00
Danielle Tomlinson 3f696be06b Add regression test for parsing null mounts 2019-02-14 18:03:35 +01:00
Mahmood Ali 87be233aca
test: improve readability of duration
Co-Authored-By: schmichael <michael.schurter@gmail.com>
2019-02-14 08:12:06 -08:00
Mahmood Ali 16d3414842
test: improve failure message
Co-Authored-By: schmichael <michael.schurter@gmail.com>
2019-02-14 08:11:37 -08:00
Danielle Tomlinson a3a1491958 drivers/docker: SIGTERM to stop containers
Windows Docker daemon does not support SIGINT, SIGTERM is the semantic
equivalent that allows for graceful shutdown before being followed up by
a SIGKILL.
2019-02-14 15:38:54 +00:00
Danielle Tomlinson b59ed9d6c9 vendor: Update to latest hcl2
* Includes fixes for handling null blocks https://github.com/hashicorp/hcl2/pull/87
* Updates go-cty as a newer version is required by hcl2
2019-02-14 13:34:46 +01:00
Michael Schurter 4814f0fb0b tests: port TestTaskRunner_Download_List from 0.8 2019-02-12 15:48:04 -08:00
Michael Schurter a152e3ef17 consul: fix task deregistration hook
Broke ShutdownDelay but the test was timing dependent so it just
appeared flaky. Made the test slower so that it should never incorrectly
pass.
2019-02-12 15:36:02 -08:00
Michael Schurter 4ad879e75e tests: port TaskRunner_DeriveToken tests from 0.8 2019-02-12 15:36:02 -08:00
Michael Schurter 6743ed9fdc tests: port TestTaskRunner_BlockForVault from 0.8
Also fix race conditions in the mock vault client.
2019-02-12 13:46:09 -08:00
Mahmood Ali f7102cd01d
tests: add hcl task driver config parsing tests (#5314)
* drivers: add config parsing tests

Add basic tests for parsing and encoding task config.

* drivers/docker: fix some config declarations

* refactor and document config parse helpers
2019-02-12 14:46:37 -05:00
Mahmood Ali aec9120994
drivers/java: restore 0.8.7 java version detection (#5317)
Restore 0.8.x behavior where java driver is marked as detected when
`java -version` exits with 0 but returns unexpected output.

Furthermore, we restore behavior when `java -version` where we parse the
first three lines of `java -version` but ignore rest.

If `java -version` returns less than 3 lines, Nomad 0.8.7 would panic.
In this implementation, we'd still mark java as detected but returns
empty version.

The 0.8.7 logic for detecting java version is found in
https://github.com/hashicorp/nomad/blob/v0.8.7/client/driver/java.go#L132-L172
.

I punt on revamping how we can be more resilient to java -version
syntax, and aimed for preserving existing behavior instead.
2019-02-12 13:41:26 -05:00
Preetha 412127e13b
Merge pull request #5287 from hashicorp/docs-fix-affinity-examples
docs: fixed some affinity examples
2019-02-12 09:13:27 -06:00
Michael Schurter 3b84e08fa4
Merge pull request #5297 from hashicorp/b-docker-logging
Docker: Fix logging config parsing
2019-02-11 06:57:52 -08:00
Gertjan Roggemans 94ca78354b docker: Fix volume driver_config options spec (#5309)
Fixes #5308
2019-02-11 09:18:44 -05:00
Danielle Tomlinson 9411daf16f
Merge pull request #5307 from mnachury/fix/exe_detection_windows
Fix executable check on windows
2019-02-08 17:46:55 -05:00
mnachury aef18aa4ca Fix executable check on windows 2019-02-08 16:05:14 -05:00
Nick Ethier e99877fe33
Merge pull request #5306 from nanoz/patch-1
Issue #4649: Remove the systemd process and threads limit
2019-02-08 10:43:49 -05:00
Arno B 341f83e7f5
Remove the systemd process and threads limit
systemd limits the number of process and threads you can create within the cgroup it creates for a given service. This is especially limiting if you have a high number of keys Nomad watches from Consul.

Fixes #4649
2019-02-08 16:22:52 +01:00
Cathy Chang f7b3fc95f6
remove apostrophe in "job's" 2019-02-06 13:54:59 -08:00
Chris Baker a4df11c4c8 restored previous example (it was correct) 2019-02-06 19:13:40 +00:00
Michael Schurter e1e4b10884 docker: fix logging config parsing
Fixes
https://groups.google.com/d/topic/nomad-tool/B3Uo6Kns2BI/discussion
2019-02-04 11:07:57 -08:00
Michael Schurter 9bf4b38ab3 plugins: update hclutils test
The test used old local copies of Docker structs and appeared to be
testing an outdated approach to task config decoding.

Updated to use real Docker structs so we can do end-to-end unit testing
of real Docker task configs.
2019-02-04 11:07:57 -08:00