Commit graph

14435 commits

Author SHA1 Message Date
Mahmood Ali 0128b0ce7a tests: port TestTaskRunner_Template_NewVaultToken
From https://github.com/hashicorp/nomad/blob/v0.8.7/client/task_runner_test.go#L1275
2019-02-22 15:53:04 -05:00
Mahmood Ali cfb80583af tests: port TestTaskRunner_Template_Artifact
From https://github.com/hashicorp/nomad/blob/v0.8.7/client/task_runner_test.go#L1195
2019-02-22 15:52:59 -05:00
Mahmood Ali 1b14214a88 tests: port TestAllocRunner_RetryArtifact
Port TestAllocRunner_RetryArtifact from https://github.com/hashicorp/nomad/blob/v0.8.7/client/alloc_runner_test.go#L610-L672

I changed the test name because it doesn't actually test that artifact
hooks is retried
2019-02-22 15:50:39 -05:00
Mahmood Ali c827e6e05a tests: port TestAllocRunner_MoveAllocDir test 2019-02-22 15:50:39 -05:00
Preetha 911c93f7bd
Merge pull request #5350 from hashicorp/b-json-logging-meta
Support json logging for CLI output for agent
2019-02-22 13:40:56 -06:00
Preetha Appan 8f9ec85fe6
fix import order 2019-02-22 13:40:13 -06:00
Charlie Voiselle 42718cd9f5 Created .github; Added CoC 2019-02-22 10:49:05 -08:00
Charlie Voiselle 006afdca9b Added comments
* caller should created eval id
* prev/next eval used in failed-follow-up
2019-02-22 10:22:52 -08:00
Michael Schurter 0da04cfb43
Merge pull request #5325 from hashicorp/test-port-08-2
Do not wrap errors from StartTask as that loses Recoverable
2019-02-22 07:06:01 -08:00
Danielle Tomlinson 8aff115fca docker: Support stats on Windows 2019-02-22 14:19:58 +01:00
Michael Schurter a2e3ea6dc9 logmon: fix reattach configuration
There were multiple bugs here:

1. Reattach unmarshalling always returned an error because you can't
   unmarshal into a nil pointer.
2. The hook data wasn't being saved because it was put on the request
   struct, not the response struct.
3. The plugin configuration should only have reattach *or* a command
   set. Not both.
4. Setting Done=true meant the hook was never re-run on agent restart so
   reattaching was never attempted.
2019-02-21 15:32:18 -08:00
Michael Schurter f5e0dba9d1 fingerprint: improve initial fingerpint message
The initial fingerprint message is actually fairly useful, so I bumped
it to Debug and fixed the output formatting.
2019-02-21 15:32:18 -08:00
Michael Schurter 38821954b7 plugins: squelch context Canceled error logs
As far as I can tell this is the most straightforward and resilient way
to skip error logging on context cancellation with grpc streams. You
cannot compare the error against context.Canceled directly as it is of
type `*status.statusError`. The next best solution I found was:

```go
resp, err := stream.Recv()
if code, ok := err.(interface{ Code() code.Code }); ok {
	if code.Code == code.Canceled {
		return
	}
}
```

However I think checking ctx.Err() directly makes the code much easier
to read and is resilient against grpc API changes.
2019-02-21 15:32:18 -08:00
Michael Schurter 01cabdff88 client: restart on recoverable StartTask errors
Fixes restarting on recoverable errors from StartTask.

Ports TestTaskRunner_Run_RecoverableStartError from 0.8 which discovered
the bug.
2019-02-21 15:30:49 -08:00
Michael Schurter e3f321cd27 test: port TestTaskRunner_RestartSignalTask_NotRunning from 0.8 2019-02-21 15:30:49 -08:00
Michael Schurter f3aa945a00 test: port TestTaskRunner_DriverNetwork from 0.8 2019-02-21 15:30:49 -08:00
Michael Schurter 518405ac33
Merge pull request #5322 from hashicorp/b-artifact-retries
Fix regression by restarting on artifact download errors
2019-02-21 15:28:51 -08:00
Omar Khawaja 084230a12b
reformat lxc guide (#5339)
* reformat lxc guide

* reformat lxc guide
2019-02-21 17:40:22 -05:00
Michael Schurter 3a9730a890
Merge pull request #5348 from hashicorp/docs-env-json
docs: clarify use of toJSON for passwords
2019-02-21 14:36:28 -08:00
Preetha Appan 3ab2e431b6
Move logger initialization to earlier step 2019-02-21 12:41:54 -06:00
Mahmood Ali 6d30284ec9
Merge pull request #5341 from hashicorp/ci-windows-docker
Run Docker tests in Windows AppVeyor CI
2019-02-21 13:17:33 -05:00
Michael Schurter 7dd6259e93 docs: clarify use of toJSON for passwords
Fixes #5347
2019-02-21 09:47:18 -08:00
Charlie Voiselle c28c195f42 Set NextEval when making failed-follow-up evals
This allows users to locate failed-follow-up evals more easily
2019-02-20 16:07:11 -08:00
Danielle Tomlinson b08e6e5925
Merge pull request #5332 from hashicorp/dani/docker_logger_container_leak
docker: Avoid leaking containers during Reattach
2019-02-20 22:53:06 +01:00
Michael Schurter b7fad990b7
Merge pull request #5336 from hashicorp/test-ar-destroy
tests: port TestAllocRunner_Destroy from 0.8
2019-02-20 13:07:51 -08:00
Michael Schurter 2553800eb8 tests: port TestAllocRunner_Destroy from 0.8
Also add destroy(ar) helper to fix a bunch of shutdown races in AR
tests.
2019-02-20 12:35:09 -08:00
Danielle Tomlinson 2610e2d9ef docker: Avoid leaking containers during Reattach
Currently if a docker_logger cannot be reattached to, we will leak the
container that was being used. This is problematic if e.g using static
ports as it means you can never recover your task, or if a service is
expensive to run and will then be running without supervision.
2019-02-20 17:47:06 +01:00
Michael Schurter 6580ed668e client: don't redownload completed artifacts on retries
Track the download status of each artifact independently so that if only
one of many artifacts fails to download, completed artifacts aren't
downloaded again.
2019-02-20 08:45:12 -08:00
Danielle Tomlinson 953755ce24
Merge pull request #5335 from hashicorp/dani/docker-logger-spawn
Increase resiliency of docker driver logging
2019-02-20 17:16:05 +01:00
Michael Schurter a1645edb0b Update drivers/docker/docklog/docker_logger.go
Co-Authored-By: dantoml <dani@tomlinson.io>
2019-02-20 17:12:56 +01:00
Danielle Tomlinson 2f18441a47 docker: Respawn docker logger during recovery
Sometimes the nomad docker_logger may be killed by a service manager
when restarting the client for upgrades or reliability reasons.

Currently if this happens, we leak the users container and try to
reschedule over it.

This commit adds a new step to the recovery process that will spawn a
new docker logger process that will fetch logs from _the current
timestamp_. This is to avoid restarting users tasks because our logging
sidecar has failed.
2019-02-20 17:12:56 +01:00
Michael Schurter 908bfab4c2 client: artifact errors are retry-able
0.9.0beta2 contains a regression where artifact download errors would
not cause a task restart and instead immediately fail the task.

This restores the pre-0.9 behavior of retrying all artifact errors and
adds missing tests.
2019-02-20 07:21:27 -08:00
Michael Schurter 79ccf00b72 tests: add new task runner test helper
Adds a new helper and removes a duplicated test.
2019-02-20 07:21:27 -08:00
Danielle Tomlinson 16d9440794
Merge pull request #5342 from hashicorp/dani/powercolors
Fix colored terminal output on Windows
2019-02-20 15:13:58 +01:00
Mahmood Ali 8c82c19831 tests: IsTravis() -> IsCI()
Replace IsTravis() references that is intended for more CI environments
rather than for Travis environment specifically.
2019-02-20 08:21:03 -05:00
Mahmood Ali 1ca2bae429 ci: Run Docker tests on AppVeyor Windows 2019-02-20 08:21:03 -05:00
Danielle Tomlinson 91c300c310 ui: Support colored output on Windows
This commit uses the go-colorable library to enable support for coloured
UI output on Windows. This acts as a compatibility layer that takes
standard unix-y terminal codes and translates them into the requisite
windows calls as required.
2019-02-20 14:01:35 +01:00
Danielle Tomlinson 3ae58eba84 vendor: Update to latest go-colorable 2019-02-20 13:56:15 +01:00
Mahmood Ali fedab3d7b0 driver/docker: Skip failing Windows tests
Skip currently Docker tests that fail on Windows for further
investigation.
2019-02-20 07:48:02 -05:00
Mahmood Ali 33ff8c3e8d tests: expect Docker on AppVeyor
Prepare to run docker on AppVeyor Windows environment
2019-02-20 07:41:47 -05:00
Mahmood Ali dd8a5c862a
Merge pull request #5321 from hashicorp/b-portmap-regression
drivers: restore port_map old json support
2019-02-19 20:58:37 -05:00
Mahmood Ali 4def8529db driver/docker: use BlockAttrs for storage_opts
storage_opts is a new field in 0.9 cycle and doesn't have backward
compatibility constraints.
2019-02-19 20:35:28 -05:00
Mahmood Ali a394cd63f4
CVE-2019-5736: Update libcontainer depedencies (#5334)
* CVE-2019-5736: Update libcontainer depedencies

Libcontainer is vulnerable to a runc container breakout, that was
reported as CVE-2019-5736[1].  Upgrading vendored libcontainer with the fix.

The runc changes are captured in 369b920277 .

[1] https://seclists.org/oss-sec/2019/q1/119
2019-02-19 20:21:18 -05:00
Preetha Appan 0149bbc608
cli Ui implementation that logs to a hclogger
This makes it so any messages output to the UI *after* the agent has started
will be logged in json format correctly
2019-02-19 17:53:14 -06:00
Omar Khawaja 2bd79d757a
adding dropdown nav to external plugins section (#5338) 2019-02-19 17:00:03 -05:00
Michael Schurter 554bf5be02
Merge pull request #5329 from hashicorp/b-health-deadline
client: fix setting alloc unhealthy at deadline
2019-02-19 08:40:53 -08:00
Michael Schurter 159042a1a3 client: fix setting alloc unhealthy at deadline
During the 0.9 client refactor the code to fail a deployment when the
deadline was reached was broken. This restores and tests that behavior.
2019-02-19 07:44:14 -08:00
Danielle Tomlinson 3cf3ac7eac dlogger: Increase resilience to docker api failure
This commit adds some extra resiliency to the docker logger in the case
of API failure from the docker daemon, by restarting the stream from the
current point in time if the stream returns and the container is still
running.
2019-02-19 15:17:54 +01:00
Charlie Voiselle 094e2a28d5 Warn that data_dir can't be in the jail (#4618) 2019-02-18 20:40:45 -05:00
Mahmood Ali 24d510290a
Fix device.attr typo (#5331)
Fix https://github.com/hashicorp/nomad/issues/5315
2019-02-16 13:07:02 -05:00