Commit graph

2174 commits

Author SHA1 Message Date
Alex Dadgar aa05ecc37c Merge pull request #2361 from hashicorp/f-docker-coordination
Docker Image Coordinator and caching
2017-02-24 14:18:21 -08:00
Alex Dadgar b5d4f39734 Docker Image Coordinator
This PR introduces a coordinator for doing CRUD on a Docker image. It
should fix racy deletion of images. The issue before was images would be
deleted between prestart and start causing an error.
2017-02-24 13:20:40 -08:00
Michael Schurter f61b7975a3 Put access to Docker volume drivers behind flag 2017-02-23 15:47:36 -08:00
Alex Dadgar 2239503e6e Fix noisy log 2017-02-23 14:40:24 -08:00
Pietro Menna 2f88d6cb68 Docker Volume Drivers
This commit adds the functionality to use Docker Volume Drivers.
2017-02-23 14:36:32 -03:00
Alex Dadgar c7e004b8c5 Merge pull request #2344 from hashicorp/b-fix-allocdir-move-test
Fix allocdir Move test and make code more defensive
2017-02-22 18:43:10 -08:00
Alex Dadgar 6936c81720 Merge pull request #2302 from hashicorp/f-task-state-setup
Adding a task event for setup
2017-02-22 18:42:57 -08:00
Alex Dadgar d2cbbac76e Merge pull request #2347 from hashicorp/b-parse
Remove defaulting from parse and fix tests
2017-02-22 18:42:43 -08:00
Alex Dadgar cef7882827 Fix tests and docs 2017-02-22 18:28:07 -08:00
Diptanu Choudhury 98921575af Adding a task event for setup 2017-02-22 18:28:07 -08:00
Alex Dadgar b5995c6cd7 Vendor + test fixes 2017-02-22 15:55:33 -08:00
Michael Schurter e5a29b82db Fix allocdir Move test and make code more defensive
A change in the behavior of `os.Rename` in Go 1.8 brought to light a
difference in the logic between `{Alloc,Task}Runner` and this test:

AllocRunner builds the alloc dir, moves dirs if necessary, and then lets
TaskRunner call TaskDir.Build().

This test called `TaskDir.Build` *before* `AllocDir.Move`, so in Go 1.8
it failed to `os.Rename over` the empty {data,local} dirs.

I updated the test to behave like the real code, but I defensively added
`os.Remove` calls as a subtle change in call order shouldn't break this
code. `os.Remove` won't remove a non-empty directory, so it's still
safe.
2017-02-21 17:22:10 -08:00
Michael Schurter 12c549d6a0 Skip setting MemorySwap on Windows
Windows doesn't support this Docker setting.

Fixes #2193
2017-02-21 13:21:42 -08:00
Alex Dadgar d0fd0e277a Don't force uppercase meta keys in env vars 2017-02-20 19:51:24 -08:00
Alex Dadgar f9323f03df Drivers log during fingerprinting
This PR fixes a regression in which some drivers did not log during
fingerprinting.
2017-02-20 19:35:51 -08:00
Alex Dadgar b49fceb491 rebase 2017-02-20 16:43:28 -08:00
Alex Dadgar c586cb653b Merge pull request #2318 from hashicorp/f-vault-debug
Server side Vault telemetry
2017-02-17 11:34:37 -08:00
Alex Dadgar 4223eb25eb Docs 2017-02-17 10:16:15 -08:00
Alex Dadgar 7203dee7ab Add allocated/unallocated metrics to client 2017-02-16 18:28:11 -08:00
Alex Dadgar c62cd5cc55 Revendor docker client 2017-02-14 17:34:05 -08:00
Alex Dadgar 0940fc08af Better derive token logging 2017-02-14 16:46:54 -08:00
Alex Dadgar 238b4bcafd Add Leader support to client 2017-02-10 17:55:19 -08:00
Michael Schurter fb59e42473 Fix test to assert new never-nil behavior 2017-02-10 10:11:15 -08:00
Michael Schurter 51e4fe9915 Use getters & setters with nil guards 2017-02-09 17:44:58 -08:00
Michael Schurter 37e7e7a3e5 Fix upgrade path for created resources
This *might* be a fix for #2295 -- I've been unable to reproduce the
bug. However, this guard seems wise regardless. I should never be
overwriting an intialized created resources with a nil.
2017-02-09 13:54:33 -08:00
Alex Dadgar b51ba01d54 Merge pull request #2293 from hashicorp/f-vendor-memdb
Vendor MemDB
2017-02-08 14:51:01 -08:00
Alex Dadgar 04862ca10e Tests compile 2017-02-07 21:30:57 -08:00
Sean Chittenden c4c321c770
Unconditionally lowercase the node ID read from disk. 2017-02-06 16:20:17 -08:00
Sean Chittenden adb5be23ef
Add better verification of a host's HostID. 2017-02-02 16:24:32 -08:00
Alex Dadgar d1f8301e73 Merge pull request #2277 from hashicorp/f-durable-uuid
Reproducible Node ID
2017-02-02 11:54:49 -08:00
Alex Dadgar 1562a7aeee Merge pull request #2262 from hashicorp/f-permissions
Template destination file permissions.
2017-02-02 11:28:44 -08:00
Sean Chittenden bb4347e277
Slight mis-merge: secret-id in dev mode is random and needs to be returned. 2017-02-01 22:20:52 -08:00
Sean Chittenden bb422a2258
Generate a durable NodeID if possible, otherwise fall back to a random HostID. 2017-02-01 22:11:33 -08:00
Alex Dadgar b40f284bd1 Merge branch 'master' into f-permissions 2017-02-01 16:57:46 -08:00
Alex Dadgar ec7bc783a4 Merge branch 'master' into b-vault-race 2017-02-01 16:57:26 -08:00
Alex Dadgar 9987a235a5 Fix race condition with Deriving vault tokens
This PR fixes a race condition in which the client was not locked while
deriving Vault tokens. This allowed the token to be set which would
cause subsequent Vault requests to fail with permission denied because
the incorrect Vault token was being used.

Further this PR makes the unsetting and unlocking of the client atomic
to avoid an even harder to hit race condition (not sure it was ever hit
but was still incorrect).
2017-02-01 16:25:59 -08:00
Diptanu Choudhury 90ac3e2774 Checking in updated gc tests 2017-02-01 13:12:46 -08:00
Alex Dadgar 6b7efa1ba6 Template destination file permissions.
This PR allows setting the file permissions of the rendered template.
2017-01-31 20:10:01 -08:00
Diptanu Choudhury 11d7cb1230 Making the GC related fields tunable 2017-01-31 15:51:20 -08:00
Michael Schurter aef3c2e380 Handle createdResourcs=nil
Combined with b522c472fdf this fixes #2256

Without these two commits in place upgrades to 0.5.3 panics.
2017-01-31 10:51:32 -08:00
Alex Dadgar 44d63318a4 Add nil guard 2017-01-31 09:19:59 -08:00
Alex Dadgar 91f485a189 slightly more verbose test 2017-01-30 16:16:26 -08:00
Alex Dadgar 7fa99037a4 up timing 2017-01-26 13:25:37 -08:00
Alex Dadgar 8196a58c4c Rename dispatch_input to dispatch_payload 2017-01-25 21:27:44 -08:00
Alex Dadgar 0933bcadf9 Fix import 2017-01-25 21:27:36 -08:00
Alex Dadgar b7e8bae128 Fix unreachable function in tests 2017-01-25 20:58:24 -08:00
Alex Dadgar 4f99243fa3 More timing fixes 2017-01-25 11:57:38 -08:00
Alex Dadgar 68fd18f5cf small env fix 2017-01-25 10:42:11 -08:00
Alex Dadgar 1e3ffec95f Fix flaky test 2017-01-24 09:40:13 -08:00
Alex Dadgar 75e3f2c996 Fix fsm/env test 2017-01-24 09:36:22 -08:00
Alex Dadgar 4e5de453c7 Fix template test 2017-01-23 16:42:02 -08:00
Alex Dadgar 985598d4b5 Fix test 2017-01-23 16:33:35 -08:00
Diptanu Choudhury 264a0c46cf Merge pull request #2223 from hashicorp/f-env-addr
Setting the Addrs of sibling tasks in the env
2017-01-23 15:05:48 -08:00
Diptanu Choudhury fda3a5250c Merge pull request #2222 from hashicorp/b-docker-image-not-found
Making the docker driver retry when an image is not found
2017-01-23 14:58:00 -08:00
Diptanu Choudhury dfd1f03ec8 Added a comment 2017-01-23 14:57:30 -08:00
Alex Dadgar 5fc629d4b7 Merge pull request #2190 from hashicorp/b-docker-auth
Better Docker Auth lookup
2017-01-23 14:50:50 -08:00
Alex Dadgar e160b2b38f Add test and better logs
This commit adds a test to retrieving auth configurations, use either
the auth block in the config or specified via the agent config and adds
a log if lookup fails.
2017-01-23 14:48:02 -08:00
Alex Dadgar 6b02229eb0 fix flaky test 2017-01-23 14:12:38 -08:00
Alex Dadgar 43a17b7d74 Close file 2017-01-23 11:37:45 -08:00
Alex Dadgar ee5296ba70 Deprecation notice 2017-01-23 11:32:51 -08:00
Alex Dadgar 3365688571 Remove SSL 2017-01-23 11:32:51 -08:00
Alex Dadgar 4a627b02ca undo 2017-01-23 11:32:51 -08:00
Alex Dadgar 08733b68d2 vendor 2017-01-23 11:32:51 -08:00
Alex Dadgar b1823a5fdb Better auth lookup 2017-01-23 11:32:51 -08:00
Alex Dadgar 3d5b31fc96 WIP 2017-01-23 11:32:51 -08:00
Alex Dadgar 625f738a6f fix test build 2017-01-23 11:26:34 -08:00
Alex Dadgar 024e9ec02a remove xxx 2017-01-23 11:04:44 -08:00
Alex Dadgar 66909bb7fc Finalize 2017-01-23 11:03:38 -08:00
Alex Dadgar 55fdf82513 test nomad env 2017-01-23 11:00:12 -08:00
Alex Dadgar 0db766872c fix lookup 2017-01-23 11:00:12 -08:00
Alex Dadgar f93f2249e9 Clear SSL 2017-01-23 11:00:12 -08:00
Alex Dadgar b71ec08b80 rebase 2017-01-23 11:00:12 -08:00
Diptanu Choudhury 84a491f85a Locking appropriately before closing the channel to indicate migration 2017-01-23 10:46:57 -08:00
Alex Dadgar ec6b6e69d4 remove reference to entrypoint 2017-01-22 14:04:41 -08:00
Alex Dadgar d051065229 Actually randomize the splay 2017-01-22 13:50:33 -08:00
Michael Schurter 054ee8df59 Fix index we get allocs by 2017-01-20 16:30:40 -08:00
Diptanu Choudhury fe4ae25eea Setting the Addrs of sibling tasks in the env 2017-01-20 14:15:53 -08:00
Michael Schurter de7dbf5ff9 Fix java tests 2017-01-20 14:12:30 -08:00
Michael Schurter c621ad3f75 Merge pull request #2216 from hashicorp/b-rootless-raw-exec
Don't link shared alloc dir into task dir for raw_exec
2017-01-20 13:18:54 -08:00
Michael Schurter a28fc4798a Merge pull request #2215 from hashicorp/b-2211-host-env
Fix image based drivers having host env vars set
2017-01-20 13:16:44 -08:00
Michael Schurter c2c1b0a049 Merge pull request #2198 from hashicorp/b-rootless-docker
Stop trying to use mount for image based drivers
2017-01-20 13:03:56 -08:00
Michael Schurter 295c2bf74a Merge pull request #2186 from hashicorp/f-driver-cleanup
Add Cleanup method to Driver interface
2017-01-20 13:02:14 -08:00
Michael Schurter 1f7b5b4b47 Rename Constructor -> Parameterized Job 2017-01-20 12:43:10 -08:00
Diptanu Choudhury be7081568c Making the docker driver retry when an image is not found 2017-01-20 12:38:04 -08:00
Michael Schurter c5f222e4a6 Update created resources before exiting cleanup 2017-01-19 16:48:23 -08:00
Michael Schurter a93d43a9cf Exit early when cleanup succeeds 2017-01-19 15:07:01 -08:00
Michael Schurter 85f68aa00c Fix incorrect lock usage 2017-01-19 11:39:18 -08:00
Michael Schurter 010a7709f1 Bump unknown resource to ERR 2017-01-19 09:48:07 -08:00
Diptanu Choudhury 1999b7eebb Merge pull request #2159 from hashicorp/b-consul-config
Fixed merging consul config
2017-01-18 16:14:54 -08:00
Diptanu Choudhury e927de02d2 Moved functions to helper from structs 2017-01-18 15:55:14 -08:00
Michael Schurter d8b4372904 Don't link shared alloc dir into task dir for raw_exec
Fixes running raw_exec tasks when nomad isn't root.
2017-01-18 11:28:34 -08:00
Michael Schurter f94210b4bc Fix image based drivers having host env vars set
Add detailed tests for GetTaskEnv to avoid this issue happening again!

Fixes #2211
2017-01-18 10:27:03 -08:00
Michael Schurter 578272b7f2 Add CreatedResources.Remove and use it 2017-01-17 16:41:59 -08:00
Michael Schurter 1bcf7cdbfe Remove outdated comment 2017-01-17 16:23:29 -08:00
Michael Schurter 82b49d4547 Updated CreatedResources as images are cleaned 2017-01-17 16:13:40 -08:00
Michael Schurter beed31ff6f Remove outdated comment 2017-01-17 16:05:21 -08:00
Michael Schurter b9d6d2c8d6 Return error from Prestart 2017-01-17 16:04:09 -08:00
Michael Schurter ea87091e58 Prevent race between alloc runners
Block ar1's periodic syncing which could recreate the state file ar2 was
destroying.
2017-01-17 13:10:20 -08:00
Michael Schurter 15952e5d17 Try to get test passing in Travis 2017-01-17 12:51:19 -08:00
Michael Schurter 255698e8af Use Image ID instead of Image Name 2017-01-13 16:53:58 -08:00