d2381c9263
Merge pull request #2853 from hashicorp/b-watcher
...
Improve alloc health watcher
2017-07-18 14:12:28 -07:00
bd43bd509c
Save deployment status
2017-07-18 12:37:52 -07:00
41f67e3535
Small fixes
2017-07-18 12:19:57 -07:00
c24e73ede7
Fix deadlock caused by syncing during destroy
...
When replacing an alloc the new alloc is blocked until the old alloc is
destroyed. This could cause a deadlock:
1. Destroying the old alloc includes a final sync of its status
2. Syncing status causes a GC
3. A GC looks for terminal allocs to cleanup
4. The GC waits for an alloc to stop completely before GC'ing
If the GC chooses the currently-being-destroyed-alloc to GC, the GC
deadlocks. If `client.max_parallel` deadlocks happen the GC is wedged
until the Nomad process is restarted.
Performing the final sync asynchronously is an ugly hack but prevents
the deadlock by allowing the final sync to occur after the alloc runner
has shutdown and been destroyed.
2017-07-18 11:12:56 -07:00
420be86e39
Test AllocDir.Copy
2017-07-17 15:46:54 -07:00
cdb2e96d99
Add AllocRunner.allocID for ease-of-use
...
Since the AllocRunner.alloc struct can be mutated, most of AllocRunner
needs to acquire a lock to get the alloc's ID. Log lines always need to
include the alloc ID, so we often skipped acquiring a lock just to grab
the ID and accepted the race.
Let's make the race detector a little happier by storing the ID in a
single assignment field.
2017-07-17 15:46:54 -07:00
181fda825a
Fix log level
2017-07-17 15:46:54 -07:00
98f6e7f10f
Don't fail if task dirs don't exist on creation
...
Task dir metadata is created in AllocRunner.Run which may not run before
an alloc is sync'd and Nomad exits. There's no reason not to just create
task dir metadata on restore if it doesn't exist.
2017-07-17 15:46:54 -07:00
51515cbe0c
Ensure allocDir is never nil and persisted safely
...
Fixes #2834
2017-07-17 15:46:54 -07:00
0821ee67f5
Fix alloc broadcaster panic on double close
2017-07-17 14:09:05 -07:00
0a6bf87365
Fix nil panic in Docker error condition
...
Fixes #2835
Yet another bug caused by overwriting container and then trying to
reference container.ID in the err handling block. Did a quick audit of
docker.go and it seems to be the last offender. See #2804 for previous
bug.
2017-07-14 10:48:19 -07:00
e9a416b731
Merge branch 'master' into fix-pending-state
2017-07-10 10:43:23 -07:00
26b16fa3ce
#2563 fixed pending state for allocations with terminal status
2017-07-09 16:18:06 +03:00
05894f4611
Small fixes
2017-07-07 17:34:50 -07:00
fecb16cfb2
Merge pull request #2793 from hashicorp/b-2776-ct-vault-servername
...
Propagate vault.tls_server_name to consul-template
2017-07-07 16:44:19 -07:00
95a9a5da71
Merge pull request #2787 from hashicorp/f-docker-test-mac
...
Test #2652 - Docker MAC Address option
2017-07-07 16:22:10 -07:00
4be4df21c9
Merge pull request #2797 from hashicorp/f-2785-docker-bridge-ip
...
Add driver.docker.bridge_ip node attribute
2017-07-07 16:20:20 -07:00
94389c3ecc
Remove debug logging
2017-07-07 16:19:42 -07:00
5e3e3818db
Merge pull request #2804 from hashicorp/b-2802-docker-panic
...
Don't panic in container list/remove/inspect race
2017-07-07 15:35:51 -07:00
67a7b0eac9
Don't panic in container list/remove/inspect race
...
Fixes #2802
While it's hard to reproduce the theoretical race is:
1. This goroutine calls ListContainers()
2. Another goroutine removes a container X
3. This goroutine attempts to InspectContainer(X)
However, this bug could be hit in the much simpler case of
InspectContainer() timing out.
In those cases an error is returned and the old code attempted to wrap
the error with the now-nil container.ID. Storing the container ID fixes
that panic.
2017-07-07 15:10:59 -07:00
bf97a2455c
Vet and small improvement on watcher failure detection
2017-07-07 14:53:01 -07:00
45712c6ca3
test fixes
2017-07-07 14:11:27 -07:00
ade9a7c768
@jippi Changed my mind! Good suggestion
2017-07-07 12:12:48 -07:00
c063eba836
Warn log
2017-07-07 12:10:04 -07:00
067ed86a47
Client watches for allocation health using task state and Consul checks
...
This PR adds watching of allocation health at the client. The client can
watch for health based on the tasks running on time and also based on
the consul checks passing.
2017-07-07 12:10:04 -07:00
001058227e
watcher per alloc
2017-07-07 12:07:08 -07:00
2e2fd26bed
Update index
2017-07-07 12:07:08 -07:00
ecee5e370e
initial watcher
2017-07-07 12:07:08 -07:00
c77944ed29
assign names
2017-07-07 12:03:11 -07:00
084dd384c1
Add driver.docker.bridge_ip node attribute
...
Fixes #2785
2017-07-07 10:14:10 -07:00
d38d48151a
Propagate vault.tls_server_name to consul-template
...
Fixes #2776
2017-07-06 16:56:50 -07:00
39edf23fd5
Merge pull request #2786 from hashicorp/f-docker-auth-soft-fail
...
Default to auth hard fail but optionally soft fail
2017-07-06 13:25:56 -07:00
bae1b7db2d
Test #2652
...
Also cleanup docker config opts docs
2017-07-06 12:46:25 -07:00
8f4353779a
Merge branch 'master' into master
2017-07-06 12:09:36 -07:00
2900f941b5
Default to auth hard fail but optionally soft fail
2017-07-06 11:35:34 -07:00
08b452adf5
Merge pull request #2781 from hashicorp/f-2678-getter-mode
...
Add support for go-getter modes
2017-07-06 11:06:40 -07:00
b000bb8598
Merge pull request #2744 from aep/master
...
Do not fail when no docker registry auth is available
2017-07-06 11:04:11 -07:00
0d3bdf7210
Add support for go-getter modes
...
Fixes #2678
2017-07-06 10:45:44 -07:00
644f0cfaa4
Consistently quote alloc ids in client logs
2017-07-06 10:24:52 -07:00
4fd9ef6a8c
Tiny client race condition fix
...
Plus some logging improvements that may help with #2563
2017-07-05 16:15:19 -07:00
8e2e26c607
rkt: use %s instead of %q when interpolating env
...
Fixes #2686
2017-07-05 09:36:17 -07:00
b2382f99f2
0 compute == error
2017-07-03 14:51:02 -07:00
ecf090e980
Fix cpu_total_compute override
2017-07-03 14:51:02 -07:00
2d741c770b
Merge pull request #2732 from hashicorp/b-persist-alloc-updates
...
Persist Alloc when EvalID changes
2017-07-03 14:46:43 -07:00
56a6f8ca8a
Merge pull request #2763 from hashicorp/f-bad-state-help
...
Add more logging to restore state errors
2017-07-03 14:45:03 -07:00
9d4b0651ef
Merge pull request #2753 from hashicorp/b-leader-dies-first
...
Destroy task group leader first
2017-07-03 14:38:04 -07:00
6e7cc3964e
Merge pull request #2709 from hashicorp/f-advertise-docker-ips
...
Advertise driver-specific addresses
2017-07-03 14:04:12 -07:00
5ec52ec24a
Destroy task group leader first
...
Before this commit all tasks in a task group were destroyed
concurrently. This meant logging sidecars might be stopped before the
leader task whose logs still need to be shipped.
This commit blocks on the leader shutting down before signalling to
followers to shutdown.
2017-07-03 13:56:56 -07:00
596727230b
Suggest wiping out alloc dir too
2017-07-03 12:29:21 -07:00
11f68bfca2
Add more logging to restore state errors
2017-07-03 11:58:41 -07:00
aa4f029f10
Do not fail when no docker registry auth is available
...
this amends the behaviour introduced with #2651
and allows pulling public images when docker.auth.helper is set
2017-06-27 11:11:18 +02:00
8fcf866a7d
Fix some tests still expecting reverted behavior
2017-06-23 16:51:38 -07:00
e81252ba45
Default no_host_uuid to true instead of false
...
The host UUID isn't unique in many virtualized cases and of dubious
value even when it is univerally unique. Default to a random UUID.
2017-06-23 16:23:01 -07:00
5a274e6683
Style and comments
2017-06-23 15:20:04 -07:00
cff8546035
Fix spelling & re-add immutable state struct
2017-06-23 13:01:39 -07:00
d359d3b554
Rename immutable -> alloc
...
meh; naming is hard
2017-06-23 10:58:36 -07:00
af2fc0f1bc
Persist Alloc when EvalID changes
2017-06-22 17:33:12 -07:00
f3a6ddc57d
Remove DRIVER env vars
...
Also make NOMAD_ADDR_* use host ip:port for consistency. NOMAD_PORT_*
varies based on port map and the driver IP isn't exposed as an env var
as the only place it can be used is in script checks anyway.
2017-06-21 17:19:08 -07:00
0633d0c286
Have Qemu return PortMap
2017-06-21 17:19:08 -07:00
38a0695687
Simplify Docker Networks processing
2017-06-21 17:19:08 -07:00
fec83b271a
Bump error log level
2017-06-21 17:19:08 -07:00
8d677bc6b9
Fix lxc tests
2017-06-21 17:19:08 -07:00
8d440b1675
Skip DRIVER env vars for labels without a port mapping
2017-06-21 17:19:08 -07:00
c0eff81383
Fix Service.AddressMode changes during task updates
2017-06-21 17:19:08 -07:00
67d154a274
Test driver network advertisement and checks
2017-06-21 17:19:08 -07:00
b9bfb84b53
Implement DriverNetwork and Service.AddressMode
...
Ideally DriverNetwork would be fully populated in Driver.Prestart, but
Docker doesn't assign the container's IP until you start the container.
However, it's important to setup the port env vars before calling
Driver.Start, so Prestart should populate that.
2017-06-21 17:19:08 -07:00
59ab34c264
Fix typos
2017-06-16 16:10:12 +02:00
b69e060071
Log PID when sending signals
2017-06-12 11:11:36 -07:00
ffb417a300
Merge pull request #2697 from hashicorp/b-port-map
...
Fix port map interpolation for docker
2017-06-09 13:29:36 -07:00
a3827d2cc6
Fix bad merge conflict resolution
2017-06-09 10:40:47 -07:00
eabd6759c6
Merge branch 'master' into add-no-overlay-option
2017-06-09 09:59:35 -07:00
5ba2662b30
Merge pull request #2687 from mmickan/issue-2685
...
Include symlinks in snapshots when migrating disks
2017-06-08 13:35:46 -07:00
784d69789e
Merge branch 'master' into add-no-overlay-option
2017-06-08 13:15:56 -07:00
7695e636d5
Fix port map interpolation for docker
...
This PR fixes an issue in which the value of the portmap could not be
interpolated.
Fixes https://github.com/hashicorp/nomad/issues/2680
2017-06-08 13:12:32 -07:00
b55f4bf601
Fix backticks in docs; refine --debug comment
2017-06-07 21:11:22 +02:00
a258a803f2
Added insecure_options config list
2017-06-07 09:58:42 +02:00
1957e9dfa6
Add a no_overlay option for the rkt task config.
2017-06-07 00:17:33 +02:00
c196d320f8
Add tests for migrating symlinks in alloc and local directories
2017-06-04 15:56:22 +09:30
236f24c9a4
Include symlinks in snapshots when migrating disks
...
Fixes #2685
2017-06-04 00:36:18 +09:30
d1dd380890
Switch to hashicorp/go-envparse
2017-06-02 15:58:52 -07:00
a552bcdb55
Move env file parsing to a library
2017-06-02 15:03:27 -07:00
3b46fe136f
small cleanup
2017-05-31 15:56:54 -07:00
8d6e28ace8
Merge branch 'master' into feature/2334
2017-05-31 14:27:07 -07:00
044f1da5ff
Merge pull request #2681 from hashicorp/b-deadlock
...
Fix a deadlock relating to blocked allocations
2017-05-31 14:26:54 -07:00
ec9cb2c751
Merge pull request #2672 from eyberg/master
...
dont throw away errors in log rotation
2017-05-31 14:14:22 -07:00
b1eea2269a
Fix deadlock
2017-05-31 14:05:47 -07:00
cb568a5cf6
Cleanup lots of leaked alloc runners in tests
2017-05-31 11:39:50 -07:00
6138564f00
Implement support for docker-credential-helpers
...
Solves: #2334
2017-05-31 12:45:02 +02:00
ffc2b36dc7
Merge pull request #2636 from hashicorp/f-gc-alloc-limit
...
Add new gc_max_allocs tuneable
2017-05-30 16:14:09 -07:00
dd51aa1cb9
Merge pull request #2654 from hashicorp/f-env-consul
...
Add envconsul-like support and refactor environment handling
2017-05-30 14:40:14 -07:00
e1a7c2d6d7
Fix Error -> Errorf
2017-05-30 12:08:59 -07:00
53d713bacb
Fix getter tests
...
And use an interface for ReplaceEnv since its all getter needs.
2017-05-26 16:52:47 -07:00
51d8231911
Fix executor tests
2017-05-26 16:46:03 -07:00
3184616936
Always use PATH-only env for rkt commands
2017-05-26 15:41:26 -07:00
83543b1d80
Use custom TaskEnv to provide PATH for rkt
2017-05-26 15:24:14 -07:00
3afade9675
Let's pretend I never committed this
2017-05-26 15:06:59 -07:00
3eb0827c19
Fix and test multi-env-template loading
2017-05-25 17:13:33 -07:00
d279c1f5e6
Merge pull request #2675 from hashicorp/b-perms
...
Fix perms to just set exec bit
2017-05-25 14:45:08 -07:00
28aef447e9
Fix perms to just set exec bit
2017-05-25 14:44:13 -07:00
3fbc58997b
dont throw away errors in log rotation
2017-05-25 11:49:33 -07:00
6d67d4dfbf
Fix formatting
2017-05-25 09:38:49 -07:00
796d3250a5
Comment and correct formatting
2017-05-25 09:30:58 -07:00
5f9cb4c514
Switch tests to mock_driver
2017-05-25 09:28:10 -07:00
ce8c4fa520
Fail fast on env template failures
2017-05-24 13:44:52 -07:00
4963cf07d7
Add env file test
2017-05-23 17:07:25 -07:00
a233b0401d
Add env testing
2017-05-23 16:46:29 -07:00
d793dde4e9
Shrink chroot to avoid timing test failure
2017-05-23 16:11:24 -07:00
15ef740ab6
Add env.Builder.UpdateTask for alloc updates
2017-05-23 16:00:57 -07:00
f324ca4683
Fix test data
2017-05-23 13:53:34 -07:00
e7db2c9b0e
Handle Driver.Prestart returning nil, nil
2017-05-23 13:53:34 -07:00
fd9bef768f
Move task env into execcontext
...
Also inject PATH into rkt commands since we're no longer appending host
env vars for it.
2017-05-23 13:53:34 -07:00
59ad200d1a
Fix env var interpolation and env tests
2017-05-23 13:53:34 -07:00
37b148fb60
Add PortMap to struct returned by Driver.Prestart
...
Moves env.Builder out of drivers entirely so one less thing to worry
about when implementing driver plugins.
2017-05-23 13:53:34 -07:00
83c641ced0
Move env template handling into consul_template.go
2017-05-23 13:53:34 -07:00
a2eb7d17c9
Improve PortMap handling and simplify Builder creation
2017-05-23 13:53:34 -07:00
8da110855e
Move path building to task dir initialization
2017-05-23 13:53:34 -07:00
d2c08ff24b
Refactor TaskEnvironment into Builder and TaskEnv
2017-05-23 13:53:33 -07:00
10b6610e56
Functional consul template env file support
2017-05-23 13:45:14 -07:00
6d7caa5f6b
Fix for test
2017-05-19 16:59:07 -07:00
6b2d5bd4fd
Add SecurityOpt as a config field in Docker driver
2017-05-19 16:18:49 -07:00
6d76fb2f17
Merge branch 'master' of github.com:hashicorp/nomad
2017-05-17 14:46:03 -07:00
ee8dd84965
Fix nil job on allocation
...
The way the copying was happening on the alloc_runner was by temporarily
setting the alloc.Job to nil, copying and then restoring it. This
created an issue in which when the alloc was shared (which it is in
server/client mode and between alloc_runner/task_runner) there were race
conditions that could create a panic.
Fixes https://github.com/hashicorp/nomad/issues/2605
2017-05-17 14:07:06 -04:00
06f937bf28
Merge pull request #2591 from hashicorp/b-2180-script-updates
...
Properly interpolate services on updated tasks
2017-05-17 09:09:01 -07:00
323a0a78f3
Fixed wrong newline
2017-05-17 16:51:22 +02:00
ea644237cf
mac address pinning in docker driver
...
This commit adds mac address pining to the docker driver.
2017-05-17 16:41:00 +02:00
3841692138
gc_max_allocs should include blocked & migrating
2017-05-12 16:03:22 -07:00
49ce86ee0a
Lower default gc_max_allocs to 50
2017-05-12 15:57:27 -07:00
0453c2709c
Add new gc_max_allocs tuneable
...
More than gc_max_allocs may be running on a node, but terminal allocs
will be garbage collected to try to keep the total number below the
limit.
2017-05-11 17:18:02 -07:00
68c3a2bd98
Fix vet errors
2017-05-11 13:08:08 -07:00
23685d65d0
Merge pull request #2610 from hashicorp/f-bolt-db
...
Client persist state using bolt-db and more efficient write patterns
2017-05-09 13:01:36 -07:00
3cd7e06fba
Fix test
2017-05-09 11:35:48 -07:00
3b15db1ac4
Ignore Consul deregister errors on executors
...
Errors here only occur if Consul is not running when Nomad is restarted.
Errors here are only an issue if:
* Consul is being used but is down or misbehaving
* The executor is old (<0.6)
* The task has services
* The services hit a pre-0.6 consul.Syncer bug
If all of those conditions are met the pre-0.6 bugs will persist for
this task until Nomad is restarted.
2017-05-09 11:28:27 -07:00
ba70cc4f01
Merge branch 'master' into f-bolt-db
2017-05-09 11:11:55 -07:00
843bc26e5d
Respond to comments
2017-05-09 10:50:24 -07:00
85210eb92f
Update consul/api to support unix socket addrs
...
Fixes #2594
2017-05-08 11:57:04 -07:00
5b8415df2c
Merge pull request #2585 from hashicorp/b-2554-container-exec
...
Execute exec/java script checks in containers
2017-05-05 10:31:18 -07:00
28e17b7a09
Test pre06ScriptCheck
2017-05-04 16:49:00 -07:00
b11d1ad1ce
Oops, remove dev logging
2017-05-04 16:27:04 -07:00
897b516117
Reuse ExecScript implementation
2017-05-04 16:21:40 -07:00
f286b6b798
Build new env from new alloc before interpolating
2017-05-04 15:06:15 -07:00
2d54ee2925
Fix tests
2017-05-03 15:14:19 -07:00
706e09748b
fix test.
2017-05-03 12:45:59 -07:00
2fb6301b37
responding to feedback on PR: remove Region from Node struct, some grammatical niceties.
2017-05-03 12:45:59 -07:00
af22cb722e
I think I did it.
2017-05-03 12:45:59 -07:00
277d5ddb36
allow region to also be interpolated, like datacenter.
2017-05-03 12:45:59 -07:00
9fa97c38a0
just found out dc and region are going to be a bit different, as Region didn't previously exist in the Node struct.
2017-05-03 12:45:59 -07:00
1a5ebace03
add helper functions to set and clear dc, region names.
2017-05-03 12:45:59 -07:00
2062d5d1dc
add dc and region to task env building function. flipped order of a couple functions to be consistent with the ordering of the rest of that section.
2017-05-03 12:45:59 -07:00
e572ec8997
starting the jelly bean trail for two new env vars
2017-05-03 12:45:59 -07:00
730e49a598
Helpful comment
2017-05-03 11:27:33 -07:00
Alex Dadgar