e7db2c9b0e
Handle Driver.Prestart returning nil, nil
2017-05-23 13:53:34 -07:00
fd9bef768f
Move task env into execcontext
...
Also inject PATH into rkt commands since we're no longer appending host
env vars for it.
2017-05-23 13:53:34 -07:00
59ad200d1a
Fix env var interpolation and env tests
2017-05-23 13:53:34 -07:00
37b148fb60
Add PortMap to struct returned by Driver.Prestart
...
Moves env.Builder out of drivers entirely so one less thing to worry
about when implementing driver plugins.
2017-05-23 13:53:34 -07:00
83c641ced0
Move env template handling into consul_template.go
2017-05-23 13:53:34 -07:00
a2eb7d17c9
Improve PortMap handling and simplify Builder creation
2017-05-23 13:53:34 -07:00
8da110855e
Move path building to task dir initialization
2017-05-23 13:53:34 -07:00
d2c08ff24b
Refactor TaskEnvironment into Builder and TaskEnv
2017-05-23 13:53:33 -07:00
10b6610e56
Functional consul template env file support
2017-05-23 13:45:14 -07:00
6d7caa5f6b
Fix for test
2017-05-19 16:59:07 -07:00
6b2d5bd4fd
Add SecurityOpt as a config field in Docker driver
2017-05-19 16:18:49 -07:00
6d76fb2f17
Merge branch 'master' of github.com:hashicorp/nomad
2017-05-17 14:46:03 -07:00
ee8dd84965
Fix nil job on allocation
...
The way the copying was happening on the alloc_runner was by temporarily
setting the alloc.Job to nil, copying and then restoring it. This
created an issue in which when the alloc was shared (which it is in
server/client mode and between alloc_runner/task_runner) there were race
conditions that could create a panic.
Fixes https://github.com/hashicorp/nomad/issues/2605
2017-05-17 14:07:06 -04:00
06f937bf28
Merge pull request #2591 from hashicorp/b-2180-script-updates
...
Properly interpolate services on updated tasks
2017-05-17 09:09:01 -07:00
323a0a78f3
Fixed wrong newline
2017-05-17 16:51:22 +02:00
ea644237cf
mac address pinning in docker driver
...
This commit adds mac address pining to the docker driver.
2017-05-17 16:41:00 +02:00
3841692138
gc_max_allocs should include blocked & migrating
2017-05-12 16:03:22 -07:00
49ce86ee0a
Lower default gc_max_allocs to 50
2017-05-12 15:57:27 -07:00
0453c2709c
Add new gc_max_allocs tuneable
...
More than gc_max_allocs may be running on a node, but terminal allocs
will be garbage collected to try to keep the total number below the
limit.
2017-05-11 17:18:02 -07:00
68c3a2bd98
Fix vet errors
2017-05-11 13:08:08 -07:00
23685d65d0
Merge pull request #2610 from hashicorp/f-bolt-db
...
Client persist state using bolt-db and more efficient write patterns
2017-05-09 13:01:36 -07:00
3cd7e06fba
Fix test
2017-05-09 11:35:48 -07:00
3b15db1ac4
Ignore Consul deregister errors on executors
...
Errors here only occur if Consul is not running when Nomad is restarted.
Errors here are only an issue if:
* Consul is being used but is down or misbehaving
* The executor is old (<0.6)
* The task has services
* The services hit a pre-0.6 consul.Syncer bug
If all of those conditions are met the pre-0.6 bugs will persist for
this task until Nomad is restarted.
2017-05-09 11:28:27 -07:00
ba70cc4f01
Merge branch 'master' into f-bolt-db
2017-05-09 11:11:55 -07:00
843bc26e5d
Respond to comments
2017-05-09 10:50:24 -07:00
85210eb92f
Update consul/api to support unix socket addrs
...
Fixes #2594
2017-05-08 11:57:04 -07:00
5b8415df2c
Merge pull request #2585 from hashicorp/b-2554-container-exec
...
Execute exec/java script checks in containers
2017-05-05 10:31:18 -07:00
28e17b7a09
Test pre06ScriptCheck
2017-05-04 16:49:00 -07:00
b11d1ad1ce
Oops, remove dev logging
2017-05-04 16:27:04 -07:00
897b516117
Reuse ExecScript implementation
2017-05-04 16:21:40 -07:00
f286b6b798
Build new env from new alloc before interpolating
2017-05-04 15:06:15 -07:00
2d54ee2925
Fix tests
2017-05-03 15:14:19 -07:00
706e09748b
fix test.
2017-05-03 12:45:59 -07:00
2fb6301b37
responding to feedback on PR: remove Region from Node struct, some grammatical niceties.
2017-05-03 12:45:59 -07:00
af22cb722e
I think I did it.
2017-05-03 12:45:59 -07:00
277d5ddb36
allow region to also be interpolated, like datacenter.
2017-05-03 12:45:59 -07:00
9fa97c38a0
just found out dc and region are going to be a bit different, as Region didn't previously exist in the Node struct.
2017-05-03 12:45:59 -07:00
1a5ebace03
add helper functions to set and clear dc, region names.
2017-05-03 12:45:59 -07:00
2062d5d1dc
add dc and region to task env building function. flipped order of a couple functions to be consistent with the ordering of the rest of that section.
2017-05-03 12:45:59 -07:00
e572ec8997
starting the jelly bean trail for two new env vars
2017-05-03 12:45:59 -07:00
730e49a598
Helpful comment
2017-05-03 11:27:33 -07:00
1d8444bc1e
Fix tests
2017-05-03 11:15:30 -07:00
e00f9c9413
Restore state + upgrade path
2017-05-02 18:21:49 -07:00
ec101b4760
Revert "metrics"
...
This reverts commit 4d6a012c6fb6f1fba6c62985d091b1a20c3198e7.
2017-05-02 09:28:11 -07:00
8e516b5dc2
Async and sync saving of client state
2017-05-01 16:16:53 -07:00
a7fd08d42a
perf
2017-05-01 16:01:50 -07:00
e010fdf8c0
metrics
2017-05-01 14:51:27 -07:00
d779defe65
Use batching
2017-05-01 14:50:34 -07:00
b94f855326
boltDB database for client state
2017-05-01 14:50:34 -07:00
bddedd7aba
Don't deepcopy job when retrieving copy of Alloc
...
This PR removes deepcopying of the job attached to the allocation in the
alloc runner. This operation is called very often so removing reflect
from the code path and the potentially large number of mallocs need to
create a job reduced memory and cpu pressure.
2017-05-01 14:50:34 -07:00
6a09d5300f
address feedback
2017-04-28 10:27:37 +01:00
418d59fd00
clean up consul earlier when destroying a task
2017-04-27 23:29:30 +01:00
cafefa049b
Properly interpolate services on updated tasks
...
Previously was interpolating the original task's services again.
Fixes #2180
Also fixes a slight memory leak in the new consul agent. Script check
handles weren't being deleted after cancellation.
2017-04-26 11:22:01 -07:00
20322a5e92
Test pre-0.6 script check upgrade path
2017-04-25 11:41:03 -07:00
b965708c51
Test env+cgroups for exec driver checks
2017-04-25 11:13:06 -07:00
c069ca0bfe
Change raw_exec to use simplified exec wrapper
2017-04-21 16:50:20 -07:00
095d2ee340
Switch java/exec to use Exec in Executor
2017-04-21 16:25:49 -07:00
a305b68159
Restart tasks on upgrade with script checks and old executors
2017-04-21 16:25:49 -07:00
a63162c7ff
Fix Windows build.
2017-04-19 13:16:48 -07:00
83f9591d75
Thanks go vet!
2017-04-19 13:05:41 -07:00
c8d3e869c6
Unregister from Consul when waiting for restart
2017-04-19 12:42:48 -07:00
16ac08ac8c
Remove stale comment
2017-04-19 12:42:47 -07:00
a3fc157233
Fix circular test imports
2017-04-19 12:42:47 -07:00
caf317e3f2
Use a DriverAbility to expose Exec functionality
2017-04-19 12:42:47 -07:00
0a59982936
Move removal from Consul into TaskRunner cleanup
2017-04-19 12:42:47 -07:00
745ad9521f
Move ScriptExecutor to driver
2017-04-19 12:42:47 -07:00
244251490a
Add UpdateTask method instead of Remove/Add
2017-04-19 12:42:47 -07:00
e204a287ed
Refactor Consul Syncer into new ServiceClient
...
Fixes #2478 #2474 #1995 #2294
The new client only handles agent and task service advertisement. Server
discovery is mostly unchanged.
The Nomad client agent now handles all Consul operations instead of the
executor handling task related operations. When upgrading from an
earlier version of Nomad existing executors will be told to deregister
from Consul so that the Nomad agent can re-register the task's services
and checks.
Drivers - other than qemu - now support an Exec method for executing
abritrary commands in a task's environment. This is used to implement
script checks.
Interfaces are used extensively to avoid interacting with Consul in
tests that don't assert any Consul related behavior.
2017-04-19 12:42:47 -07:00
4b18f916d9
Set ownership on directories in chroot
...
Also support getOwner on all Unixes as they all have `Stat_t.{U,G}id`
2017-04-17 12:41:33 -07:00
83fb374d0b
Don't disable hardlinking!
2017-04-17 11:03:15 -07:00
547d5a81b4
Chown files when copying into chroot
...
Fixes #2552
Not needed when hardlinking. Only adds Linux support but other OS's may
be easy.
2017-04-17 11:03:15 -07:00
61f4a2dac6
Sync allocation state before waiting for a destroy
...
This change ensures that the client syncs allocation state with the
servers before entering its wait loop for the allocation to be
destroyed.
Fixes https://github.com/hashicorp/nomad/issues/2563
2017-04-14 13:09:54 -07:00
67d29f684d
Merge pull request #2519 from hashicorp/b-sticky-tmp
...
Add sticky bit to temp directory
2017-04-12 14:59:14 -07:00
ad5fea9875
Merge pull request #2541 from hashicorp/f-stable-distributed-id
...
Hash host ID so its stable and well distributed
2017-04-11 11:27:53 -07:00
cdd624ff5b
Add ExtraHosts to Docker driver
...
This PR allows job submitters to add extra hosts to the containers
/etc/hosts file.
Fixes https://github.com/hashicorp/nomad/issues/2546
2017-04-11 10:52:41 -07:00
a96018fccc
Fixed typo
2017-04-10 11:45:11 -07:00
2321e8a4a0
Hash host ID so its stable and well distributed
...
This PR takes the host ID and runs it through a hash so that it is well
distributed. This makes it so that machines that report similar host IDs
are easily distinguished.
Instances of similar IDs occur on EC2 where the ID is prefixed and on
motherboards created in the same batch.
Fixes https://github.com/hashicorp/nomad/issues/2534
2017-04-10 11:44:51 -07:00
45ad1298e6
Added a test
2017-04-08 13:44:21 -07:00
eed4a9124e
Merge branch 'f-ipv6-fingerprint' of github.com:hashicorp/nomad into f-ipv6-fingerprint
2017-04-07 18:37:19 -07:00
353f193983
Removed redundant code
2017-04-07 18:28:22 -07:00
4e3e008864
Merge branch 'master' into f-ipv6-fingerprint
2017-04-07 17:11:24 -07:00
963b9baf14
Ignoring link local addresses
2017-04-07 16:04:36 -07:00
81188906a5
Merge pull request #2535 from clinta/docker-ip
...
Allow specifying container IP with docker driver
2017-04-07 12:59:12 -07:00
d71ddcb756
Allow specifying container IP with docker driver
2017-04-07 11:56:07 -04:00
aede1478db
Create AssertUntil helper func
2017-04-06 17:05:09 -07:00
b9e71f4349
Finding the appropriate cidr block
2017-04-06 16:04:59 -07:00
521354ee99
Improve test timings
...
1234ms was far longer than needed and not sleeping between iterations
was just mean.
2017-04-06 11:10:36 -07:00
a81c387adf
Require TLS for server RPC when enabled
...
Fixes #2525
We used to be checking a RequireTLS field that was never set. Instead we
can just check the TLSConfig.EnableRPC field and require TLS if it's
enabled.
Added a few unfortunately slow integration tests to assert the intended
behavior of misconfigured RPC TLS.
Also disable a lot of noisy test logging when -v isn't specified.
2017-04-06 09:34:36 -07:00
0944d9d640
Making the fingerprinter support ipv6 ips
2017-04-06 01:06:32 -07:00
d0da30324e
Split driver tests
2017-04-04 15:12:14 -07:00
f95af65c75
Split task runner tests
2017-04-04 15:02:09 -07:00
f2ba6db607
Add sticky bit to temp directory
...
Fixes an issue where the Ruby runtime expects the sticky bit to be set
on the temp directory. The sticky bit is commonly set on the temp
directory since it is usually shared by many users. This change brings
ours in line with that assumption.
2017-04-04 10:48:29 -07:00
c52000f792
FinishedAt only records when the task has actually started
2017-03-31 17:06:05 -07:00
81b78f77e1
Track task start/finish time & improve logs errors
...
This PR adds tracking to when a task starts and finishes and the logs
API takes advantage of this and returns better errors when asking for
logs that do not exist.
2017-03-31 16:14:11 -07:00
6bee23047a
Fix variable capture and add tests
...
This PR fixes token revocation and adds tests to make sure it is
working. The 0.5.6 RC1's token revocation does not work becasue the
token's value is captured at the instantiation of the deferred
stoprenewal statement rather than its exectution.
2017-03-29 13:17:50 -07:00
804f83f1fb
Fix typo and be consistent in naming
2017-03-29 09:33:54 +02:00
ae3810052d
Merge pull request #2482 from hashicorp/f-2289-better-artifact-err
...
Improve artifact download error message
2017-03-28 12:48:22 -07:00
2d07b75308
Recoverable struct field must be exported
2017-03-28 12:18:57 -07:00
1653b42400
Merge branch 'master' into b-2491-task-env
2017-03-28 11:28:00 -07:00
1e95ae7e6a
Merge pull request #2495 from hashicorp/b-vault-stop-renew
...
Stop Vault token renew on task exit
2017-03-28 11:14:18 -07:00
5e7e19de4b
Merge pull request #2461 from hashicorp/b-groups
...
Various fixes for setting user/group of task
2017-03-28 11:13:27 -07:00
3b493b29b0
Merge pull request #2488 from hashicorp/f-node-values-template
...
Access Node Meta and Attrs in template
2017-03-28 11:10:29 -07:00
5b36a90723
feedback
2017-03-28 11:10:11 -07:00
d58bcde183
Make test more accurate and add changelog entry
2017-03-28 10:53:26 -07:00
d1645f47b1
Stop Vault token renew on task exit
...
This PR fixes an oversight in which the client would attempt to renew a
token even after the task exits.
Fixes https://github.com/hashicorp/nomad/issues/2475
2017-03-28 10:53:15 -07:00
9a0c66b55b
Don't take a reference a var in a loop
...
Fixes #2491
2017-03-28 10:16:27 -07:00
41dcc72033
Recoverable is a method now, not a field
2017-03-27 15:41:35 -07:00
507862ade3
Add WrapRecoverable helper
2017-03-27 15:37:15 -07:00
23562d77c9
Access Node Meta and Attrs in template
...
This PR allows accessing the Node's attributes and metadata as in a
template.
```
template {
data = "{{ env \"attr.unique.network.ip-address\" }}"
destination = "local/out"
}
```
2017-03-27 15:15:33 -07:00
4ecebe7d8c
Proper reference counting through task restarts
...
This PR fixes an issue in which the reference count on a Docker image
would become inflated through task restarts.
2017-03-25 17:05:53 -07:00
0e6c564406
Improve artifact download error message
...
Fixes #2289
Unfortunately took more RecoverableError hijinx than I would have liked.
There might be a better way.
2017-03-24 15:26:05 -07:00
7e6c08191d
Fix vet
2017-03-24 12:24:47 -07:00
c3551c761e
Fix panic when restarting non-running task
...
This PR fixes an issue that is hit when running templates with restart
mode in which the client could panic when the handle is not running.
Fixes https://github.com/hashicorp/nomad/issues/2479
2017-03-24 12:04:22 -07:00
a171a014b3
Various fixes for setting user/group of task
...
This PR fixes two issues:
* Folder permissions in -dev mode were incorrect and not suitable for
running as a particular user.
* Was not setting the group membership properly for the launched
process.
Fixes https://github.com/hashicorp/nomad/issues/2160
2017-03-20 14:21:13 -07:00
39a8ce6cdb
Fix test deadlock due to alloc runner not running
...
Don't actualy call AllocRunner.Run() because that executes so much
unneeded code for this test. Manually close the waitCh to simulate Run()
exiting.
2017-03-16 13:22:08 -07:00
9f8067e340
Fix test by adding new env vars
2017-03-14 17:15:12 -07:00
d220d9405b
return the recoverable err
2017-03-14 16:33:36 -07:00
177bd14718
rename cpu_total_compute and docs
2017-03-14 14:15:49 -07:00
5f7bf577cd
Docker doesn't need to init the stats helper
2017-03-14 13:41:57 -07:00
a1a7941dec
Various fixes
...
This PR:
* Uses Go 1.8 executable lookup
* Stores any err message from stats init method
* Allows overriding of Cpu Compute for hosts where it can't be detected
2017-03-14 12:56:31 -07:00
2a105c4eaa
Merge pull request #2427 from hashicorp/f-gc-limit
...
Limit parallelism during garbage collection
2017-03-14 10:45:29 -07:00
7700a3447e
Review fixes
2017-03-14 10:45:15 -07:00
047b41f469
Merge pull request #2408 from hashicorp/b-2358-rkt-volume-name
...
Replace `_` with `-` in task names for rkt volumes
2017-03-14 10:10:00 -07:00
a116d6441d
Merge pull request #2426 from hashicorp/f-tg-port-ip-env-var
...
Add NOMAD_{IP,PORT}_<task>_<label> env vars
2017-03-13 12:00:59 -07:00
b5e53652aa
Handle git ssh artifacts
...
This PR adds handling for downloading git artifacts using ssh with the
format git@github.com:hashicorp/go-getter.git
Fixes https://github.com/hashicorp/nomad/issues/2430
2017-03-11 15:12:41 -08:00
3f68aae7ab
TODO
2017-03-11 13:56:19 -08:00
fd8d6d0d89
Merge pull request #2424 from jen20/solaris-alloc-dirs
...
client/allocdir: Add missing functions on Solaris
2017-03-11 13:55:38 -08:00
5716eee753
remove escape
2017-03-10 16:46:22 -08:00
70e4feb045
Limit parallelism during garbage collection
...
This PR introduces a parallelism limit during garbage collection. This
is used to avoid large resource usage spikes if garbage collecting many
allocations at once.
2017-03-10 16:27:00 -08:00
71783e8e41
Add NOMAD_{IP,PORT}_<task>_<label> env vars
2017-03-10 12:17:43 -08:00
a60127b72e
Merge pull request #2425 from hashicorp/f-client-metrics
...
Add metrics to show allocations on the client
2017-03-09 21:05:53 -08:00
8238a8601e
Address comment
2017-03-09 21:05:34 -08:00
9011a7984c
Add metrics to show allocations on the client
...
This PR adds the following metrics to the client:
client.allocations.migrating
client.allocations.blocked
client.allocations.pending
client.allocations.running
client.allocations.terminal
Also adds some missing fields to the API version of the evaluation.
2017-03-09 12:37:41 -08:00
b3d1ef0af6
client/allocdir: Add missing functions on Solaris
...
This commit adds Solaris versions of the following functions:
- `linkDir`
- `unlinkDir`
- `createSecretDir`
- `removeSecretDir`
I believe this requires Go 1.8 in order to compile, as the unlink
syscall was previously missing.
2017-03-09 13:49:14 -05:00
16adc44358
Round two of env var cleaning
...
Should bring us into conformance with IEEE Std 1003.1, 2004 Edition:
http://pubs.opengroup.org/onlinepubs/009695399/basedefs/xbd_chap08.html
1 alloc/op and ~80ns/op on my machine.
2017-03-08 16:46:13 -08:00
a724779720
Replace `_` with `-` in task names for rkt volumes
...
Fixes #2358
2017-03-07 14:34:57 -08:00
518e4f75a6
Replace "-" in env var keys with "_"
...
Fixes #2405
2017-03-06 19:57:44 -08:00
36f3788cb7
Fix lint errors
2017-03-02 21:03:05 -08:00
3fb285f7d3
Fix TestAllocRunner_SaveRestoreState
2017-03-02 20:45:46 -08:00
4cba170e4b
Mkdir -> MkdirAll to avoid error when folder already exists
2017-03-02 19:35:31 -08:00
5cd43e837a
Merge branch 'master' into b-remount
2017-03-02 19:23:13 -08:00
200d31e2e6
Fix tests broken by API change
2017-03-02 15:54:12 -08:00
8ac632c3ff
Fix API breaks in tests
2017-03-02 15:50:18 -08:00
d12949fbbd
Cleanup comments/names
2017-03-02 15:44:52 -08:00
e03f64ea6a
Safely ensure {dev,proc,alloc} are mounted
...
If they're unmounted by a reboot they'll be properly remounted.
2017-03-02 13:21:34 -08:00
d5b7c0c302
unlinkDir should not error if already unlinked
2017-03-02 13:20:47 -08:00
a9f9c485b4
Fix typos
2017-03-02 13:20:05 -08:00
f5ce13598b
Merge pull request #2391 from hashicorp/b-idempotent-umount
...
Make sure unmounting the secrets dir is idempotent
2017-03-02 11:24:01 -08:00
1cfe9f88b8
Make sure unmounting the secrets dir is idemptoent
2017-03-01 17:19:39 -08:00
2aa962a357
Merge pull request #2389 from tmichaud314/2388-docker-driver-auth-file
...
Fixes docker-driver docker.auth.config processing
2017-03-01 15:39:35 -08:00
Michael Schurter