b2a494c24e
Ensuring we can register service names with duplicate name but different ports
2017-01-17 12:26:07 -08:00
a3a3656dbb
Switch to use recoverable errors from Cleanup
...
TaskRunner handles retrying but Cleanup handles all of CreatedResources.
2017-01-13 16:46:08 -08:00
a616e9d970
Merge pull request #2163 from hashicorp/b-summary
...
Job Summary: Fix queued accounting and remove in-place state store updates
2017-01-11 13:36:36 -08:00
aafb9ca8b2
Merge pull request #2177 from hashicorp/b-blocking-getallocs
...
GetAllocs uses a blocking query
2017-01-11 13:24:32 -08:00
efbb2894c7
Review fixes
2017-01-11 13:18:36 -08:00
0f046b179a
Merge pull request #2155 from hashicorp/f-cancel
...
Cancel blocked evals upon successful one for job
2017-01-11 13:10:35 -08:00
c19985244a
GetAllocs uses a blocking query
...
This PR makes GetAllocs use a blocking query as well as adding a sanity
check to the clients watchAllocation code to ensure it gets the correct
allocations.
This PR fixes https://github.com/hashicorp/nomad/issues/2119 and
https://github.com/hashicorp/nomad/issues/2153 .
The issue was that the client was talking to two different servers, one
to check which allocations to pull and the other to pull those
allocations. However the latter call was not with a blocking query and
thus the client would not retreive the allocations it requested.
The logging has been improved to make the problem more clear as well.
2017-01-10 13:30:35 -08:00
d0f918495b
Store pointer of JobSummary in state store and remove in-place modifications of the object and replace with Copy-Update-Insert operations
2017-01-08 13:55:03 -08:00
c253f5b17d
Fixed merging consul config
2017-01-05 15:15:43 -08:00
8d5f0fea69
Merge pull request #2128 from hashicorp/f-dispatch
...
Nomad Constructor Jobs and Dispatch
2017-01-06 05:22:49 +08:00
86980e08f0
Cancel blocked evals upon successful one for job
...
This PR causes blocked evaluations to be cancelled if there is a
subsequent successful evaluation for the job. This fixes UX problems
showing failed placements when there are not any in reality and makes GC
possible for these jobs in certain cases.
Fixes https://github.com/hashicorp/nomad/issues/2124
2017-01-04 16:16:04 -08:00
4a9a574d9d
Merge pull request #2054 from hashicorp/f-prestart
...
Add Driver.Prestart method
2016-12-20 16:18:56 -08:00
2aa235f8f2
Rename InitializationMessage to DriverMessage
2016-12-20 11:51:09 -08:00
9cdd576720
Updated changelog and fixed tests
2016-12-20 11:32:17 -08:00
9786708526
Added comments
2016-12-19 18:10:02 -08:00
4e8035756b
Fix test and prevent job with payload from being submitted
2016-12-18 16:32:14 -08:00
072ff1c3ee
ensure file doesn't escape
2016-12-18 15:48:30 -08:00
aa904b495e
Require batch
2016-12-16 15:20:12 -08:00
2761e1d8ea
fix tests
2016-12-16 10:21:56 -08:00
a1dd78c24b
Scheduler combines meta from job > group > task
2016-12-15 17:08:38 -08:00
d8de7b59a5
JobDiff
2016-12-15 15:40:18 -08:00
bf1e157bd8
Children fixes + nomad status outputs summaries
...
Children object is always initialized instead of lazily.
`nomad status` outputs children summaries and has specialized view for
constructor jobs.
2016-12-14 16:58:54 -08:00
5a742528c9
dispatch help
2016-12-14 15:19:27 -08:00
4a5c3c8db0
Rename structs
2016-12-14 14:28:43 -08:00
1235fc6581
summary tests
2016-12-13 16:15:40 -08:00
af2865ea48
Don't modify jobs status inplace
2016-12-12 11:42:47 -08:00
8885ab70a6
Handle the delete case
2016-12-06 20:15:10 -08:00
ef79e77e52
Children summary
2016-12-06 17:06:57 -08:00
c005fcb973
Add structs
2016-12-05 17:24:37 -08:00
92228da6a8
Dispatch tests
2016-12-02 15:37:26 -08:00
770ed703d0
Add Driver.Prestart method
...
The Driver.Prestart method currently does very little but lays the
foundation for where lifecycle plugins can interleave execution _after_
task environment setup but _before_ the task starts.
Currently Prestart does two things:
* Any driver specific task environment building
* Download Docker images
This change also attaches a TaskEvent emitter to Drivers, so they can
emit events during task initialization.
2016-12-02 11:03:48 -08:00
f0b6d5953c
Remove paused
2016-12-01 13:17:34 -08:00
b232dc4c94
Decompress
2016-11-28 16:05:56 -08:00
ac3ed04961
Snappy + Dispatch name change
...
Change the generated name to timestamp + 8 characters of UUID
Add Snappy compression to the input
2016-11-25 20:02:18 -08:00
cfd9593e7a
dispatch beginning
2016-11-25 18:04:55 -08:00
5191b4d33a
Making the status command return the allocs of currently registered job
2016-11-24 16:31:30 +01:00
8fb83fbb6c
Parse
2016-11-23 15:48:36 -08:00
54bcde8e36
Dispatch structs
2016-11-23 15:03:13 -08:00
b7765a6bf5
Removing QueryMeta from ServerMembersResponse
2016-11-10 15:42:42 -08:00
56ed1d3cd8
Fixing the upgrade path for ephemeral disk
2016-11-08 15:24:51 -08:00
822e32de6d
Fix error checking
2016-11-08 11:04:11 -08:00
981fabdb26
Add unrecoverable test
2016-11-08 10:38:54 -08:00
a1b4b20eff
Changing the API to use the new servermember datastructure
2016-11-06 10:30:43 -08:00
9e0507e878
Add an RPC endpoint for server members
2016-11-06 10:16:12 -08:00
15f085a4d7
Merge pull request #1931 from hashicorp/rename-vault-config
...
Rename vault config
2016-11-06 10:14:25 -08:00
40b9d3bb2d
Fixed comment
2016-11-03 14:45:03 -07:00
22681bd8ce
Making AllowUnauthenticated true by default
2016-11-03 14:38:34 -07:00
b6f9df5415
Renaming TLS related vault config
2016-11-03 14:24:39 -07:00
2dbdf0f755
Adding a default ephemeral disk if it's not added by default
2016-11-02 15:07:22 -07:00
3f69ab130e
can not -> cannot
2016-11-01 16:05:51 -07:00
cf38103b85
Change the 0 timeout, interval validation to be more user friendly
2016-11-01 16:05:51 -07:00
808718523b
Merge pull request #1909 from hashicorp/region-wrapper
...
Making Nomad TLS configs region aware
2016-11-01 14:31:39 -07:00
536c2921e9
Remove ServerName because we verify based on region
2016-11-01 14:17:31 -07:00
1a8fa8c8d5
Making Nomad TLS configs region aware
2016-11-01 11:55:29 -07:00
8f365bfbbf
Update vault_test to use minimal set of policies + start of Vault documentation
2016-11-01 10:39:15 -07:00
df4398beac
Implement blocking queries for /v1/job/evaluations
2016-10-29 17:30:34 -07:00
fde7a24865
Consul-template fixes + PreviousAlloc in api
2016-10-28 15:50:35 -07:00
ffefb6d3c1
Fix flaky test
2016-10-27 11:48:00 -07:00
9c9ab621c3
Merge pull request #1872 from hashicorp/f-dequeue
...
Add scheduler version enforcement
2016-10-27 11:42:11 -07:00
bbe6e3d0b6
Larger delay on mismatch
2016-10-27 11:41:43 -07:00
ddf5fb82b5
Small cleanups
2016-10-27 10:51:11 -07:00
03fde26656
Fix inplace update for pre secret node
2016-10-26 22:05:44 -07:00
a1d08c2aba
Add scheduler version enforcement
2016-10-26 14:52:48 -07:00
84722234b4
Fixed a bunch of TLS related failures
2016-10-26 14:08:46 -07:00
8e3d9d4f4f
Merge pull request #1864 from hashicorp/b-allow-empty-resource
...
Allow omitting resource block entirely
2016-10-26 13:26:04 -07:00
41c5bdb510
Document missing task resources handling
2016-10-26 13:21:09 -07:00
067fcda3fe
Making the cli use TLS if the client has enabled TLS
2016-10-26 11:13:53 -07:00
3ea1fe8d20
Allow omitting resource block entirely
2016-10-26 10:36:41 -07:00
50ca5e1e9d
Merge pull request #1853 from hashicorp/f-rpc-http-tls
...
TLS support for http and RPC
2016-10-25 16:14:43 -07:00
7c61e115bd
Moved tlsutil into helpers
2016-10-25 16:05:37 -07:00
cf35aeac84
Moving the TLSConfig to structs
2016-10-25 15:57:38 -07:00
d88f1d3e3d
Merge pull request #1854 from hashicorp/f-vault-role-validate
...
Validate the Vault role being used
2016-10-25 14:41:40 -07:00
d3649f5d98
check period
2016-10-25 14:37:54 -07:00
4082732d3a
Interpolate and then validate services
2016-10-25 14:27:49 -07:00
8e07c2750e
Merge pull request #1839 from hashicorp/f-signal-constraints
...
Signal creates an auto-constraints
2016-10-25 11:09:33 -07:00
22eb507955
Upper case signals
2016-10-25 11:09:22 -07:00
3d04efb21f
Validate the Vault role being used
2016-10-24 16:53:47 -07:00
03eba049ed
Merge pull request #1848 from hashicorp/f-vault-error
...
Thread through whether DeriveToken error is recoverable or not
2016-10-24 15:01:18 -07:00
fc8856e82b
Remove default
2016-10-24 15:01:08 -07:00
e03927bb5c
Changed the way TLS config is parsed
2016-10-24 13:56:19 -07:00
e85d0ebace
Merge pull request #1840 from hashicorp/f-kill-fail
...
Change how we mark tasks as failed and allow consul-template to fail tasks
2016-10-24 13:40:52 -07:00
2e3118e69c
Implemented TLS support for http and rpc
2016-10-23 22:22:00 -07:00
ede3a814ba
Small fixes
2016-10-22 18:20:50 -07:00
0070178741
Thread through whether DeriveToken error is recoverable or not
2016-10-22 18:08:30 -07:00
285e80ac0f
Remove disk usage enforcement
...
Many thanks to @iverberk for the original PR (#1609 ), but we ended up
not wanting to ship this implementation with 0.5.
We'll come back to it after 0.5 and hopefully find a way to leverage
filesystem accounting and quotas, so we can skip the expensive polling.
2016-10-21 13:55:51 -07:00
46a7d1a0d7
Change how we mark tasks as failed and allow consul-template to fail tasks
2016-10-20 17:27:16 -07:00
aadc9e3017
Add implicit signal constraint and validate that a driver can handle the signal. Also fixes a bug with plan and implicit constraints by adding them to the job being planned
2016-10-20 13:55:35 -07:00
989827e402
Add set contains
2016-10-19 13:06:28 -07:00
7f17c93f89
Ensuring that ephemeral disk is present before deciding if we should migrate ( #1827 )
2016-10-19 11:12:25 -07:00
d6606ecbe5
tests
2016-10-18 14:54:14 -07:00
36cfe6e89e
Large refactor of task runner and Vault token rehandling
2016-10-18 11:24:20 -07:00
af036be754
Struct/api/parsing
2016-10-18 11:24:20 -07:00
53eeec9bc1
Merge pull request #1801 from hashicorp/f-signals
...
Consul-template signal change mode
2016-10-18 11:23:47 -07:00
a0dcd4a228
comment
2016-10-18 11:23:27 -07:00
0f6e0d10b6
Enable serf encryption ( #1791 )
...
* Added the keygen command
* Added support for gossip encryption
* Changed the URL for keyring management
* Fixed the cli
* Added some tests
* Added tests for keyring operations
* Added a test for removal of keys
* Added some docs
* Fixed some docs
* Added general options
2016-10-17 10:48:04 -07:00
db4e676d73
Merge pull request #1803 from hashicorp/b-vault-parse
...
Fix Vault Config parsing of booleans
2016-10-11 13:47:46 -07:00
82960c46d8
Tests
2016-10-11 13:28:18 -07:00
83f647ed84
Replace "the the" with "the" in documentation and comments
2016-10-11 15:31:40 -04:00
751aa114bf
Fix Vault parsing of booleans
2016-10-10 18:04:39 -07:00
3d0edaa91e
Ensure templates have different destinations
2016-10-10 15:19:00 -07:00
bc35eaee21
Task runner sends signals
2016-10-10 15:09:00 -07:00
68c5fe78f8
Tests
2016-10-06 15:17:34 -07:00
8fb07bb083
Fix handling of restart in TaskEvents
2016-10-06 15:06:54 -07:00
8eb7fa91cf
Start of integration
2016-10-06 15:05:49 -07:00
3693de99d8
Merge pull request #1783 from hashicorp/f-consul-template
...
Consul template manager
2016-10-06 15:05:01 -07:00
acdbc2e1c9
Fix diff test
2016-10-06 14:14:59 -07:00
19a6aefd68
more vendoring
2016-10-06 12:36:44 -07:00
d2837dec44
Do not allow path to escape the alloc dir for the FS commands
2016-10-03 14:58:44 -07:00
4eaabd675c
Consul Template Manager
2016-10-03 12:59:31 -07:00
d50c395421
Getting snapshot of allocation from remote node ( #1741 )
...
* Added the alloc dir move
* Moving allocdirs when starting allocations
* Added the migrate flag to ephemeral disk
* Stopping migration if the allocation doesn't need migration any more
* Added the GetAllocDir method
* refactored code
* Added a test for alloc runner
* Incorporated review comments
2016-10-03 09:59:57 -07:00
9b98d70aec
Fixed task group validation tests
2016-09-27 17:57:30 -07:00
80085ddda5
Merge pull request #1735 from hashicorp/b-bootstrap-flapping
...
Retry all servers on RPC call failure
2016-09-27 16:33:15 -07:00
fca2becaa5
Parse
2016-09-26 15:36:11 -07:00
b4c3f015bc
Template diff
2016-09-26 15:36:11 -07:00
e925f8ca58
Add template structs
2016-09-26 15:36:11 -07:00
dae7f88118
Not setting a drained node as preferred node ( #1740 )
2016-09-23 21:15:50 -07:00
2ab5264595
Retry all servers on RPC call failure
...
rpcproxy is refactored into serverlist which prioritizes good servers
over servers in a remote DC or who have had a failure.
Registration, heartbeating, and alloc status updating will retry faster
when new servers are discovered.
Consul discovery will be retried more quickly when no servers are
available (eg on startup or an outage).
2016-09-23 11:44:48 -07:00
d64ef28c39
Handle the various valid root cases
2016-09-21 17:30:57 -07:00
f99d84d2c3
Renew root tokens where applicable
2016-09-21 16:49:15 -07:00
08d79893b0
Merge pull request #1727 from kaskavalci/patch-4
...
Change error message
2016-09-21 13:53:13 -07:00
59df3dcf58
Vault diff
2016-09-21 13:49:34 -07:00
12de69a66f
Struct and parse
2016-09-21 11:31:09 -07:00
5a9919bfbf
Fix VaultPolicies returning non-empty map when there are no policies
2016-09-21 11:14:09 -07:00
d166f8e3bf
Lowercase message
2016-09-21 20:43:15 +03:00
389beda383
Change error message
...
Local Disk -> Ephemeral Disk
2016-09-21 17:57:19 +03:00
50efdb00e9
Merge pull request #1713 from hashicorp/f-alloc-runner-vault
...
Vault integration in client
2016-09-20 16:15:55 -07:00
64de46432a
Merge pull request #1677 from hashicorp/f-vault-implicit-constraint
...
Vault implicit Task Group constraint + allow root tokens
2016-09-20 16:15:32 -07:00
ada5d8dd0c
alloc-status
2016-09-14 18:27:13 -07:00
1b3c5e98c8
Renaming LocalDisk to EphemeralDisk ( #1710 )
...
Renaming LocalDisk to EphemeralDisk
2016-09-14 15:43:42 -07:00
6702a29071
Vault token threaded
2016-09-14 13:30:01 -07:00
6028682ad2
Adding LocalDisk to alloc.Job
2016-09-01 17:41:50 -07:00
874dfbc84a
Implicit vault constraint
2016-09-01 14:23:40 -07:00
b0adaa5301
Allow root token
2016-09-01 12:05:08 -07:00
3c9936ae4a
Merge pull request #1659 from hashicorp/f-revoke-accessors
...
Token revocation and keeping only a single Vault client active among servers
2016-08-31 14:10:46 -07:00
6047414fb9
address comments
2016-08-31 14:10:33 -07:00
d94bb45ad3
Added some more comments
2016-08-31 14:06:31 -07:00
64c57d9136
Added a test
2016-08-31 13:40:43 -07:00
52e9946da9
Implemented SetPrefferingNodes in stack
2016-08-30 16:17:50 -07:00
bfee7b30a3
Introducing shared resources in alloc
2016-08-29 13:49:25 -07:00
48696ba0cc
Use tomb to shutdown
...
Token revocation
Remove from the statestore
Revoke tokens
Don't error when Vault is disabled as this could cause issue if the operator ever goes from enabled to disabled
update server interface to allow enable/disable and config loading
test the new functions
Leader revoke
Use active
2016-08-28 14:06:25 -07:00
3dec7cd2c9
Added LocalDisk to diff
2016-08-26 20:38:50 -07:00
13497913f9
Ensuring resources are re-calculated properly in fsm
2016-08-26 20:13:11 -07:00
e79cb67391
Changing implementation of AllocsFit
2016-08-26 17:28:29 -05:00
2f681b6415
Added copy method to LocalDisk
2016-08-26 14:24:47 -05:00
b844dc3600
Fixed more tests
2016-08-26 01:51:19 -05:00
3447658bba
Added scheduler tests to ensure disk constraints are honored
2016-08-25 15:31:56 -05:00
ffaf6c6299
Fixed some tests
2016-08-25 13:56:39 -05:00
8105613c25
Added an upgrade path for existing jobs with no local disk
2016-08-25 13:00:20 -05:00
ec73c768f1
Making the scheduler use LocalDisk instead of Resources.DiskMB
2016-08-25 12:27:42 -05:00
d156f32f94
Implemented job spec parsing for sticky volumes
2016-08-24 13:51:15 -05:00
05fe72e89e
fixed the exec fingerprinter test
2016-08-23 16:40:56 -05:00
db2806a9c1
Merge pull request #1629 from hashicorp/f-derive-token
...
Server Deriving Tokens on behalf of Clients
2016-08-23 13:58:47 -07:00
1da8566322
Merge pull request #1580 from hashicorp/f-disk-usage-monitoring
...
Monitor and enforce shared allocation directory disk usage
2016-08-23 09:49:53 -07:00
76d324a8f0
fix comment
2016-08-22 11:41:47 -07:00
4ca623bcfe
blocking chained allocations until previous allocation hasn't terminated
2016-08-22 11:34:24 -05:00
19be6b57b2
fixes
2016-08-19 20:02:32 -07:00
7f18074637
remove debug
2016-08-19 17:30:16 -07:00
c394a45b72
create token tests
2016-08-19 16:40:37 -07:00
909f552d9e
tests
2016-08-19 16:40:37 -07:00
10cd844ca8
Commit Vault Accessors to vault and return the response
2016-08-19 16:40:37 -07:00
901000f789
Raft message, fsm and state store table
2016-08-19 16:40:37 -07:00
19752edfaf
Pipeline Vault token creation
2016-08-19 16:40:37 -07:00
123a26ffea
Rate limiting
2016-08-19 16:40:37 -07:00
94b870a58b
Start
2016-08-19 16:40:37 -07:00
e7e8d5ab50
Merge pull request #1597 from hashicorp/f-node-secret-id
...
Nodes generate Secret ID and used retrieving allocations/registering
2016-08-19 10:51:21 -07:00
16285a0dc6
Enforce serverside secret id match
2016-08-19 10:50:49 -07:00
f8cd51b6e9
Enabling vault if token is present
2016-08-18 12:03:50 -07:00
2a17895a83
Disk resource monitoring and enforcement
2016-08-18 07:59:03 +02:00
895c31f605
Nodes generate Secret ID and used for retrieving allocations and registering
2016-08-17 16:31:47 -07:00
cabc9f829d
Merge pull request #1614 from hashicorp/f-nomad-0.5
...
F nomad 0.5
2016-08-17 16:27:50 -07:00
9bd9948c5b
Job Register endpoint validates token
2016-08-17 16:25:38 -07:00
f9f019fa62
LookupToken
2016-08-17 16:25:38 -07:00
a981fb4e0e
test renewal
2016-08-17 16:25:38 -07:00
a8efce874f
Token renewal and beginning of tests
2016-08-17 16:25:38 -07:00
713e310670
Renew loop
2016-08-17 16:25:38 -07:00
750a44b2c0
Create a Vault interface for the server
2016-08-17 16:25:38 -07:00
6e2f0a2776
Server has Vault API client
2016-08-17 16:25:38 -07:00
59892e4004
Merge pull request #1612 from hashicorp/diptanu-0-5-branch
...
Chained Allocs
2016-08-17 16:23:32 -07:00
4135b4ece7
Address field name feedback
2016-08-17 16:23:29 -07:00
bd0a89bdae
Added mapstructure tag for VaultToken
2016-08-17 16:23:29 -07:00
ad088db6eb
Store the Vault token once per job
2016-08-17 16:23:29 -07:00
7d899b6c60
Pass Vault config to client
2016-08-17 16:23:29 -07:00
14b4312502
Add vault struct
2016-08-17 16:23:29 -07:00
eac2675faf
Add enabled field
2016-08-17 16:23:29 -07:00
c913e4396f
Add Vault config to server
2016-08-17 16:23:29 -07:00
1584cfe93e
small fixes
2016-08-17 16:23:29 -07:00
0ca4a9fa4f
Change token/role names
2016-08-17 16:23:29 -07:00
adb3ce847f
change config variable names to match vault
2016-08-17 16:23:29 -07:00
fab7893774
vendor + api
2016-08-17 16:23:29 -07:00
b32128aa23
Initial config block
2016-08-17 16:23:29 -07:00
de6e662eb4
Fix service validate test
2016-08-17 11:09:40 -07:00
a3bcc1cbb1
Fix network dynamic port test
2016-08-17 11:08:21 -07:00
be51f1b265
Fix TaskDiff test
2016-08-17 11:07:11 -07:00
b6c39349b7
struct: tiny: output case number of the diff test
2016-08-17 19:15:59 +09:00
2e22fea61d
Merge pull request #1599 from hoffoo/initial_check_state
...
Add support for initial check status
2016-08-16 15:16:47 -07:00
8a23780aee
Fix bitmap test and check bitmap bounds
2016-08-16 15:16:35 -07:00
8fc52974a3
fix initial status tests
2016-08-16 14:34:36 -07:00
69bc3a8fc8
Add support for initial check status
2016-08-16 12:05:15 -07:00
c1a455983d
Added the chained alloc for system scheduler
2016-08-16 10:49:45 -07:00
ce0b78525d
inclusive range
2016-08-15 13:13:04 -07:00
761cc40cd2
Fixed a make vet warning
2016-08-12 12:09:44 -07:00
dd7e69006e
Not running tests parallal
2016-08-11 21:53:27 -07:00
01e08a64ee
Merge pull request #1569 from hashicorp/fix-network-port-collisions
...
Fix network port collisions when asking for dyn ports
2016-08-11 16:19:45 -07:00
d81b20c1a6
Fix network port collisions when asking for dyn ports
2016-08-11 16:18:45 -07:00
007a538515
Fix core scheduler tests
2016-08-11 14:36:22 -07:00
6e5c47a315
Merge pull request #1526 from hashicorp/b-random-ports
...
Set difference when picking random ports
2016-08-10 16:37:57 -07:00
5a37e720c5
Fixes plus address feedback
2016-08-10 16:37:26 -07:00
b8fd989d3a
Try stochastic and fallback to precise
2016-08-10 11:47:20 -07:00
ab94c8eed9
Marking allocations which are not terminal and are on down nodes as lost
2016-08-09 13:11:58 -07:00
3cc684211a
Added a test to ensure summaries are correct when a node goes down
2016-08-09 10:16:17 -07:00
c63a78b9a3
Removing the check related to checking version of server before reconciling in leader
2016-08-05 17:48:37 -07:00
1518f23d0a
Making servers reconcile job summaries when they acquire leadership
2016-08-05 16:47:36 -07:00
1b620bcdd8
Add a test
2016-08-05 16:23:41 -07:00
9089a279a1
Set difference when picking random ports
2016-08-05 16:08:35 -07:00
6dc5b1972c
Setting job's create index as summary create index during reconciliation
2016-08-04 15:14:01 -07:00
2fb67fefb5
Merge pull request #1516 from hashicorp/f-lost-state-sched
...
Make scheduler mark allocations as lost
2016-08-04 11:36:02 -07:00
88d383c47f
Updated tests and comments
2016-08-04 11:29:36 -07:00
e33bda76bf
test sched doesn't mark complete as lost + core_sched tests
2016-08-04 11:24:17 -07:00
c24e8ba7d8
Not updating summary if job is de-registered
2016-08-03 17:00:08 -07:00
74caed0c7a
Added an endpoint for users to reconcile job summaries
2016-08-03 16:12:47 -07:00
ac3328e812
Make scheduler mark allocations as lost
2016-08-03 15:57:46 -07:00
1b60e0823a
Added a test for restoring the summaries in fsm
2016-08-03 11:58:36 -07:00
4197e62e78
Remove old way of marking lost
2016-08-03 11:20:56 -07:00
b95cf91ee3
using the job associated with the alloc to determine if job is present
2016-08-02 19:14:05 -07:00
6f8c40fca7
Not updating summary if create index of summary not same as job's create index
2016-08-02 18:59:45 -07:00
b2d388bcba
Merge pull request #1508 from hashicorp/b-dont-update-job
...
Do not update the job of allocations that are being stopped
2016-08-02 18:58:39 -07:00
2332a58944
Do not update the job of allocations that are being stopped
2016-08-02 17:53:31 -07:00
87fdeb5393
Updated the logic to update job summary
2016-08-02 16:08:20 -07:00
3966a46996
Updating the summary after we have updated the current allocation when client updates the alloc
2016-08-02 15:06:39 -07:00
92e32e46f1
Updating the summary after we have updated the current allocation
2016-08-02 14:59:41 -07:00
3aa4f39094
Checking if a job is nil before updating the allocation
2016-08-01 17:05:48 -07:00
b69b7129a6
Using the parnet transaction to query the allocation while updating summary
2016-08-01 16:46:05 -07:00
b0e1f02e26
Not updating job summaries if jobs are not present
2016-07-28 15:24:27 -07:00
0dd8a84de0
Marking the desired state of an allocation as stop if the node on which it runs disconnects
2016-07-27 17:07:08 -07:00
b857d7c6c1
Copying job summary before mutating it
2016-07-27 14:46:46 -07:00
1bab053490
Updated some tests
2016-07-26 15:11:48 -07:00
10a5c06a5a
Running the tests in verbose mode
2016-07-26 14:02:47 -07:00
9943053239
Fixed a test
2016-07-25 22:22:55 -07:00
d1a6bdb4ba
Making the queued allocations bind late
2016-07-25 22:11:11 -07:00
5bded8d54d
Setting the right indexes while creating Job Summary
2016-07-25 17:51:20 -07:00
3089833397
Reconciling the queued allocations during restore
2016-07-25 17:31:40 -07:00
cc37ec33cf
Renamed Job.GetJobSummary to Job.Summary
2016-07-25 17:31:40 -07:00
6193529040
Fixed more tests
2016-07-25 17:31:40 -07:00
de2c79f421
Added test for blocking query of job summary endpoint
2016-07-25 17:26:38 -07:00
cce5f483ae
Added some more tests
2016-07-25 17:26:38 -07:00
f1c9427c37
Added code to create missing job summaries
2016-07-25 17:26:38 -07:00
dabb83063b
Review comments
2016-07-25 17:26:38 -07:00
50842b88c7
Fixed some bugs
2016-07-25 17:26:38 -07:00
1405687a88
Fixed some error messages and conditions
2016-07-25 17:26:38 -07:00
ef97956333
Added support for retreiving job summary in api
2016-07-25 17:26:38 -07:00
632ced5eb2
Adding the summary to the Job Stub
2016-07-25 17:26:38 -07:00
a5bb0ca6fc
Moved the job endpoint around
2016-07-25 17:26:38 -07:00
7bafb7c675
Updating the job summary while mutating jobs and allocation objects
2016-07-25 17:26:38 -07:00
109b05cb29
Applying changes to job updates via FSM
2016-07-25 17:26:38 -07:00
affbf5b6e4
Updating the job summary table only if an evaluation has any Queued Allocations
2016-07-25 17:26:38 -07:00
2ff2acbfc0
Added the job summary related endpoints
2016-07-25 17:26:38 -07:00
1cc0bc392b
Setting the number of queued allocations per task group
2016-07-25 17:26:38 -07:00
e26f826189
fix job gc tests
2016-07-25 14:56:23 -07:00
0db55c1dce
Revert "Fix job gc tests"
...
This reverts commit 4be50ac8c78b09d603d9680064391d449b268436.
2016-07-25 14:53:07 -07:00
e61aa2484a
Fix job gc tests
2016-07-25 14:49:57 -07:00
42df093939
Merge pull request #1456 from hashicorp/b-system-job
...
Node Register handles transistioning to ready and creating evals
2016-07-25 12:46:35 -07:00
c4d7f62189
add down to up test
2016-07-25 12:46:18 -07:00
90748cedad
Add killing event and mark task as not running when killed
2016-07-21 15:49:54 -07:00
ebac5cb283
Node.Register handles the case of transistioning to ready and creating evals
2016-07-21 15:22:02 -07:00
af09ef0832
fix validation tests
2016-07-20 16:43:20 -07:00
e0114fee05
InitFields to Canonicalize
2016-07-20 16:08:52 -07:00
0634eeb3e0
Sanatize incoming slices/maps
2016-07-20 16:00:02 -07:00
c8a52f36d5
Merge pull request #1429 from nak3/default-resources
...
Update comments for the DefaultResources and DefaultLogConfig
2016-07-20 10:09:36 -07:00
d7e397d3f9
Merge pull request #1439 from nak3/fix-error-message
...
Add missed service name of the error message for the invalid port
2016-07-20 10:08:45 -07:00
473eb6561a
Stop using index for task and task group validation
2016-07-20 22:23:35 +09:00
c24e886a5f
Add missed service name of the error message for the invalid port
2016-07-20 20:41:24 +09:00
466d7ac1ec
Update comments for the DefaultResources and DefaultLogConfig
2016-07-19 15:37:54 +09:00
c28027bc9e
Merge pull request #1421 from hashicorp/f-system-count-zero
...
Allow count 0 on system jobs
2016-07-13 14:39:23 -06:00
6bc7009f8c
Allow count 0 on system jobs
2016-07-13 13:50:08 -06:00
3836d6e54e
Merge pull request #1383 from hashicorp/f-job-summary
...
Job Summary - Part 1
2016-07-13 13:34:27 -06:00
e35369ec83
Fixed typos in comments
2016-07-13 13:25:07 -06:00
487c66b84d
Removing the queued state of Job Summary and alloc desired status false
2016-07-13 13:20:46 -06:00
c8e7b909c7
Merge pull request #1404 from hashicorp/f-streaming
...
Implement a streaming API and tail in the fs command
2016-07-12 17:23:04 -06:00
daa83a4f3e
Renamed jobsummary to job_summary
2016-07-12 16:00:35 -06:00
5d782abd50
Refactored the test
2016-07-12 14:37:51 -06:00
00b9b4c6e8
Accounting lost state of allocations
2016-07-12 14:27:45 -06:00
b87cf12f6f
Merge pull request #1403 from hashicorp/f-hold-rpc
...
Gracefully handle short lived outages by holding RPC calls
2016-07-12 13:52:33 -06:00
e8d1aee3f4
Added a method for listing jobs whose id matches a prefix
2016-07-12 11:41:13 -06:00
313d7aa7f5
Added a test to ensure client alloc updates are happening properly
2016-07-12 11:41:13 -06:00
91b828d299
Updated logic to handle change in desired status of allocation when client status is still pending
2016-07-12 11:41:13 -06:00
6937c0f7f3
Added test for job summary restore
2016-07-12 11:41:13 -06:00
5e6f9ef69e
Added methods to save and restore job summary snapshots
2016-07-12 11:41:13 -06:00
ba71757dfb
handled the logic of task group count going up
2016-07-12 11:41:13 -06:00
67953b1583
Added a test to ensure correctness of job summary when client updates alloc
2016-07-12 11:41:13 -06:00
837b70f285
Added test to make sure summary gets deleted when job gets deleted
2016-07-12 11:41:13 -06:00
0606840080
Implemented logic to update the job summary when allocs are inserted
2016-07-12 11:41:13 -06:00
083f81d17f
Implemented job state accounting logic for upsert job
2016-07-12 11:41:13 -06:00
ebf9fbf1d6
Added a schema for summarizing status of jobs
2016-07-12 11:41:13 -06:00
4ea9ceee38
Handling allocations with client state pending
2016-07-12 11:29:23 -06:00
2cf2ed6758
Changing the state of an allocation to lost if the node on which it was running was marked as down
2016-07-11 18:24:04 -06:00
bc0bfc3ae5
Merge pull request #1398 from hashicorp/b-check-timeout
...
Fixed the validation logic for check timeout and interval
2016-07-10 12:16:50 -07:00
51ae7ace25
initial tail impl
2016-07-10 13:57:04 -04:00
75abbc74a5
nomad: modify forward RPC to hold when no known leader
2016-07-10 13:36:55 -04:00
699c4fc68c
nomad: Add RPCHoldTimeout to tune RPC hold interval
2016-07-10 13:35:48 -04:00
b4fe764f07
Added a test
2016-07-08 22:33:04 -07:00
19f0867816
Fixed the validation logic for check timeout
2016-07-08 22:26:03 -07:00
48b9684b1e
Using net.JoinHostPort instead of handcrafting addrs
2016-07-08 16:45:14 -07:00
b180223f4b
Allowing ports to be overriden in check definitions
2016-07-08 14:14:25 -07:00
099cee067d
comments
2016-06-28 10:02:06 -07:00
3f0a47f9e4
Disallow EvalGC to reap batch jobs evals/allocs and make JobGC only oneshot GCs everything
2016-06-27 22:54:03 -07:00
6ca552c451
Reblock test
2016-06-24 10:26:13 -07:00
fd3e469d5e
Remove requeue because it is a subset of EnqueueAll now
2016-06-24 10:14:34 -07:00
2f8bb4b235
When enqueuing into eval broker always pass blocked eval's token
2016-06-23 22:40:22 -07:00
ccf93d7e44
UnblockFailed needs to untrack the job
2016-06-23 15:35:21 -07:00
b1c2a9ddb9
UnblockFailed needs to untrack the job
2016-06-23 15:26:26 -07:00
3a8a27bcff
refresh index eval id in log
2016-06-22 13:48:41 -07:00
e43c460534
Fixed name of a test
2016-06-22 13:04:54 -07:00
0fe8746692
GC-ing dead batch jobs
2016-06-22 11:40:27 -07:00
8ceb7ead20
Do not use snapshot
2016-06-22 09:33:15 -07:00
91f6976423
tighter index bound when creating GC evals
2016-06-22 09:11:25 -07:00
25decca3ca
Worker waitForIndex uses StateStore index, not Raft Applied Index
2016-06-22 09:04:22 -07:00
8bdb38d016
Code golf
...
Pointed out by: @dadgar
2016-06-21 14:26:01 -07:00
df4fe2e502
Fix the shuffling of remote datacenters.
...
Pointed out by: @ryanuber
2016-06-21 13:37:22 -07:00
9e287858de
Merge pull request #1310 from hashicorp/b-logger
...
Create and pass only one `logger` object around per Agent
2016-06-17 12:16:35 -07:00
46e2d54acf
Provide `nomad.Config` with a default `LogOutput` of `os.StdErr`
2016-06-17 06:44:10 -07:00
9a60999100
Pass a logger arg to `NewClient` and `NewServer`
2016-06-16 23:29:23 -07:00
871a31a8ec
Teach config.ConsulConfig how to construct a consulapi TLS client.
...
Said differently, centralize the creation of consul's client config
in one place and use it everywhere.
2016-06-16 22:51:06 -07:00
d17af396ca
Create config.DefaultConsulConfig()
2016-06-16 20:41:05 -07:00
a658299235
Misc typos
2016-06-16 16:17:17 -07:00
ec77a1869e
Test for errors
2016-06-16 14:43:46 -07:00
31313b68cf
Don't assign to an atomic w/o using atomic setter func
2016-06-16 14:43:46 -07:00
af55b74114
Merge pull request #1276 from hashicorp/f-consul-server-autojoin
...
Teach Nomad servers how to fall back to Consul.
2016-06-16 14:40:45 -07:00
7c24487850
Fix up various error handling
2016-06-16 14:40:09 -07:00
71cd9984ae
Immediately query Consul upon initialization if we have no peers.
...
Also don't attempt to join the Server with itself.
2016-06-16 14:27:10 -07:00
65319252b9
Rework `server_auto_join` to use a timer instead of the peer count.
...
It is perfectly viable for an admin to downsize a Nomad Server cluster
down to 1, 2, or `num % 2 == 0` (however ill-advised such activities
may be). And instead of using `bootstrap_expect`, use a timeout-based
strategy. If the `bootstrapFn` hasn't observed a leader in 15s it will
fall back to Consul and will poll every ~60s until it sees a leader.
2016-06-16 12:14:03 -07:00
b0fecbefc1
Define `BootstrapExepct` as an `int32` so it can be manipulated atomically.
2016-06-16 12:00:15 -07:00
ea5d11e628
remove consul reference
2016-06-15 17:23:02 -07:00
bf14fd355f
plan displays launch time of periodic jobs
2016-06-15 13:34:45 -07:00
14f9d2a947
Use the config's log output
2016-06-15 12:40:51 -07:00
5b0def194a
Namespace the log messages
2016-06-15 12:40:51 -07:00
bffc82d668
Do not consider the number of Serf members when considering falling back to Consul.
2016-06-15 12:40:51 -07:00
324af8d7f1
Guard the auto-join functionality behind its `consul.server_auto_join` tunable
2016-06-15 12:40:51 -07:00
5e0ced2ae7
Shuffle all datacenters vs only the nearest N datacenters.
...
Per discussion, we want to be aggressive about fanning out vs possibly
fixating on only local DCs. With RPC forwarding in place, a random walk
may be less optimal from a network latency perspective, but it is guaranteed
to eventually result in a converged state because all DCs are candidates
during the bootstrapping process.
2016-06-15 12:40:51 -07:00
2123460cf0
Bump various Consul search limits
...
Client: Search limit increased from 4 random DCs to 8 random DCs, plus nearest.
Server: Search factor increased from 3 to 5 times the bootstrap_expect.
This should allow for faster convergence in large environments (e.g.
sub-5min for 10K Consul DCs).
2016-06-15 12:40:51 -07:00
e8d1264dbc
Short-circuit the bootstrapFn if we have a leader
2016-06-15 12:40:51 -07:00
f05514335b
Teach Nomad servers how to fall back to Consul.
2016-06-15 12:40:51 -07:00
aea21affdb
Document consul configuration
2016-06-14 15:21:57 -07:00
6e22b680ce
Disambiguate `auto_join` from `auto_register`, rename reg to `auto_advertise`.
...
Provide an option that describes the value to the user vs the
operation performed by the software. Momentarily introducing
`auto_join`
2016-06-14 12:11:38 -07:00
4f14d51013
Fix up validation and allow existing unset timeouts to continue to be unset
2016-06-13 18:55:15 -07:00
c3a3fdc230
Upon further review, the Timeout needs to be validate for more than script checks.
...
This value is used for Consul HTTP and TCP checks.
2016-06-13 18:28:27 -07:00
baac19cad6
Remove diff check for ServiceID, may it R.I.P.
2016-06-13 18:22:53 -07:00
79c675cf72
Guard against an interval and timeout being less than 1s
2016-06-13 18:19:40 -07:00
af8db7ec18
Don't export ServiceCheck validate
2016-06-13 18:17:43 -07:00
08c88102a7
There is no "docker" check type
2016-06-13 18:15:07 -07:00
8bbf4a55e5
Fix IDs and domain scoping
2016-06-13 16:30:58 -07:00
8e231fa382
Rename ConsulService back to Service
2016-06-12 16:36:49 -07:00
3024c080e8
Removing artifact check for java and qemu drivers
2016-06-12 12:57:35 +02:00
480a281031
Merge pull request #1243 from hashicorp/f-run-modify-index
...
Add check-index flag to nomad run
2016-06-11 16:12:53 -07:00
2f036231e5
Merge pull request #1201 from hashicorp/f-dyn-server-list
...
Dynamic Server Lists/Client Bootstrapping via consul.
2016-06-11 18:58:25 -04:00
59b0a7b3f6
Merge pull request #1256 from hashicorp/b-node-gc
...
Improve partial garbage collection of allocations
2016-06-11 15:41:00 -07:00
bbd8dfa798
goling(1) compliance pass (e.g. Rpc* -> RPC)
2016-06-10 23:38:28 -04:00
98bf249625
Partial GC allocations
2016-06-10 18:32:37 -07:00
7ccc7d20a0
test
2016-06-10 15:48:59 -07:00
b064b392fc
Only unblock if missed class was added after eval snapshot index
2016-06-10 15:24:06 -07:00
948663c89a
Fix another unit test not expecting ServiceID
2016-06-10 16:50:35 -04:00
d99467ef5e
Always create a consul.Syncer. Use a default Consul Config if necessary.
2016-06-10 15:55:27 -04:00
3d64daafd9
Fold RaftPeers() into its only call site now
2016-06-10 15:54:39 -04:00
0ba1da9c9c
Always pass in a snapshot before calling constructNodeServerInfoResponse()
2016-06-10 15:54:39 -04:00
1df6fc253f
Rename `updateNodeUpdateResponse` to `constructNodeServerInfoResponse`
2016-06-10 15:54:39 -04:00
077203fe93
Update the structure of ConsulService to match reality.
...
ConsulService is the configuration for a Consul Service
2016-06-10 15:54:39 -04:00
197feae679
Sync services with Consul by comparing the AgentServiceReg w/ ConsulService
...
The source of truth is the local Nomad Agent. Any services not local that
have a matching prefix are removed. Changed services are re-registered
and missing services are re-added.
2016-06-10 15:54:39 -04:00
9a223936bb
Generate and sync Consul ServiceIDs consistently
2016-06-10 15:54:39 -04:00
95c9d1a63e
Per-comment, remove structs.Allocation's Services attribute.
...
Nuke PopulateServiceIDs() now that it's also no longer needed.
2016-06-10 15:54:39 -04:00
7956eb0c80
Rename structs.Task's `Service` attribute to `ConsulService`
2016-06-10 15:54:39 -04:00
fda03c5c9e
Change the signature of the PeriodicCallback to return an error
...
I *KNEW* I should have done this when I wrote it, but didn't want to
go back and audit the handlers to include the appropriate return
handling, but now that the code is taking shape, make this change.
2016-06-10 15:54:39 -04:00
4973ec32bb
Rename structs.Services to structs.ConsulServices
2016-06-10 15:54:39 -04:00
060300007e
Use a monotonically incrementing number to create unique node names.
...
Also remove the space from the "name" of the node
2016-06-10 15:50:11 -04:00
1ec7d6c266
Push down the server list even on node registration and evaluation
...
Be mindful of the cost of taking a snapshot from the statestore and
reuse the snapshot if one has already been taken.
2016-06-10 15:50:11 -04:00
bff57a0dce
Reconcile, clean up, and centralize API version numbers (major and minor).
...
Reduce future confusion by introducing a minor version that is gossiped out
via the `mvn` Serf tag (Minor Version Number, `vsn` is already being used for
to communicate `Major Version Number`).
Background: hashicorp/consul/issues/1346#issuecomment-151663152
2016-06-10 15:50:11 -04:00
dde6a4074d
Nuke trace-level logging in heartbeats
2016-06-10 15:50:11 -04:00
d76c042a13
Invert error handling logic
2016-06-10 15:50:11 -04:00
1fe979a5e4
Remove types.ShutdownChannel and replace with `chan struct{}`
2016-06-10 15:50:11 -04:00
438becb28b
Pass the datacenter name in the heartbeat
...
Servers that are part of a different datacenter are added as backup
servers instead of primary servers.
2016-06-10 15:50:11 -04:00
89168b0c51
Invert check definition so the error is first
2016-06-10 15:50:11 -04:00
dc78baedfd
Fix typo in the comment to reflect the actual function name.
2016-06-10 15:50:11 -04:00
410d85cc78
Rename the package from `client/rpc_proxy` to `client/rpcproxy`
...
Also rename `NewRpcProxy()` to just `New()` to avoid package stutter.
2016-06-10 15:50:11 -04:00
1aefdb1e15
Use the correctly typed `rand.Int*` variant
2016-06-10 15:50:11 -04:00
3a1dc9a194
Use `rand.Int*n()` where appropriate
2016-06-10 15:50:11 -04:00
e727fd8c3c
Centralize the creation of a consul/api.Config struct.
...
While documented, the consul.timeout parameter wasn't ever set
except one-off in the Consul fingerprinter.
2016-06-10 15:50:11 -04:00
f695d6d70d
Reconcile consul's address configuration section.
...
There were conflicting directives previously, both consul.addr and
consul.address were required to achieve the desired behavior. The
documentation said `consul.address` was the canonical name for the
parameter, so consolidate configuration parameters to `consul.address`.
2016-06-10 15:50:11 -04:00
e60580b279
Define a type for the PeriodicCallback handlers and ShutdownChannel
2016-06-10 15:50:11 -04:00
17116fc5a7
Rebalance Nomad client RPCs among different Nomad servers.
...
Implement client/rpc_proxy.RpcProxy.
2016-06-10 15:50:11 -04:00
b509da2d0c
Create a `nomad/structs/config` to break an import cycle.
...
Flattening and normalizing the various Consul config structures and
services has led to an import cycle. Break this by creating a new package
that is intended to be terminal in the import DAG.
2016-06-10 15:48:36 -04:00
6d162e1e03
Fix copy pasta comment.
...
These parameters are used to bootstrap Nomad servers, not Consul servers.
2016-06-10 15:48:36 -04:00
4e2835d5ff
Use the correctly typed `rand.Int*` variant
2016-06-10 15:48:36 -04:00
49deaae2ae
Seed random once in main
2016-06-10 15:48:36 -04:00
db97a88f94
Fix small typo
2016-06-10 15:48:36 -04:00
66b4b2a99f
Use `rand.Int*n()` where appropriate
2016-06-10 15:48:36 -04:00
e36686a17d
Use consul/lib's RandomStagger
...
Removes four redundant copies of the method in the process.
2016-06-10 15:48:36 -04:00
e0e7d94450
Use consul/lib's RateScaledInterval
2016-06-10 15:48:36 -04:00
527afa5119
Merge pull request #1244 from hashicorp/b-eval-reblock-test-hardening
...
Don't dequeue requeued evals in tests
2016-06-09 11:35:42 -07:00
5d181d203c
Add check-index flag to nomad run
2016-06-08 17:56:32 -07:00
b7e3a45fef
fix channel being nil on restore
2016-06-07 15:03:08 -07:00
ecdce9a641
don't dequeue
2016-06-07 09:51:20 -07:00
cc95d5d332
GC Nodes even if they have terminal allocations
2016-06-03 16:24:41 -07:00
Diptanu Choudhury