Commit Graph

402 Commits

Author SHA1 Message Date
Alex Dadgar 2650bb1d12 Distinct Property supports arbitrary limit
This PR enhances the distinct_property constraint such that a limit can
be specified in the RTarget/value parameter. This allows constraints
such as:

```
constraint {
  distinct_property = "${meta.rack}"
  value = "2"
}
```

This restricts any given rack from running more than 2 allocations from
the task group.

Fixes https://github.com/hashicorp/nomad/issues/1146
2017-07-31 16:52:13 -07:00
Alex Dadgar 4f69355a66 Fix incorrect destructive update with distinct_property constraint
This PR fixes an issue in which an update to a task group with a
distinct property constraint would result in an incorrect destructive
update.
2017-07-31 11:17:35 -07:00
Michael Schurter 5f1f91a46c Use go-testing-interface instead of testing
This drops the testings stdlib pkg from our dependencies. Saves a
whopping 46kb on our binary (was really hoping for more of a win there),
but also avoids potential ugliness with how testing sets flags.
2017-07-25 15:35:19 -07:00
Alex Dadgar 492239d3ee Improve multiple group handling in a deployment
This PR resolves a bug in which a job with multiple task groups would
create new deployment objects each, thus clearing out all other task
groups deployment state.
2017-07-25 11:27:47 -07:00
Alex Dadgar 184bfd4836 Better comment 2017-07-20 12:31:08 -07:00
Alex Dadgar 248315a2d9 Handle destructive changes before placements
This PR updates the generic scheduler to handle destructive changes
before handling placements. This is important because the destructive
change may be due to a lowering of resources. If this is the case, the
handling of the destructive changes first may make it possible for the
placement to happen.

To reason about this imagine there is one node with CPU = 500.

If the group originally had:
* `count = 1`
* `cpu = 400`

And then the job was updated such that the group had:
* `count = 4`
* `cpu = 120`

If the original alloc isn't discounted first, nothing would be able to
place.
2017-07-20 12:24:27 -07:00
Alex Dadgar ce265e0aff Update full node test to test more advanced case 2017-07-20 12:23:40 -07:00
Alex Dadgar a9ec1d6ca7 Fix update limit calculation to avoid panic
This PR fixes the rolling update limit calculation to avoid a panic when
there are more allocations for a deployment that haven't determined
their health than the max_parallel count of the task group.

Fixes https://github.com/hashicorp/nomad/issues/2820
2017-07-19 11:11:47 -07:00
Alex Dadgar 22e84d00ab Fix deep copy of driver config 2017-07-17 17:53:21 -07:00
Alex Dadgar 641e178416 Stop before trying to place 2017-07-17 17:18:12 -07:00
Alex Dadgar 66a90326e1 Treat destructive updates atomically 2017-07-16 10:35:38 -07:00
Alex Dadgar f86760db3c Basic logs 2017-07-07 16:49:08 -07:00
Alex Dadgar 20005f925a Rolling node drains using max_parallel and stagger
This PR adds rolling node drains done at max_parallel and stagger of the
update spec. It brings it inline with old behavior.
2017-07-07 12:12:48 -07:00
Alex Dadgar 3a29b38108 Status description shows requiring promotion 2017-07-07 12:12:48 -07:00
Alex Dadgar 9f016606aa Fix some tests, eval monitor shows deployment id and deployment cancels based on version 2017-07-07 12:12:48 -07:00
Alex Dadgar 9aa1f2fea2 Respond to comments 2017-07-07 12:10:04 -07:00
Alex Dadgar 454083ba1b Remove canary 2017-07-07 12:10:04 -07:00
Alex Dadgar d352d85bb9 Test scheduler's handling of canaries/inplace updates 2017-07-07 12:10:04 -07:00
Alex Dadgar 83c60483f2 Test marking as complete 2017-07-07 12:10:04 -07:00
Alex Dadgar 477c713df5 Plan apply handles canaries and success is set via update 2017-07-07 12:10:04 -07:00
Alex Dadgar 1e8b5e75a5 Fix handling of failed job 2017-07-07 12:10:04 -07:00
Alex Dadgar e229d3650b Attach eval id 2017-07-07 12:10:04 -07:00
Alex Dadgar af1935e1e1 Mark complete 2017-07-07 12:10:04 -07:00
Alex Dadgar 8424a3b380 Change canary handling 2017-07-07 12:10:04 -07:00
Alex Dadgar c10d7ab871 Remove promoted bit from allocation 2017-07-07 12:10:04 -07:00
Alex Dadgar 09dfa2fc10 Rename CreateDeployments and remove cancelling behavior in state_store 2017-07-07 12:10:04 -07:00
Alex Dadgar 067ed86a47 Client watches for allocation health using task state and Consul checks
This PR adds watching of allocation health at the client. The client can
watch for health based on the tasks running on time and also based on
the consul checks passing.
2017-07-07 12:10:04 -07:00
Alex Dadgar e7034691ea deployment status 2017-07-07 12:07:07 -07:00
Alex Dadgar d04877d23c initial impl 2017-07-07 12:03:11 -07:00
Alex Dadgar 27a6e6b6d1 update description of the alloc update factory function 2017-07-07 12:03:11 -07:00
Alex Dadgar ce2319be9b cleanup limit detection 2017-07-07 12:03:11 -07:00
Alex Dadgar b2573b01f9 Fix canary handling 2017-07-07 12:03:11 -07:00
Alex Dadgar 7952240d69 Deployment tests 2017-07-07 12:03:11 -07:00
Alex Dadgar ce55559f12 Non-Canary/Deployment Tests 2017-07-07 12:03:11 -07:00
Alex Dadgar d111dd5c10 Pull out in-place updating into a passed in function; reduce inputs to reconciler 2017-07-07 12:03:11 -07:00
Alex Dadgar c77944ed29 assign names 2017-07-07 12:03:11 -07:00
Alex Dadgar ecacd44888 handle batch filtering 2017-07-07 12:03:11 -07:00
Alex Dadgar 4c123500ee Remove old 2017-07-07 12:03:11 -07:00
Alex Dadgar 270e26c600 Populate desired state per tg 2017-07-07 12:03:11 -07:00
Alex Dadgar 23dcd175ef Show canaries on plan 2017-07-07 12:03:11 -07:00
Alex Dadgar cf5baba808 handle annotations 2017-07-07 12:03:11 -07:00
Alex Dadgar a46f7c3eb8 Todos 2017-07-07 12:03:11 -07:00
Alex Dadgar 00d962b8b5 Some comments and cleanup 2017-07-07 12:03:11 -07:00
Alex Dadgar 994ad285b7 Split reconcile file 2017-07-07 12:03:11 -07:00
Alex Dadgar 07b1c3e5db Only upsert a job if the spec changes and push deployment creation into reconciler 2017-07-07 12:03:11 -07:00
Alex Dadgar 0d42b5d421 initial reconciler 2017-07-07 12:01:17 -07:00
Alex Dadgar b3f4db0930 cancel deployments 2017-07-07 12:01:17 -07:00
Alex Dadgar 8169590d76 Fix tests 2017-05-01 13:54:26 -07:00
Alex Dadgar 5a2449d236 Respond to review comments 2017-04-19 10:54:03 -07:00
Alex Dadgar 3145086a42 non-purge deregisters 2017-04-15 17:08:05 -07:00
Alex Dadgar 2c31d4036b Skip inplace update on terminal batch allocation
This PR skips adding an inplace update to a successfully terminal batch
job to the plan. This avoids extra data in the plan and avoids
triggering updates on all clients that have the terminal allocation.
This is matching behavior of the service scheduler.

/cc @armon for review
2017-03-11 17:19:22 -08:00
Alex Dadgar bb12ff69a6 Fix in-place update 2017-03-09 22:03:10 -08:00
Alex Dadgar 601cbd7784 Feedback addressed 2017-03-09 21:36:27 -08:00
Alex Dadgar b65d248dee Fix filtering issue and add a test that would catch it 2017-03-09 16:20:39 -08:00
Alex Dadgar 7945e4564c Refactor 2017-03-09 15:26:46 -08:00
Alex Dadgar 60c42f745a Split distinct property and host iterator and add iterator to system stack 2017-03-08 19:00:10 -08:00
Alex Dadgar 319b24081f cleanup 2017-03-08 17:57:31 -08:00
Alex Dadgar a439bf709d Property Set 2017-03-08 17:50:40 -08:00
Alex Dadgar d83a8fe9f2 Unoptimized implementation + testing 2017-03-07 14:48:54 -08:00
Alex Dadgar 87d971a6b8 Double the anti-affinity for placing same task group on node 2017-03-06 11:52:53 -08:00
Alex Dadgar 5be806a3df Fix vet script and fix vet problems
This PR fixes our vet script and fixes all the missed vet changes.

It also fixes pointers being printed in `nomad stop <job>` and `nomad
node-status <node>`.
2017-02-27 16:00:19 -08:00
Alex Dadgar 04862ca10e Tests compile 2017-02-07 21:30:57 -08:00
Alex Dadgar b69b357c7f Nomad builds 2017-02-07 20:31:23 -08:00
Alex Dadgar 302a0cf382 Fix adjust test 2017-01-08 14:14:35 -08:00
Alex Dadgar 2c838a80f6 Detect newly created allocation's properly 2017-01-08 13:55:03 -08:00
Alex Dadgar 8d5f0fea69 Merge pull request #2128 from hashicorp/f-dispatch
Nomad Constructor Jobs and Dispatch
2017-01-06 05:22:49 +08:00
Diptanu Choudhury 9cdd576720 Updated changelog and fixed tests 2016-12-20 11:32:17 -08:00
Alex Dadgar a1dd78c24b Scheduler combines meta from job > group > task 2016-12-15 17:08:38 -08:00
Diptanu Choudhury 5191b4d33a Making the status command return the allocs of currently registered job 2016-11-24 16:31:30 +01:00
Alex Dadgar a1d08c2aba Add scheduler version enforcement 2016-10-26 14:52:48 -07:00
Alex Dadgar 989827e402 Add set contains 2016-10-19 13:06:28 -07:00
Alex Dadgar 36cfe6e89e Large refactor of task runner and Vault token rehandling 2016-10-18 11:24:20 -07:00
Ben Barnard 83f647ed84 Replace "the the" with "the" in documentation and comments 2016-10-11 15:31:40 -04:00
Diptanu Choudhury dae7f88118 Not setting a drained node as preferred node (#1740) 2016-09-23 21:15:50 -07:00
Diptanu Choudhury 45afc0b4e1 Added logic to ensure scheduler knows job defn has been updated when ephemeral disks has been updated (#1725) 2016-09-21 14:00:02 -07:00
Alex Dadgar bc500a536c tasks updated 2016-09-21 11:31:09 -07:00
Diptanu Choudhury 36edabb487 Fixed the logic of calculating queued allocation in sys sched (#1724) 2016-09-20 12:05:19 -07:00
Alex Dadgar 683380c25c Merge pull request #1715 from hashicorp/b-dead-system-nodes
Fix bug where dead nodes weren't properly handled by system scheduler
2016-09-19 11:49:44 -07:00
Alex Dadgar 47551e93b4 Fix bug in which dead nodes weren't being properly handled by system scheduler 2016-09-19 11:49:27 -07:00
Diptanu Choudhury 1b3c5e98c8 Renaming LocalDisk to EphemeralDisk (#1710)
Renaming LocalDisk to EphemeralDisk
2016-09-14 15:43:42 -07:00
Diptanu Choudhury d94bb45ad3 Added some more comments 2016-08-31 14:06:31 -07:00
Diptanu Choudhury 52e9946da9 Implemented SetPrefferingNodes in stack 2016-08-30 16:17:50 -07:00
Diptanu Choudhury bfee7b30a3 Introducing shared resources in alloc 2016-08-29 13:49:25 -07:00
Diptanu Choudhury 13497913f9 Ensuring resources are re-calculated properly in fsm 2016-08-26 20:13:11 -07:00
Diptanu Choudhury e79cb67391 Changing implementation of AllocsFit 2016-08-26 17:28:29 -05:00
Diptanu Choudhury 3447658bba Added scheduler tests to ensure disk constraints are honored 2016-08-25 15:31:56 -05:00
Diptanu Choudhury ffaf6c6299 Fixed some tests 2016-08-25 13:56:39 -05:00
Diptanu Choudhury ec73c768f1 Making the scheduler use LocalDisk instead of Resources.DiskMB 2016-08-25 12:27:42 -05:00
Diptanu Choudhury c1a455983d Added the chained alloc for system scheduler 2016-08-16 10:49:45 -07:00
Diptanu Choudhury 1de89776d7 Marking an allocation chained if we are creating this to replace an old one 2016-08-15 17:52:41 -07:00
Alex Dadgar 64f7eff612 Plan on system scheduler doesn't count nodes who don't meet constraints 2016-08-11 15:26:25 -07:00
Diptanu Choudhury 23fcb9f5c9 Ensuring system sched doesn't increment queued count when nodes are filtered 2016-08-10 14:33:13 -07:00
Diptanu Choudhury 13bab5b1ad Added scheduler tests 2016-08-09 14:52:25 -07:00
Diptanu Choudhury ab94c8eed9 Marking allocations which are not terminal and are on down nodes as lost 2016-08-09 13:11:58 -07:00
Alex Dadgar e33bda76bf test sched doesn't mark complete as lost + core_sched tests 2016-08-04 11:24:17 -07:00
Alex Dadgar ac3328e812 Make scheduler mark allocations as lost 2016-08-03 15:57:46 -07:00
Alex Dadgar 3a9f3a31bc KillTimeout can be modified in place 2016-08-01 20:19:12 -07:00
Alex Dadgar e661c09898 fix filter logic 2016-07-28 15:57:56 -07:00
Alex Dadgar ddbd9261c1 Merge pull request #1471 from hashicorp/b-handle-old-batch-allocs
filterCompleteAllocs filters replaced batch allocs
2016-07-28 14:31:19 -07:00
Diptanu Choudhury eb08405467 Updated tests and added logic to system sched 2016-07-28 14:02:50 -07:00