Commit graph

42 commits

Author SHA1 Message Date
Mahmood Ali 759eade78b missed fixing one invocation 2020-05-01 13:38:46 -04:00
Mahmood Ali b9e3cde865 tests and some clean up 2020-05-01 13:13:30 -04:00
Michael Schurter 4c5a0cae35 core: fix node reservation scoring
The BinPackIter accounted for node reservations twice when scoring nodes
which could bias scores toward nodes with reservations.

Pseudo-code for previous algorithm:
```
	proposed  = reservedResources + sum(allocsResources)
	available = nodeResources - reservedResources
	score     = 1 - (proposed / available)
```

The node's reserved resources are added to the total resources used by
allocations, and then the node's reserved resources are later
substracted from the node's overall resources.

The new algorithm is:
```
	proposed  = sum(allocResources)
	available = nodeResources - reservedResources
	score     = 1 - (proposed / available)
```

The node's reserved resources are no longer added to the total resources
used by allocations.

My guess as to how this bug happened is that the resource utilization
variable (`util`) is calculated and returned by the `AllocsFit` function
which needs to take reserved resources into account as a basic
feasibility check.

To avoid re-calculating alloc resource usage (because there may be a
large number of allocs), we reused `util` in the `ScoreFit` function.
`ScoreFit` properly accounts for reserved resources by subtracting them
from the node's overall resources. However since `util` _also_ took
reserved resources into account the score would be incorrect.

Prior to the fix the added test output:
```
Node: reserved     Score: 1.0000
Node: reserved2    Score: 1.0000
Node: no-reserved  Score: 0.9741
```

The scores being 1.0 for *both* nodes with reserved resources is a good
hint something is wrong as they should receive different scores. Upon
further inspection the double accounting of reserved resources caused
their scores to be >1.0 and clamped.

After the fix the added test outputs:
```
Node: no-reserved  Score: 0.9741
Node: reserved     Score: 0.9480
Node: reserved2    Score: 0.8717
```
2020-04-15 15:13:30 -07:00
Michael Schurter 0374069f82 scheduler: update tests with modern error helper 2019-12-02 20:25:52 -08:00
Preetha Appan e6a496bac0
Code review feedback 2019-07-31 01:04:08 -04:00
Preetha Appan 99eca85206
Scheduler changes to support network at task group level
Also includes unit tests for binpacker and preemption.
The tests verify that network resources specified at the
task group level are properly accounted for
2019-07-31 01:04:08 -04:00
Alex Dadgar 41265d4d61 Change types of weights on spread/affinity 2019-01-30 12:20:38 -08:00
Alex Dadgar 5198ff05c3 convert driver to device for device constraint/attributes 2019-01-23 10:58:45 -08:00
Preetha Appan f20f2ca8e9
Fixes device scheduling unit tests
Also changes the logic for score when there is more than one task
requesting a device. Since inter task affinities are already normalized,
we take the average of the scores across tasks.
2018-11-08 10:31:19 -06:00
Alex Dadgar a7ca737fb6 review comments 2018-11-07 11:31:52 -08:00
Alex Dadgar 6fa893c801 affinities 2018-11-07 10:32:03 -08:00
Alex Dadgar feb83a2be3 assign devices 2018-11-07 10:32:03 -08:00
Alex Dadgar 01f8e5b95f renames 2018-10-04 14:57:25 -07:00
Alex Dadgar 52f9cd7637 fixing tests 2018-10-04 14:26:19 -07:00
Preetha Appan eccf128c5c
Some minor changes from code review 2018-09-04 16:10:11 -05:00
Preetha Appan 70bfd0c0cb
Address some review feedback 2018-09-04 16:10:11 -05:00
Preetha Appan 5eacd6ada4
Implement affinity support in generic scheduler 2018-09-04 16:10:11 -05:00
Preetha Appan 21b7b79d5d
Add helper methods, use require and other code review feedback 2018-01-31 09:56:53 -06:00
Preetha Appan 031c566ada
Reschedule previous allocs and track their reschedule attempts 2018-01-31 09:56:53 -06:00
Michael Schurter a66c53d45a Remove structs import from api
Goes a step further and removes structs import from api's tests as well
by moving GenerateUUID to its own package.
2017-09-29 10:36:08 -07:00
Alex Dadgar 4173834231 Enable more linters 2017-09-26 15:26:33 -07:00
Alex Dadgar 84d06f6abe Sync namespace changes 2017-09-07 17:04:21 -07:00
Alex Dadgar 8169590d76 Fix tests 2017-05-01 13:54:26 -07:00
Diptanu Choudhury 1b3c5e98c8 Renaming LocalDisk to EphemeralDisk (#1710)
Renaming LocalDisk to EphemeralDisk
2016-09-14 15:43:42 -07:00
Diptanu Choudhury ec73c768f1 Making the scheduler use LocalDisk instead of Resources.DiskMB 2016-08-25 12:27:42 -05:00
Diptanu Choudhury 8f0d2a2775 Fixed some more tests 2016-07-25 17:26:38 -07:00
Alex Dadgar 2de9299cab ProposedAllocs dedups in-place updated allocations 2016-03-21 18:09:32 -07:00
Alex Dadgar 92823b71a8 merge 2015-12-16 15:01:15 -08:00
Alex Dadgar 2218a79815 Add garbage collection to jobs 2015-12-16 15:00:45 -08:00
Armon Dadgar 625308661a scheduler: binpacker makes network offers 2015-09-13 14:31:32 -07:00
Armon Dadgar ea0795995d Use a single implementation of GenerateUUID 2015-09-07 15:23:03 -07:00
Armon Dadgar 1a5579384a nomad: cleanup API descrepencies 2015-09-06 20:47:42 -07:00
Armon Dadgar 2ee6947844 scheduler: updating for new APIs 2015-08-25 17:06:06 -07:00
Armon Dadgar 6d60c4c623 scheduler: adding JobAntiAffinityIterator 2015-08-16 10:32:25 -07:00
Armon Dadgar da90c453ce nomad: adding index on EvalID and Status to alloc 2015-08-15 13:27:42 -07:00
Armon Dadgar f5e37c926c scheduler: testing bin pack scoring 2015-08-13 13:08:15 -07:00
Armon Dadgar 022e583a2a scheduler: testing basic bin pack 2015-08-13 12:08:53 -07:00
Armon Dadgar b0c15ed894 scheduler: refactor tests 2015-08-13 12:02:42 -07:00
Armon Dadgar df21ab3d10 scheduler: working on bin pack 2015-08-13 11:54:59 -07:00
Armon Dadgar 861a5e2097 scheduler: Adding State to Context 2015-08-13 11:33:58 -07:00
Armon Dadgar a19e98202d schedueler: adding best fit scoring 2015-08-13 11:28:02 -07:00
Armon Dadgar acef950c5f scheduler: testing more iterators 2015-08-13 10:13:11 -07:00