Commit graph

20940 commits

Author SHA1 Message Date
Mahmood Ali bf1c0dcf17 driver/exec: set soft memory limit
Linux offers soft memory limit:
https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/memory.html#soft-limits
, and
https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v2.html?highlight=memory.low
.

We can set soft memory limits through libcontainer
`Resources.MemoryReservation`: https://pkg.go.dev/github.com/opencontainers/runc@v0.1.1/libcontainer/configs#Resources
2021-03-30 16:55:58 -04:00
Mahmood Ali 85502c1739 oversubscription: e2e tests! 2021-03-30 16:55:58 -04:00
Mahmood Ali f44a04454d oversubscription: driver/exec to honor MemoryMaxMB 2021-03-30 16:55:58 -04:00
Mahmood Ali 275feb5bec oversubscription: docker to honor MemoryMaxMB values 2021-03-30 16:55:58 -04:00
Mahmood Ali 95d85b9cac oversubscription: set the linux memory limit
Use the MemoryMaxMB as the LinuxResources limit. This is intended to ease
drivers implementation and adoption of the features: drivers that use
`resources.LinuxResources.MemoryLimitBytes` don't need to be updated.

Drivers that use NomadResources will need to updated to track the new
field value. Given that tasks aren't guaranteed to use up the excess
memory limit, this is a reasonable compromise.
2021-03-30 16:55:58 -04:00
Mahmood Ali 04a96016e4 oversubscription: add memory_max to hclv1
Allow specifying the `memory_max` field in HCL under the resources block.

Though HCLv1 is deprecated, I've updated them to ease our testing.
2021-03-30 16:55:58 -04:00
Mahmood Ali 18b581656d oversubscription: adds CLI and API support
This commit updates the API to pass the MemoryMaxMB field, and the CLI to show
the max set for the task.

Also, start parsing the MemoryMaxMB in hcl2, as it's set by tags.

A sample CLI output; note the additional `Max: ` for "task":

```
$ nomad alloc status 96fbeb0b
ID                  = 96fbeb0b-a0b3-aa95-62bf-b8a39492fd5c
[...]

Task "cgroup-fetcher" is "running"
Task Resources
CPU        Memory         Disk     Addresses
0/500 MHz  32 MiB/20 MiB  300 MiB

Task Events:
[...]

Task "task" is "running"
Task Resources
CPU        Memory          Disk     Addresses
0/500 MHz  176 KiB/20 MiB  300 MiB
           Max: 30 MiB

Task Events:
[...]
```
2021-03-30 16:55:58 -04:00
Mahmood Ali 0c2551270a oversubscription: Add MemoryMaxMB to internal structs
Start tracking a new MemoryMaxMB field that represents the maximum memory a task
may use in the client. This allows tasks to specify a memory reservation (to be
used by scheduler when placing the task) but use excess memory used on the
client if the client has any.

This commit adds the server tracking for the value, and ensures that allocations
AllocatedResource fields include the value.
2021-03-30 16:55:58 -04:00
Tim Gross f820021f9e deps: bump gopsutil to v3.21.2 2021-03-30 16:02:51 -04:00
Tim Gross 4d30e1ce1a changelog: add section for backports from 1.1.0 to 1.0.5
* Fixed order
* Added missing changlog entry for GH-10145
* Fixed misclassified entry for GH-10158
2021-03-30 15:53:17 -04:00
Bryce Kalow bf70a670ae
fix(circle): change website branch filter to main (#10261) 2021-03-30 10:29:25 -05:00
Seth Hoenig 03ed2a8035
Merge pull request #10243 from apollo13/issue10239
Automatically populate `CONSUL_HTTP_ADDR` for connect native tasks in host networking mode.
2021-03-30 09:00:17 -05:00
Nick Ethier daecfa61e6
Merge pull request #10203 from hashicorp/f-cpu-cores
Reserved Cores [1/4]: Structs and scheduler implementation
2021-03-29 14:05:54 -04:00
Zachary Shilton 34936d4262
website: use global featured-slider component (#10254)
* website: use global featured-slider component

* website: delete unused local featured-slider
2021-03-29 11:37:48 -04:00
Bryce Kalow 0bf390116e
website: add should-build script (#10194) 2021-03-29 10:06:20 -05:00
Dave May ba4da7efca
debug: Remove extra linefeed in monitor.log (#10252) 2021-03-29 09:22:27 -04:00
Florian Apolloner b9b71e7ac5 Automatically populate CONSUL_HTTP_ADDR for connect native tasks in host networking mode. Fixes #10239 2021-03-28 14:34:31 +02:00
Chris Baker cda1cb039e
Merge pull request #10125 from hashicorp/docs-autoscaler-409-plugins
autoscaler plugin docs
2021-03-28 07:07:24 -05:00
Luiz Aoqui f728c9303b
docs: minor updates on autoscaling plugin guides 2021-03-26 16:55:24 -04:00
Tim Gross b0d2eed932 redirect from HTTP root to UI should include query params
The OTT feature relies on having a query parameter for a one-time token which
gets handled by the UI. We need to make sure that query param is preserved
when redirecting from the root URL to the `/ui/` URI.
2021-03-26 14:54:41 -04:00
Tim Gross e6a7c71694
docs: note that Connect requires a hard-coded port
Co-authored-by: Kris Hicks <khicks@hashicorp.com>
2021-03-26 14:43:29 -04:00
Shishir 205e579d46
Update containerd task driver docs. (#10244) 2021-03-26 14:42:27 -04:00
Drew Bailey 7e78d4a607
e2e license smoke test (#10242) 2021-03-26 13:21:47 -04:00
Buck Doyle 07831ab455
Update to v4 of Ember Power Select (#10226)
This closes #10146.

Because of cibernox/ember-power-select#1203, which documents
the current impossibility of attaching test selectors to a
PowerSelect invocation, this uses test selectors on parent
containers instead, occasionally adding wrappers when needed.
I chose to leave the existing test selectors in the hopes that
we can return to using them eventually, but I could easily
remove them if it seems like extra noise now.

Presumably for the same reason, @class no longer works, so
this adjusts the scoping of global search CSS to preserve the style
of the search control.

I also included an update to the latest version of
ember-test-selectors, since we were far behind and I tried
that before finding the aforelinked issue.

Finally, this replaces ember-cli-uglify with ember-cli-terser to address
production build failures as described at ember-cli/ember-cli#9290.
2021-03-26 08:55:12 -05:00
Mahmood Ali dbc3850358
Merge pull request #10145 from hashicorp/b-periodic-init-status
periodic: always reset periodic children status
2021-03-26 09:19:08 -04:00
James Rasell 7c4238c823
Merge pull request #10230 from hashicorp/f-add-allocation-terminal-status-funcs-api
api: add Allocation client and server terminal status funcs.
2021-03-26 08:03:37 +01:00
Mahmood Ali 5d75705edd dispatched parameterized job should clear status too 2021-03-25 15:14:21 -04:00
Drew Bailey 64084f3209
e2e allow setting an enterprise license environment variable (#10233)
* allow setting an enterprise license environment variable

* update comment

* address pr comments
2021-03-25 14:35:55 -04:00
Taylor Dolezal 0714e8951c Change enable to enabled 2021-03-25 14:10:54 -04:00
Michael Lange 0eeaec60fc
Merge pull request #10208 from hashicorp/f-ui/use-new-chart-stuff
UI: Showed host reservations and allocation utilization by task on existing stats charts
2021-03-25 09:18:42 -07:00
Mahmood Ali e643742a38 Add a test for parameterized summary counts 2021-03-25 11:27:09 -04:00
Mahmood Ali b0e048bfa4 periodic: always reset periodic children status
Fixes a bug where Nomad reports negative or incorrect running children
counts for periodic jobs.

The periodic dispatcher derives a child job without reseting the status.
If the periodic job has a `running` status, the derived job will start
as `running` status and transition to `pending`.  Since this is
unexpected transition, the counting in StateStore.setJobSummary gets out of sync and
result in negative/incorrect values.

Note that this only affects periodic jobs after a leader transition.
During the first job registration, the job is added with `pending` or
`""` status. However, after a leader transition, the new leader
repopulates the dispatcher heap with `"running"` status and triggers the
bug.
2021-03-25 11:27:09 -04:00
James Rasell 8dc2a9c6e1
api: add Allocation client and server terminal status funcs. 2021-03-25 08:52:59 +01:00
Michael Lange f1f124340b Remove errant log line 2021-03-24 15:34:46 -07:00
Michael Lange 4f51a55293 Correctly sort tasks in alloc stat tracker so the main task takes precedence 2021-03-24 15:10:46 -07:00
Michael Lange c9590ac715 Guard against empty data when determining active linechart data 2021-03-24 15:09:47 -07:00
Michael Lange 7b60146045 Fix line-wrapping issue with chart tooltips 2021-03-24 14:13:30 -07:00
Michael Lange d787dda298 Use more explicit branching of reserved amounts based on metric type 2021-03-24 13:55:18 -07:00
Michael Lange ab9c824c42 Remove old primary metric 2021-03-24 13:52:52 -07:00
Michael Lange 38e7804afc Update topology info box primary metric 2021-03-24 13:52:52 -07:00
Michael Lange 7ea7a1699d Updated old tests 2021-03-24 13:52:52 -07:00
Michael Lange d77c362c09 Tests for PrimaryMetric::Task 2021-03-24 13:52:52 -07:00
Michael Lange ae22c09398 Tests for PrimaryMetric::Allocation 2021-03-24 13:52:52 -07:00
Michael Lange ac4f173389 Tests for PrimaryMetric::Node 2021-03-24 13:52:52 -07:00
Michael Lange 464b51ac03 Remove now unused chartClass arg from LineChart 2021-03-24 13:52:52 -07:00
Michael Lange 747219feb7 Improved language around what the client stat charts represent 2021-03-24 13:52:52 -07:00
Michael Lange 3f46f77f92 New PrimaryMetric::Task component 2021-03-24 13:52:51 -07:00
Michael Lange 0b4a1e510f Apply the new multi-line primary metric to the alloc index page 2021-03-24 13:52:51 -07:00
Michael Lange 0016290871 Memoize the tracker and the series properties to avoid repeat computation
In addition to this computation being wasteful, it introduces a bug
where the allocation on a stats tracker can update twice in one render,
which isn't allowed in Glimmer (ironically, Glimmmer's lack of
auto-memoization introduced the issue).
2021-03-24 13:52:51 -07:00
Michael Lange 6a8d3340b0 Refactored multi-series allocation variant of the primary metric component 2021-03-24 13:52:51 -07:00