luxolus
/
open-nomad
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity

node pools: add pool as label on client metrics (#17528) 

This changeset adds the node pool as a label anywhere we're already emitting
labels with additional information such as node class or ID about the client.
This commit is contained in:
Tim Gross 2023-06-14 15:58:38 -04:00 committed by GitHub
parent 5f509b8ce0
commit dc9fae34ca
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 31 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
client/client.go
View File

@ -2992,6 +2992,7 @@ func (c *Client) emitStats() {
{Name: "node_id", Value: c.NodeID()},
{Name: "datacenter", Value: c.Datacenter()},
{Name: "node_class", Value: emittedNodeClass},
{Name: "node_pool", Value: c.Node().NodePool},
}
// Start collecting host stats right away and then keep collecting every

57
website/content/docs/operations/metrics-reference.mdx
View File

@ -151,34 +151,35 @@ parameterized or periodic job respectively. For example, a dispatch job with the
Nomad will emit [tagged metrics][tagged-metrics], in the below format:
| Metric | Description | Unit | Type | Labels |
| --------------------------------------- | ----------------------------------------------------------------------------------- | ---------- | ----- | ------------------------------------------------------------------------------------- |
| `nomad.client.allocated.cpu` | Total amount of CPU shares the scheduler has allocated to tasks | Mhz | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocated.memory` | Total amount of memory the scheduler has allocated to tasks | Megabytes | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocated.disk` | Total amount of disk space the scheduler has allocated to tasks | Megabytes | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.blocked` | Number of allocations waiting for previous versions to exit | Integer | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.migrating` | Number of allocations migrating data from previous versions (see [`sticky`][sticky])| Integer | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.pending` | Number of allocations pending (received by the client but not yet running) | Integer | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.running` | Number of allocations running | Integer | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.start` | Number of allocations starting | Integer | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.terminal` | Number of allocations terminal | Integer | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.allocs.oom_killed` | Number of allocations OOM killed | Integer | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.cpu.idle` | CPU utilization in idle state | Percentage | Gauge | cpu, datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.cpu.system` | CPU utilization in system space | Percentage | Gauge | cpu, datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.cpu.total` | Total CPU utilization | Percentage | Gauge | cpu, datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.cpu.user` | CPU utilization in user space | Percentage | Gauge | cpu, datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.available` | Amount of space which is available | Bytes | Gauge | datacenter, disk, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.inodes_percent` | Disk space consumed by the inodes | Percentage | Gauge | datacenter, disk, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.size` | Total size of the device | Bytes | Gauge | datacenter, disk, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.used_percent` | Percentage of disk space used | Percentage | Gauge | datacenter, disk, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.used` | Amount of space which has been used | Bytes | Gauge | datacenter, disk, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.memory.available` | Total amount of memory available to processes which includes free and cached memory | Bytes | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.memory.free` | Amount of memory which is free | Bytes | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.memory.total` | Total amount of physical memory on the node | Bytes | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.host.memory.used` | Amount of memory used by processes | Bytes | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.unallocated.cpu` | Total amount of CPU shares free for the scheduler to allocate to tasks | Mhz | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.unallocated.disk` | Total amount of disk space free for the scheduler to allocate to tasks | Megabytes | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.unallocated.memory` | Total amount of memory free for the scheduler to allocate to tasks | Megabytes | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
| `nomad.client.uptime` | Uptime of the host running the Nomad client | Seconds | Gauge | datacenter, host, node_class, node_id, node_scheduling_eligibility, node_status |
|-----------------------------------------|--------------------------------------------------------------------------------------|------------|-------|--------------------------------------------------------------------------------------------------|
| `nomad.client.allocated.cpu` | Total amount of CPU shares the scheduler has allocated to tasks | Mhz | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocated.memory` | Total amount of memory the scheduler has allocated to tasks | Megabytes | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocated.disk` | Total amount of disk space the scheduler has allocated to tasks | Megabytes | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.blocked` | Number of allocations waiting for previous versions to exit | Integer | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.migrating` | Number of allocations migrating data from previous versions (see [`sticky`][sticky]) | Integer | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.pending` | Number of allocations pending (received by the client but not yet running) | Integer | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.running` | Number of allocations running | Integer | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.start` | Number of allocations starting | Integer | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocations.terminal` | Number of allocations terminal | Integer | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.allocs.oom_killed` | Number of allocations OOM killed | Integer | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.cpu.idle` | CPU utilization in idle state | Percentage | Gauge | cpu, datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.cpu.system` | CPU utilization in system space | Percentage | Gauge | cpu, datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.cpu.total` | Total CPU utilization | Percentage | Gauge | cpu, datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.cpu.user` | CPU utilization in user space | Percentage | Gauge | cpu, datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.available` | Amount of space which is available | Bytes | Gauge | datacenter, disk, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.inodes_percent` | Disk space consumed by the inodes | Percentage | Gauge | datacenter, disk, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.size` | Total size of the device | Bytes | Gauge | datacenter, disk, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.used_percent` | Percentage of disk space used | Percentage | Gauge | datacenter, disk, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.disk.used` | Amount of space which has been used | Bytes | Gauge | datacenter, disk, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.memory.available` | Total amount of memory available to processes which includes free and cached memory | Bytes | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.memory.free` | Amount of memory which is free | Bytes | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.memory.total` | Total amount of physical memory on the node | Bytes | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.host.memory.used` | Amount of memory used by processes | Bytes | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.unallocated.cpu` | Total amount of CPU shares free for the scheduler to allocate to tasks | Mhz | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.unallocated.disk` | Total amount of disk space free for the scheduler to allocate to tasks | Megabytes | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.unallocated.memory` | Total amount of memory free for the scheduler to allocate to tasks | Megabytes | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
| `nomad.client.uptime` | Uptime of the host running the Nomad client | Seconds | Gauge | datacenter, host, node_class, node_id, node_pool, node_scheduling_eligibility, node_status |
## Allocation Metrics