open-nomad/client/fingerprint
Seth Hoenig 87f4b71df0
client/fingerprint: correctly fingerprint E/P cores of Apple Silicon chips (#16672)
* client/fingerprint: correctly fingerprint E/P cores of Apple Silicon chips

This PR adds detection of asymetric core types (Power & Efficiency) (P/E)
when running on M1/M2 Apple Silicon CPUs. This functionality is provided
by shoenig/go-m1cpu which makes use of the Apple IOKit framework to read
undocumented registers containing CPU performance data. Currently working
on getting that functionality merged upstream into gopsutil, but gopsutil
would still not support detecting P vs E cores like this PR does.

Also refactors the CPUFingerprinter code to handle the mixed core
types, now setting power vs efficiency cpu attributes.

For now the scheduler is still unaware of mixed core types - on Apple
platforms tasks cannot reserve cores anyway so it doesn't matter, but
at least now the total CPU shares available will be correct.

Future work should include adding support for detecting P/E cores on
the latest and upcoming Intel chips, where computation of total cpu shares
is currently incorrect. For that, we should also include updating the
scheduler to be core-type aware, so that tasks of resources.cores on Linux
platforms can be assigned the correct number of CPU shares for the core
type(s) they have been assigned.

node attributes before

cpu.arch                  = arm64
cpu.modelname             = Apple M2 Pro
cpu.numcores              = 12
cpu.reservablecores       = 0
cpu.totalcompute          = 1000

node attributes after

cpu.arch                  = arm64
cpu.frequency.efficiency  = 2424
cpu.frequency.power       = 3504
cpu.modelname             = Apple M2 Pro
cpu.numcores.efficiency   = 4
cpu.numcores.power        = 8
cpu.reservablecores       = 0
cpu.totalcompute          = 37728

* fingerprint/cpu: follow up cr items
2023-03-28 08:27:58 -05:00
..
test_fixtures cni: handle multi-path cni_path when fingerprinting plugins (#16163) 2023-02-13 14:55:56 -06:00
arch.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
arch_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
bridge.go client/fingerprint: detect unloaded dynamic bridge kernel module 2020-11-09 13:56:14 -06:00
bridge_default.go client: Add option to enable hairpinMode on Nomad bridge (#15961) 2023-02-02 10:12:15 -05:00
bridge_linux.go client: Add option to enable hairpinMode on Nomad bridge (#15961) 2023-02-02 10:12:15 -05:00
bridge_linux_test.go deps: Update ioutil deprecated library references to os and io respectively in the client package (#16318) 2023-03-08 13:25:10 -06:00
cgroup.go CI: make make check clean on macOS (#14528) 2022-09-09 12:26:34 -04:00
cgroup_default.go client: enable support for cgroups v2 2022-03-23 11:35:27 -05:00
cgroup_linux.go CI: make make check clean on macOS (#14528) 2022-09-09 12:26:34 -04:00
cgroup_test.go client: enable support for cgroups v2 2022-03-23 11:35:27 -05:00
cni.go fingerprint: add fingerprinting for CNI plugins presense and version (#15452) 2022-12-05 14:22:47 -06:00
cni_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
consul.go client: accommodate Consul 1.14.0 gRPC and agent self changes. (#15309) 2022-11-21 09:19:09 -06:00
consul_test.go deps: Update ioutil deprecated library references to os and io respectively in the client package (#16318) 2023-03-08 13:25:10 -06:00
cpu.go client/fingerprint: correctly fingerprint E/P cores of Apple Silicon chips (#16672) 2023-03-28 08:27:58 -05:00
cpu_darwin_test.go client/fingerprint: correctly fingerprint E/P cores of Apple Silicon chips (#16672) 2023-03-28 08:27:58 -05:00
cpu_default.go client/fingerprint: correctly fingerprint E/P cores of Apple Silicon chips (#16672) 2023-03-28 08:27:58 -05:00
cpu_default_test.go client/fingerprint: correctly fingerprint E/P cores of Apple Silicon chips (#16672) 2023-03-28 08:27:58 -05:00
cpu_linux.go client/fingerprint: correctly fingerprint E/P cores of Apple Silicon chips (#16672) 2023-03-28 08:27:58 -05:00
env_aws.go client: Add AWS EC2 instance-life-cycle from metadata to client fingerprint (#12371) 2022-03-25 11:50:52 -04:00
env_aws_cpu.go env/aws: update ec2 cpu info data (#16417) 2023-03-09 14:33:21 -08:00
env_aws_test.go client: Add AWS EC2 instance-life-cycle from metadata to client fingerprint (#12371) 2022-03-25 11:50:52 -04:00
env_azure.go deps: Update ioutil deprecated library references to os and io respectively in the client package (#16318) 2023-03-08 13:25:10 -06:00
env_azure_test.go testing: setting env var incompatible with parallel tests (#14405) 2022-08-30 14:49:03 -04:00
env_digitalocean.go deps: Update ioutil deprecated library references to os and io respectively in the client package (#16318) 2023-03-08 13:25:10 -06:00
env_digitalocean_test.go testing: setting env var incompatible with parallel tests (#14405) 2022-08-30 14:49:03 -04:00
env_gce.go deps: Update ioutil deprecated library references to os and io respectively in the client package (#16318) 2023-03-08 13:25:10 -06:00
env_gce_test.go testing: setting env var incompatible with parallel tests (#14405) 2022-08-30 14:49:03 -04:00
fingerprint.go client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
fingerprint_default.go gofmt all the files 2021-10-01 10:14:28 -04:00
fingerprint_linux.go CNI Implementation (#7518) 2020-06-18 11:05:29 -07:00
fingerprint_test.go fingerprint code refactor 2019-11-26 10:51:28 -05:00
host.go fingerprint kernel architecture name (#13182) 2022-06-02 15:51:00 -04:00
host_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
landlock.go client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
landlock_test.go client: sandbox go-getter subprocess with landlock (#15328) 2022-12-07 16:02:25 -06:00
memory.go deps: bump gopsutil to v3.21.2 2021-03-30 16:02:51 -04:00
memory_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
network.go fix host network reserved port fingerprint (#11728) 2021-12-22 15:29:54 -05:00
network_default.go gofmt all the files 2021-10-01 10:14:28 -04:00
network_linux.go deps: Update ioutil deprecated library references to os and io respectively in the client package (#16318) 2023-03-08 13:25:10 -06:00
network_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
network_windows.go Disable PowerShell profile and simplify fingerprinting link speed on Windows (#11183) 2021-09-22 11:17:47 -04:00
nomad.go client: add service discovery feature enabled attribute. 2022-03-14 12:42:01 +01:00
nomad_test.go Merge branch 'main' into f-1.3-boogie-nights 2022-03-23 09:41:25 +01:00
plugins_cni.go cni: handle multi-path cni_path when fingerprinting plugins (#16163) 2023-02-13 14:55:56 -06:00
plugins_cni_test.go cni: handle multi-path cni_path when fingerprinting plugins (#16163) 2023-02-13 14:55:56 -06:00
signal.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
signal_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
storage.go client: add disk_total_mb and disk_free_mb config options (#15852) 2023-01-24 09:14:22 -05:00
storage_test.go ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
storage_unix.go gofmt all the files 2021-10-01 10:14:28 -04:00
storage_windows.go Revert "lint: ignore generated windows syscall wrappers" 2019-10-22 08:23:44 -04:00
structs.go Add gosimple linter (#9590) 2020-12-09 11:05:18 -08:00
vault.go vault: configure user agent on Nomad vault clients (#15745) 2023-01-10 10:39:45 -06:00
vault_test.go fingerprint: lengthen Vault check after seen (#14693) 2022-09-26 12:14:19 -07:00
zstorage_windows.go Revert "lint: ignore generated windows syscall wrappers" 2019-10-22 08:23:44 -04:00