open-nomad/client/fingerprint
Seth Hoenig f8596a3602 env_aws: use best-effort lookup table for CPU performance in EC2
Fixes #7681

The current behavior of the CPU fingerprinter in AWS is that it
reads the **current** speed from `/proc/cpuinfo` (`CPU MHz` field).

This is because the max CPU frequency is not available by reading
anything on the EC2 instance itself. Normally on Linux one would
look at e.g. `sys/devices/system/cpu/cpuN/cpufreq/cpuinfo_max_freq`
or perhaps parse the values from the `CPU max MHz` field in
`/proc/cpuinfo`, but those values are not available.

Furthermore, no metadata about the CPU is made available in the
EC2 metadata service.
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-categories.html

Since `go-psutil` cannot determine the max CPU speed it defaults to
the current CPU speed, which could be basically any number between
0 and the true max. This is particularly bad on large, powerful
reserved instances which often idle at ~800 MHz while Nomad does
its fingerprinting (typically IO bound), which Nomad then uses as
the max, which results in severe loss of available resources.

Since the CPU specification is unavailable programmatically (at least
not without sudo) use a best-effort lookup table. This table was
generated by going through every instance type in AWS documentation
and copy-pasting the numbers.
https://aws.amazon.com/ec2/instance-types/

This approach obviously is not ideal as future instance types will
need to be added as they are introduced to AWS. However, using the
table should only be an improvement over the status quo since right
now Nomad miscalculates available CPU resources on all instance types.
2020-04-28 19:01:33 -06:00
..
arch.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
arch_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
cgroup.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
cgroup_default.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
cgroup_linux.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
cgroup_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
consul.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
consul_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
cpu.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
cpu_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
env_aws.go env_aws: use best-effort lookup table for CPU performance in EC2 2020-04-28 19:01:33 -06:00
env_aws_test.go env_aws: use best-effort lookup table for CPU performance in EC2 2020-04-28 19:01:33 -06:00
env_gce.go goimports 2019-01-22 15:44:31 -08:00
env_gce_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
fingerprint.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
fingerprint_default.go Alpha-sort the build platforms 2016-07-11 12:23:46 -07:00
fingerprint_linux.go Centralize the fingerprintrs. 2016-07-09 22:31:14 -07:00
fingerprint_test.go fingerprint code refactor 2019-11-26 10:51:28 -05:00
host.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
host_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
memory.go fingerprint: Add backwards compatibility comment 2019-09-04 17:38:35 +02:00
memory_test.go fingerprint: Restore support for legacy memory fingerprint 2019-09-04 17:10:28 +02:00
network.go client fingerprinter doesn't overwrite manual configuration 2019-04-19 15:23:48 -04:00
network_default.go Link speed for windows network fingerprinting 2016-07-20 22:13:50 +02:00
network_linux.go fix build errors post merges 2018-10-16 16:53:31 -07:00
network_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
network_windows.go compile on windows 2018-10-16 16:56:56 -07:00
network_windows_test.go fingerprinter: Use HCLogger for windows 2019-01-17 18:43:13 +01:00
nomad.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
nomad_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
signal.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
signal_test.go client uses passed logger and fix fingerprinters 2018-10-16 16:53:30 -07:00
storage.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
storage_test.go client uses passed logger and fix fingerprinters 2018-10-16 16:53:30 -07:00
storage_unix.go Stub out FreeBSD support for Nomad 2016-05-09 11:56:35 -07:00
storage_windows.go Revert "lint: ignore generated windows syscall wrappers" 2019-10-22 08:23:44 -04:00
structs.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
vault.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
vault_test.go client: Move fingerprint structs to pkg 2018-12-01 17:10:39 +01:00
zstorage_windows.go Revert "lint: ignore generated windows syscall wrappers" 2019-10-22 08:23:44 -04:00