Commit graph

291 commits

Author SHA1 Message Date
Yorick Gersie 95f81f3eeb fix nil pointer in fingerprinting AWS env leading to crash
HTTP Client returns a nil response if an error has occured. We first
  need to check for an error before being able to check the HTTP response
  code.
2019-04-19 11:07:13 +02:00
Lang Martin a2a1e7829d Revert accidental merge of pr #5482
Revert "fingerprint Constraints and Affinities have Equals, as set"
This reverts commit 596f16fb5f1a4a6766a57b3311af806d22382609.

Revert "client tests assert the independent handling of interface and speed"
This reverts commit 7857ac5993a578474d0570819f99b7b6e027de40.

Revert "structs missed applying a style change from the review"
This reverts commit 658916e3274efa438beadc2535f47109d0c2f0f2.

Revert "client, structs comments"
This reverts commit be2838d6baa9d382a5013fa80ea016856f28ade2.

Revert "client fingerprint updateNetworks preserves the network configuration"
This reverts commit fc309cb430e62d8e66267a724f006ae9abe1c63c.

Revert "client_test cleanup comments from review"
This reverts commit bc0bf4efb9114e699bc662f50c8f12319b6b3445.

Revert "client Networks Equals is set equality"
This reverts commit f8d432345b54b1953a4a4c719b9269f845e3e573.

Revert "struct cleanup indentation in RequestedDevice Equals"
This reverts commit f4746411cab328215def6508955b160a53452da3.

Revert "struct Equals checks for identity before value checking"
This reverts commit 0767a4665ed30ab8d9586a59a74db75d51fd9226.

Revert "fix client-test, avoid hardwired platform dependecy on lo0"
This reverts commit e89dbb2ab182b6368507dbcd33c3342223eb0ae7.

Revert "refactor error in client fingerprint to include the offending data"
This reverts commit a7fed726c6e0264d42a58410d840adde780a30f5.

Revert "add client updateNodeResources to merge but preserve manual config"
This reverts commit 84bd433c7e1d030193e054ec23474380ff3b9032.

Revert "refactor struts.RequestedDevice to have its own Equals"
This reverts commit 689782524090e51183474516715aa2f34908b8e6.

Revert "refactor structs.Resource.Networks to have its own Equals"
This reverts commit 49e2e6c77bb3eaa4577772b36c62205061c92fa1.

Revert "refactor structs.Resource.Devices to have its own Equals"
This reverts commit 4ede9226bb971ae42cc203560ed0029897aec2c9.

Revert "add COMPAT(0.10): Remove in 0.10 notes to impl for structs.Resources"
This reverts commit 49fbaace5298d5ccf031eb7ebec93906e1d468b5.

Revert "add structs.Resources Equals"
This reverts commit 8528a2a2a6450e4462a1d02741571b5efcb45f0b.

Revert "test that fingerprint resources are updated, net not clobbered"
This reverts commit 8ee02ddd23bafc87b9fce52b60c6026335bb722d.
2019-04-11 10:29:40 -04:00
Lang Martin a9db848974 refactor error in client fingerprint to include the offending data 2019-04-11 09:56:22 -04:00
Alex Dadgar 4bdccab550 goimports 2019-01-22 15:44:31 -08:00
Danielle Tomlinson 7a5d511349 fingerprinter: Use HCLogger for windows 2019-01-17 18:43:13 +01:00
Danielle Tomlinson 66c521ca17 client: Move fingerprint structs to pkg
This removes a cyclical dependency when importing client/structs from
dependencies of the plugin_loader, specifically, drivers. Due to
client/config also depending on the plugin_loader.

It also better reflects the ownership of fingerprint structs, as they
are fairly internal to the fingerprint manager.
2018-12-01 17:10:39 +01:00
Alex Dadgar 7946a14aa8 Fix lints 2018-10-16 16:56:56 -07:00
Alex Dadgar 89dafaaea9 compile on windows 2018-10-16 16:56:56 -07:00
Michael Schurter 86bd329539 fix build errors post merges 2018-10-16 16:53:31 -07:00
Alex Dadgar 8504505c0d client uses passed logger and fix fingerprinters 2018-10-16 16:53:30 -07:00
Alex Dadgar a78cefec18 use int64 2018-10-16 15:34:32 -07:00
Preetha Appan 7c0d8c646c
Change CPU/Disk/MemoryMB to int everywhere in new resource structs 2018-10-16 16:21:42 -05:00
Alex Dadgar 01f8e5b95f renames 2018-10-04 14:57:25 -07:00
Alex Dadgar 52f9cd7637 fixing tests 2018-10-04 14:26:19 -07:00
Alex Dadgar 3183153315 Node resources on client 2018-09-29 17:23:41 -07:00
James Rasell 75f95ccf09
Merge branch 'master' into f_gh_4381 2018-06-19 17:51:57 +02:00
Alex Dadgar 22757d964e lint 2018-06-13 16:06:39 -07:00
Alex Dadgar 300b1a7a15 Tests only use testlog package logger 2018-06-13 15:40:56 -07:00
James Rasell b9009c419c
Add 'nomad.advertise.address' to client meta via NomadFingerPrint
This change removes the addition of the advertise address to the
exported task env vars and instead moves this work into the
NomadFingerprint.Fingerprint which adds this value to the client
attrs. This can then be used within a Nomad job like
${attr.nomad.advertise.address}.
2018-06-08 09:44:10 +02:00
Alex Dadgar c0386819b3 bump version/lint/generated files 2018-06-01 15:23:10 -07:00
Alex Dadgar de98774f2c Add test and docs 2018-05-31 18:05:03 -07:00
Seth Vargo df4fe7e76c
Set user-agent when talking to GCE metadata 2018-04-10 10:36:46 -04:00
Preetha Appan 329428b49f
Code review feedback and unit test 2018-03-28 10:07:15 -05:00
Mildred Ki'Lya 1017cbe8ab
Allow to specify total memory on agent configuration
Allow to set the total memory of an agent in its configuration file. This
can be used in case the automatic detection doesn't work or in specific
environments when memory overcommit (using swap for example) can be
desirable.
2018-03-27 15:46:18 -05:00
Chelsea Holland Komlo 0bde357731 add concept of health checks to fingerprinters and nodes
fix up feedback from code review

add driver info for all drivers to node
2018-03-21 15:15:25 -04:00
Josh Soref 6222bd564e spelling: verify 2018-03-11 19:13:32 +00:00
Josh Soref 96dba3e267 spelling: mount 2018-03-11 18:27:18 +00:00
Josh Soref 75d1240446 spelling: interface 2018-03-11 18:15:37 +00:00
Josh Soref b67449796a spelling: added 2018-03-11 17:34:28 +00:00
Chelsea Holland Komlo b8e8064835 code review fixup 2018-01-31 18:34:03 -05:00
Chelsea Holland Komlo 7b53474a6e add applicable boolean to fingerprint response
public fields and remove getter functions
2018-01-31 13:21:45 -05:00
Chelsea Holland Komlo 14147c8496 remove attributes from periodic fingerprints when state changes
write test for client periodic fingerprinters
2018-01-29 13:48:54 -05:00
Chelsea Holland Komlo 7c19de797c create safe getters and setters for fingerprint response 2018-01-26 11:22:05 -05:00
Chelsea Holland Komlo 3d38868b88 add test case for available cgroups 2018-01-25 06:08:07 -05:00
Chelsea Holland Komlo 9a8344333b refactor Fingerprint to request/response construct 2018-01-24 11:54:02 -05:00
Charlie Voiselle 7a231897a5
Merge pull request #3556 from angrycub/f-fingerprint-log-level
Dropped loglevel for AWS fingerprinter env read misses to DEBUG
2017-11-16 16:27:25 -05:00
Charlie Voiselle 969ddf9c2a Lowered to DEBUG from AD feedback 2017-11-16 14:13:03 -05:00
Alex Dadgar 97ec3974a9 Use interface attached to default route 2017-11-15 11:32:32 -08:00
Charlie Voiselle 1197637251 Dropped loglevel for AWS fingerprinter env reads
Certain environments use WARN for serious logging; however, it's very
possible to have machines without some of the fingerprinted keys
(public-ipv4 and public-hostname specifcally).  Setting log level to
INFO seems more consistent with this possibility.
2017-11-15 18:20:59 +00:00
Alex Dadgar ee31e15f51 Better interface selection heuristic
This PR introduces a better interface selection heuristic such that we
select interfaces with globally routable unicast addresses over link
local addresses.

Fixes https://github.com/hashicorp/nomad/issues/3487
2017-11-13 15:13:43 -08:00
Michael Schurter 59ff94cd71 Don't panic on unexpeced Consul response
Fixes #3326
2017-10-11 18:25:54 -07:00
Alex Dadgar 4173834231 Enable more linters 2017-09-26 15:26:33 -07:00
Alex Dadgar fd7d614ae4 Handle interfaces that only have link-local addrs
This PR changes the fingerprint handling of network interfaces that only
contain link local addresses. The new behavior is to prefer globally
routable addresses and if none are detected, to fall back to link local
addresses if the operator hasn't disallowed it. This gives us pre 0.6
behavior for interfaces with only link local addresses but 0.6+ behavior
for IPv6 interfaces that will always have a link-local address.

Fixes https://github.com/hashicorp/nomad/issues/3005

/cc diptanuc
2017-08-23 15:32:22 -07:00
Alex Dadgar d6187cd3e8 Fix tests 2017-08-16 16:26:52 -07:00
Alex Dadgar 1a86aecf55 Add version package
This PR adds a version package and consolidates version strings into a
Version struct.
2017-08-16 15:44:21 -07:00
Charlie Voiselle 507c75bd16 Updated AWS fingerprint test for ami-id
In https://github.com/hashicorp/nomad/pull/2999, I changed ami-id
to non-unique.  This updates the test to reflect that.
2017-08-11 12:54:27 -04:00
Charlie Voiselle ae466eaaa7 AMI ID is potentally non-unique
Changed the keys map to reflect that.
2017-08-09 12:53:54 -04:00
Luke Farnell f0ced87b95 fixed all spelling mistakes for goreport 2017-08-07 17:13:05 -04:00
Alex Dadgar a9c786a4fe Make test Vault pick random ports 2017-07-25 17:40:59 -07:00
Alex Dadgar 56f9cf86df Speed up client startup 2017-07-20 22:34:24 -07:00
Michael Schurter b2382f99f2 0 compute == error 2017-07-03 14:51:02 -07:00
Michael Schurter ecf090e980 Fix cpu_total_compute override 2017-07-03 14:51:02 -07:00
Diptanu Choudhury a96018fccc Fixed typo 2017-04-10 11:45:11 -07:00
Diptanu Choudhury 45ad1298e6 Added a test 2017-04-08 13:44:21 -07:00
Diptanu Choudhury 353f193983 Removed redundant code 2017-04-07 18:28:22 -07:00
Diptanu Choudhury 963b9baf14 Ignoring link local addresses 2017-04-07 16:04:36 -07:00
Diptanu Choudhury b9e71f4349 Finding the appropriate cidr block 2017-04-06 16:04:59 -07:00
Diptanu Choudhury 0944d9d640 Making the fingerprinter support ipv6 ips 2017-04-06 01:06:32 -07:00
Alex Dadgar 177bd14718 rename cpu_total_compute and docs 2017-03-14 14:15:49 -07:00
Alex Dadgar a1a7941dec Various fixes
This PR:
* Uses Go 1.8 executable lookup
* Stores any err message from stats init method
* Allows overriding of Cpu Compute for hosts where it can't be detected
2017-03-14 12:56:31 -07:00
Dave Walker (Daviey) 4958154640 Enable kernel.version on Windows
Previously, this value was guarded against running on Windows
because it called the `uname` command which is unlikely to
be there.

This change now sets the value from gopsutil, which might
well be an empty string.

Signed-off-by: Dave Walker (Daviey) <email@daviey.com>
2017-03-01 18:48:23 +00:00
Dave Walker (Daviey) 090e760928 Don't exec uname for node attribute kernel.version
Previously with client fingerprinting, sys/exec's Command
function was being used to execute `uname -r` and the return
string processed into the kernel.version node attribute.

This change uses gopsutil/host KernelVersion function
instead.  This means we can drop the os/exec, strings and
fmt imports... and not execute an external binary.

Signed-off-by: Dave Walker (Daviey) <email@daviey.com>
2017-03-01 09:45:56 +00:00
Diptanu Choudhury eb123416c5 Fixed namespacing for the cpu arch 2017-01-06 14:23:22 -08:00
Alex Dadgar c47ebd508e Remove old TODOs 2016-11-15 16:23:37 -08:00
Alex Dadgar 9497991590 Updated AWS speeds and network_speed now overrides
This PR:

* Makes AWS network speeds more granular
* Makes `network_speed` an override and not a default
* Adds a default of 1000 MBits if no network link speed is detected.

Fixes #1985
2016-11-15 13:55:51 -08:00
Alex Dadgar 92f526d902 Run environmental fingerprinters after host fingerprinters and do an override 2016-11-07 12:21:50 -08:00
Alex Dadgar 41b5679015 Advertise signalling abilities 2016-10-19 15:06:23 -07:00
Alex Dadgar 751aa114bf Fix Vault parsing of booleans 2016-10-10 18:04:39 -07:00
Alex Dadgar 4a8fba5cf7 small fixes 2016-09-01 13:38:31 -07:00
Alex Dadgar 8ca3a16825 Fingerprint 2016-09-01 11:10:14 -07:00
Kenjiro Nakayama 71371fc592 tiny: Fix duplicated error message in CPU fingerprint 2016-08-07 12:49:40 +09:00
Michal Wieczorek b6b3e24541 Link speed for windows network fingerprinting - tests 2016-07-22 22:49:03 +02:00
Michal Wieczorek 679fefc155 Link speed for windows network fingerprinting 2016-07-20 22:13:50 +02:00
Sean Chittenden a9b3f5e552
Alpha-sort the build platforms 2016-07-11 12:23:46 -07:00
Sean Chittenden 267198742f
Merge branch 'master' into f-resource-isolation-fingerprinter 2016-07-11 12:23:09 -07:00
Sean Chittenden 20d87f1782
Remove cgroup fingerprinter from non-linux systems.
If someone wants to extend or reuse Cgroup detenction in the future they
can move `cgroup_linux.go` to `cgroup.go` and add the relevant build
tags.

Requested by: @dadgar
2016-07-11 12:16:56 -07:00
Alex Dadgar f11b1ce079 Get windows to build 2016-07-11 11:52:41 -06:00
Sean Chittenden 9966169596
Merge branch 'f-resource-isolation-cleanup' into f-resource-isolation-fingerprinter 2016-07-11 00:10:21 -07:00
Sean Chittenden f39e84b672
Improve readability: use of a switch vs two if's 2016-07-10 20:18:57 -07:00
Sean Chittenden 2ffbeee06c
Skip the network fingerprinter test when offline.
Conditionalize the network fingerprinter test so that it works when a
user is offline.  Similarly, when the network fingerprint test fails in
the future pass a HINT to the user to set an env var to allow the test
to be skipped in the future.
2016-07-10 20:16:06 -07:00
Sean Chittenden 710173e9cb
Build the Cgroup fingerprinter on only Linux.
Change the logic from `!linux` to an empty build tag so that *if*
another platform picks up Cgroups support they can add themselves to the
necessary build tags for this fingerprinter and be on their way.
Because this technology isn't inherently Linux-specific and isn't
mutually exclusive of other resource isolation containers, resist the
urge to rename the Cgroup fingerprinter to something generic like the
ResourceContainerFingerprinter.
2016-07-10 13:55:06 -07:00
Sean Chittenden bdd7022fdc
Centralize the fingerprintrs.
Add platform specific fingerprinters per platform.

Requested by: @diptanu
2016-07-09 22:31:14 -07:00
Sean Chittenden 1e2e0ca050
Initialize the list of available fingerprinters per platform. 2016-07-09 00:22:42 -07:00
Sean Chittenden 03c571c61b
Consolidate fingerprinters into a single map. 2016-07-08 23:37:14 -07:00
Jake Champlin f094969c7b
Update failing tests 2016-06-23 11:28:17 -04:00
Alex Dadgar 14e950f882 Treat float as int 2016-06-22 15:09:39 -07:00
Alex Dadgar 4ff8edd2da Floor CPU MHz and total compute and mark hostname as unique 2016-06-22 15:01:36 -07:00
Sean Chittenden 21b84fc3e6
Memoize the CPU stats. Error if CPU fingerprinting fails. 2016-06-17 12:13:53 -07:00
Sean Chittenden 686c125fea
Record and use only the first Mhz from the CPU fingerprinter.
Assume all cores are the same speed.
2016-06-17 11:06:57 -07:00
Sean Chittenden 4cc90753f8
In the debug log, split the unit from the measurement
awk(1) friendly is UNIX(tm) friendly.
2016-06-16 23:07:13 -07:00
Sean Chittenden 2dcb591cd8
Warn when we're unable to fingerprint the CPU Mhz 2016-06-16 23:07:13 -07:00
Sean Chittenden b8e63411c0
Explicitly call cpu.Counts() to determine the CPU core count
Much safer than counting the number of InfoStat structs returned.
2016-06-16 23:07:13 -07:00
Sean Chittenden ec77a1869e
Test for errors 2016-06-16 14:43:46 -07:00
Sean Chittenden bcbec34937
Only actively test Consul when env CONSUL_HTTP_ADDR is set 2016-06-10 15:50:11 -04:00
Sean Chittenden 6fdf9135cb
Provide a default ConsulConfig for client/config.DefaultConfig()
Change the unit test to only test if the consul link exists, not the
value of the link.  The old test was hostname specific and therefore
would always be different based on the environment running the tests.
2016-06-10 15:50:11 -04:00
Sean Chittenden bff57a0dce
Reconcile, clean up, and centralize API version numbers (major and minor).
Reduce future confusion by introducing a minor version that is gossiped out
via the `mvn` Serf tag (Minor Version Number, `vsn` is already being used for
to communicate `Major Version Number`).

Background: hashicorp/consul/issues/1346#issuecomment-151663152
2016-06-10 15:50:11 -04:00
Sean Chittenden e727fd8c3c
Centralize the creation of a consul/api.Config struct.
While documented, the consul.timeout parameter wasn't ever set
except one-off in the Consul fingerprinter.
2016-06-10 15:50:11 -04:00
Diptanu Choudhury 15d867d5f1 Updated gopsutil 2016-05-28 19:42:34 -07:00
Sean Chittenden dc28ab0cb5
Speling police 2016-05-15 09:41:34 -07:00
Sean Chittenden 9fd4b82390
Establish a floor of one core for the number of cores.
In most cases the upstream library [shirou/gopsutil](https://github.com/shirou/gopsutil)
needs to be fixed.
2016-05-09 12:22:40 -07:00