luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
24235 commits 1 branch 0 tags 311 MiB
Commit graph

331 commits

Author SHA1 Message Date
Alex Dadgar a78cefec18 use int64 2018-10-16 15:34:32 -07:00
Preetha Appan 7c0d8c646c
Change CPU/Disk/MemoryMB to int everywhere in new resource structs 2018-10-16 16:21:42 -05:00
Alex Dadgar 01f8e5b95f renames 2018-10-04 14:57:25 -07:00
Alex Dadgar 52f9cd7637 fixing tests 2018-10-04 14:26:19 -07:00
Alex Dadgar 3183153315 Node resources on client 2018-09-29 17:23:41 -07:00
James Rasell 75f95ccf09
Merge branch 'master' into f_gh_4381 2018-06-19 17:51:57 +02:00
Alex Dadgar 22757d964e lint 2018-06-13 16:06:39 -07:00
Alex Dadgar 300b1a7a15 Tests only use testlog package logger 2018-06-13 15:40:56 -07:00
James Rasell b9009c419c
Add 'nomad.advertise.address' to client meta via NomadFingerPrint 
This change removes the addition of the advertise address to the
exported task env vars and instead moves this work into the
NomadFingerprint.Fingerprint which adds this value to the client
attrs. This can then be used within a Nomad job like
${attr.nomad.advertise.address}.
 2018-06-08 09:44:10 +02:00
Alex Dadgar c0386819b3 bump version/lint/generated files 2018-06-01 15:23:10 -07:00
Alex Dadgar de98774f2c Add test and docs 2018-05-31 18:05:03 -07:00
Seth Vargo df4fe7e76c
Set user-agent when talking to GCE metadata 2018-04-10 10:36:46 -04:00
Preetha Appan 329428b49f
Code review feedback and unit test 2018-03-28 10:07:15 -05:00
Mildred Ki'Lya 1017cbe8ab
Allow to specify total memory on agent configuration 
Allow to set the total memory of an agent in its configuration file. This
can be used in case the automatic detection doesn't work or in specific
environments when memory overcommit (using swap for example) can be
desirable.
 2018-03-27 15:46:18 -05:00
Chelsea Holland Komlo 0bde357731 add concept of health checks to fingerprinters and nodes 
fix up feedback from code review

add driver info for all drivers to node
 2018-03-21 15:15:25 -04:00
Josh Soref 6222bd564e spelling: verify 2018-03-11 19:13:32 +00:00
Josh Soref 96dba3e267 spelling: mount 2018-03-11 18:27:18 +00:00
Josh Soref 75d1240446 spelling: interface 2018-03-11 18:15:37 +00:00
Josh Soref b67449796a spelling: added 2018-03-11 17:34:28 +00:00
Chelsea Holland Komlo b8e8064835 code review fixup 2018-01-31 18:34:03 -05:00
Chelsea Holland Komlo 7b53474a6e add applicable boolean to fingerprint response 
public fields and remove getter functions
 2018-01-31 13:21:45 -05:00
Chelsea Holland Komlo 14147c8496 remove attributes from periodic fingerprints when state changes 
write test for client periodic fingerprinters
 2018-01-29 13:48:54 -05:00
Chelsea Holland Komlo 7c19de797c create safe getters and setters for fingerprint response 2018-01-26 11:22:05 -05:00
Chelsea Holland Komlo 3d38868b88 add test case for available cgroups 2018-01-25 06:08:07 -05:00
Chelsea Holland Komlo 9a8344333b refactor Fingerprint to request/response construct 2018-01-24 11:54:02 -05:00
Charlie Voiselle 7a231897a5
Merge pull request #3556 from angrycub/f-fingerprint-log-level 
Dropped loglevel for AWS fingerprinter env read misses to DEBUG
 2017-11-16 16:27:25 -05:00
Charlie Voiselle 969ddf9c2a Lowered to DEBUG from AD feedback 2017-11-16 14:13:03 -05:00
Alex Dadgar 97ec3974a9 Use interface attached to default route 2017-11-15 11:32:32 -08:00
Charlie Voiselle 1197637251 Dropped loglevel for AWS fingerprinter env reads 
Certain environments use WARN for serious logging; however, it's very
possible to have machines without some of the fingerprinted keys
(public-ipv4 and public-hostname specifcally).  Setting log level to
INFO seems more consistent with this possibility.
 2017-11-15 18:20:59 +00:00
Alex Dadgar ee31e15f51 Better interface selection heuristic 
This PR introduces a better interface selection heuristic such that we
select interfaces with globally routable unicast addresses over link
local addresses.

Fixes https://github.com/hashicorp/nomad/issues/3487
 2017-11-13 15:13:43 -08:00
Michael Schurter 59ff94cd71 Don't panic on unexpeced Consul response 
Fixes #3326
 2017-10-11 18:25:54 -07:00
Alex Dadgar 4173834231 Enable more linters 2017-09-26 15:26:33 -07:00
Alex Dadgar fd7d614ae4 Handle interfaces that only have link-local addrs 
This PR changes the fingerprint handling of network interfaces that only
contain link local addresses. The new behavior is to prefer globally
routable addresses and if none are detected, to fall back to link local
addresses if the operator hasn't disallowed it. This gives us pre 0.6
behavior for interfaces with only link local addresses but 0.6+ behavior
for IPv6 interfaces that will always have a link-local address.

Fixes https://github.com/hashicorp/nomad/issues/3005

/cc diptanuc
 2017-08-23 15:32:22 -07:00
Alex Dadgar d6187cd3e8 Fix tests 2017-08-16 16:26:52 -07:00
Alex Dadgar 1a86aecf55 Add version package 
This PR adds a version package and consolidates version strings into a
Version struct.
 2017-08-16 15:44:21 -07:00
Charlie Voiselle 507c75bd16 Updated AWS fingerprint test for ami-id 
In https://github.com/hashicorp/nomad/pull/2999, I changed ami-id
to non-unique.  This updates the test to reflect that.
 2017-08-11 12:54:27 -04:00
Charlie Voiselle ae466eaaa7 AMI ID is potentally non-unique 
Changed the keys map to reflect that.
 2017-08-09 12:53:54 -04:00
Luke Farnell f0ced87b95 fixed all spelling mistakes for goreport 2017-08-07 17:13:05 -04:00
Alex Dadgar a9c786a4fe Make test Vault pick random ports 2017-07-25 17:40:59 -07:00
Alex Dadgar 56f9cf86df Speed up client startup 2017-07-20 22:34:24 -07:00
Michael Schurter b2382f99f2 0 compute == error 2017-07-03 14:51:02 -07:00
Michael Schurter ecf090e980 Fix cpu_total_compute override 2017-07-03 14:51:02 -07:00
Diptanu Choudhury a96018fccc Fixed typo 2017-04-10 11:45:11 -07:00
Diptanu Choudhury 45ad1298e6 Added a test 2017-04-08 13:44:21 -07:00
Diptanu Choudhury 353f193983 Removed redundant code 2017-04-07 18:28:22 -07:00
Diptanu Choudhury 963b9baf14 Ignoring link local addresses 2017-04-07 16:04:36 -07:00
Diptanu Choudhury b9e71f4349 Finding the appropriate cidr block 2017-04-06 16:04:59 -07:00
Diptanu Choudhury 0944d9d640 Making the fingerprinter support ipv6 ips 2017-04-06 01:06:32 -07:00
Alex Dadgar 177bd14718 rename cpu_total_compute and docs 2017-03-14 14:15:49 -07:00
Alex Dadgar a1a7941dec Various fixes 
This PR:
* Uses Go 1.8 executable lookup
* Stores any err message from stats init method
* Allows overriding of Cpu Compute for hosts where it can't be detected
 2017-03-14 12:56:31 -07:00
Dave Walker (Daviey) 4958154640 Enable kernel.version on Windows 
Previously, this value was guarded against running on Windows
because it called the `uname` command which is unlikely to
be there.

This change now sets the value from gopsutil, which might
well be an empty string.

Signed-off-by: Dave Walker (Daviey) <email@daviey.com>
 2017-03-01 18:48:23 +00:00
Dave Walker (Daviey) 090e760928 Don't exec uname for node attribute kernel.version 
Previously with client fingerprinting, sys/exec's Command
function was being used to execute `uname -r` and the return
string processed into the kernel.version node attribute.

This change uses gopsutil/host KernelVersion function
instead.  This means we can drop the os/exec, strings and
fmt imports... and not execute an external binary.

Signed-off-by: Dave Walker (Daviey) <email@daviey.com>
 2017-03-01 09:45:56 +00:00
Diptanu Choudhury eb123416c5 Fixed namespacing for the cpu arch 2017-01-06 14:23:22 -08:00
Alex Dadgar c47ebd508e Remove old TODOs 2016-11-15 16:23:37 -08:00
Alex Dadgar 9497991590 Updated AWS speeds and network_speed now overrides 
This PR:

* Makes AWS network speeds more granular
* Makes `network_speed` an override and not a default
* Adds a default of 1000 MBits if no network link speed is detected.

Fixes #1985
 2016-11-15 13:55:51 -08:00
Alex Dadgar 92f526d902 Run environmental fingerprinters after host fingerprinters and do an override 2016-11-07 12:21:50 -08:00
Alex Dadgar 41b5679015 Advertise signalling abilities 2016-10-19 15:06:23 -07:00
Alex Dadgar 751aa114bf Fix Vault parsing of booleans 2016-10-10 18:04:39 -07:00
Alex Dadgar 4a8fba5cf7 small fixes 2016-09-01 13:38:31 -07:00
Alex Dadgar 8ca3a16825 Fingerprint 2016-09-01 11:10:14 -07:00
Kenjiro Nakayama 71371fc592 tiny: Fix duplicated error message in CPU fingerprint 2016-08-07 12:49:40 +09:00
Michal Wieczorek b6b3e24541 Link speed for windows network fingerprinting - tests 2016-07-22 22:49:03 +02:00
Michal Wieczorek 679fefc155 Link speed for windows network fingerprinting 2016-07-20 22:13:50 +02:00
Sean Chittenden a9b3f5e552
Alpha-sort the build platforms 2016-07-11 12:23:46 -07:00
Sean Chittenden 267198742f
Merge branch 'master' into f-resource-isolation-fingerprinter 2016-07-11 12:23:09 -07:00
Sean Chittenden 20d87f1782
Remove cgroup fingerprinter from non-linux systems. 
If someone wants to extend or reuse Cgroup detenction in the future they
can move `cgroup_linux.go` to `cgroup.go` and add the relevant build
tags.

Requested by: @dadgar
 2016-07-11 12:16:56 -07:00
Alex Dadgar f11b1ce079 Get windows to build 2016-07-11 11:52:41 -06:00
Sean Chittenden 9966169596
Merge branch 'f-resource-isolation-cleanup' into f-resource-isolation-fingerprinter 2016-07-11 00:10:21 -07:00
Sean Chittenden f39e84b672
Improve readability: use of a switch vs two if's 2016-07-10 20:18:57 -07:00
Sean Chittenden 2ffbeee06c
Skip the network fingerprinter test when offline. 
Conditionalize the network fingerprinter test so that it works when a
user is offline.  Similarly, when the network fingerprint test fails in
the future pass a HINT to the user to set an env var to allow the test
to be skipped in the future.
 2016-07-10 20:16:06 -07:00
Sean Chittenden 710173e9cb
Build the Cgroup fingerprinter on only Linux. 
Change the logic from `!linux` to an empty build tag so that *if*
another platform picks up Cgroups support they can add themselves to the
necessary build tags for this fingerprinter and be on their way.
Because this technology isn't inherently Linux-specific and isn't
mutually exclusive of other resource isolation containers, resist the
urge to rename the Cgroup fingerprinter to something generic like the
ResourceContainerFingerprinter.
 2016-07-10 13:55:06 -07:00
Sean Chittenden bdd7022fdc
Centralize the fingerprintrs. 
Add platform specific fingerprinters per platform.

Requested by: @diptanu
 2016-07-09 22:31:14 -07:00
Sean Chittenden 1e2e0ca050
Initialize the list of available fingerprinters per platform. 2016-07-09 00:22:42 -07:00
Sean Chittenden 03c571c61b
Consolidate fingerprinters into a single map. 2016-07-08 23:37:14 -07:00
Jake Champlin f094969c7b
Update failing tests 2016-06-23 11:28:17 -04:00
Alex Dadgar 14e950f882 Treat float as int 2016-06-22 15:09:39 -07:00
Alex Dadgar 4ff8edd2da Floor CPU MHz and total compute and mark hostname as unique 2016-06-22 15:01:36 -07:00
Sean Chittenden 21b84fc3e6
Memoize the CPU stats. Error if CPU fingerprinting fails. 2016-06-17 12:13:53 -07:00
Sean Chittenden 686c125fea
Record and use only the first Mhz from the CPU fingerprinter. 
Assume all cores are the same speed.
 2016-06-17 11:06:57 -07:00
Sean Chittenden 4cc90753f8
In the debug log, split the unit from the measurement 
awk(1) friendly is UNIX(tm) friendly.
 2016-06-16 23:07:13 -07:00
Sean Chittenden 2dcb591cd8
Warn when we're unable to fingerprint the CPU Mhz 2016-06-16 23:07:13 -07:00
Sean Chittenden b8e63411c0
Explicitly call cpu.Counts() to determine the CPU core count 
Much safer than counting the number of InfoStat structs returned.
 2016-06-16 23:07:13 -07:00
Sean Chittenden ec77a1869e
Test for errors 2016-06-16 14:43:46 -07:00
Sean Chittenden bcbec34937
Only actively test Consul when env CONSUL_HTTP_ADDR is set 2016-06-10 15:50:11 -04:00
Sean Chittenden 6fdf9135cb
Provide a default ConsulConfig for client/config.DefaultConfig() 
Change the unit test to only test if the consul link exists, not the
value of the link.  The old test was hostname specific and therefore
would always be different based on the environment running the tests.
 2016-06-10 15:50:11 -04:00
Sean Chittenden bff57a0dce
Reconcile, clean up, and centralize API version numbers (major and minor). 
Reduce future confusion by introducing a minor version that is gossiped out
via the `mvn` Serf tag (Minor Version Number, `vsn` is already being used for
to communicate `Major Version Number`).

Background: hashicorp/consul/issues/1346#issuecomment-151663152
 2016-06-10 15:50:11 -04:00
Sean Chittenden e727fd8c3c
Centralize the creation of a consul/api.Config struct. 
While documented, the consul.timeout parameter wasn't ever set
except one-off in the Consul fingerprinter.
 2016-06-10 15:50:11 -04:00
Diptanu Choudhury 15d867d5f1 Updated gopsutil 2016-05-28 19:42:34 -07:00
Sean Chittenden dc28ab0cb5
Speling police 2016-05-15 09:41:34 -07:00
Sean Chittenden 9fd4b82390
Establish a floor of one core for the number of cores. 
In most cases the upstream library [shirou/gopsutil](https://github.com/shirou/gopsutil)
needs to be fixed.
 2016-05-09 12:22:40 -07:00
Sean Chittenden 53d4681b61
Emit various debugging information with the results of the fingerprinter 2016-05-09 12:21:51 -07:00
Sean Chittenden a2b71983c0
cgroups are a decidedly Linux-only feature. Build and test as such. 2016-05-09 11:56:35 -07:00
Sean Chittenden 8e88b11596
Decrease the log-level from WARN to DEBUG when fingerprinting GCE 
This brings the logging here inline with the remaining fingerprinting code
 2016-05-09 11:56:35 -07:00
Sean Chittenden 2ff8b10e05
Rename cgroup_universal.go to cgroup_default.go 
Universal implies it is applicable to Linux, whereas default implies
the fallthrough for the non-Linux build targets.
 2016-05-09 11:56:35 -07:00
Sean Chittenden f8e9a759d3
Move network detection code to Linux-specific build 
Non-Linux build targets get the current default of 0.  This will allow
platform specific fingerprinting for network capabilities.
 2016-05-09 11:56:35 -07:00
Sean Chittenden f2e01f0eab
Stub out FreeBSD support for Nomad 
Compiles, but is not functional (yet).
 2016-05-09 11:56:35 -07:00
Troy Swanson f16468c27a Make logging more consistent 2016-04-15 15:42:57 -05:00
Alex Dadgar aefa4b9207 add nomad fingerprinter 2016-03-22 17:12:30 -07:00
Alex Dadgar 4ce455be31 Merge pull request #712 from iverberk/f-cgroup-finterprinter 
Add periodic cgroup fingerprinter
 2016-01-29 08:50:43 -08:00
Ivo Verberk 22fef1766b Add comments and small improvements to cgroup fingerprinter 2016-01-29 14:34:29 +01:00
Ivo Verberk 2921cf0d92 Fix comment 2016-01-27 10:56:35 +01:00
Ivo Verberk 5f86e28974 Add a periodic cgroup fingerprinter 2016-01-27 07:22:25 +01:00
Alex Dadgar a85800188c Respond to comments 2016-01-26 14:55:38 -08:00
Alex Dadgar d5c77cd4a4 Update client fingerprinters 2016-01-26 10:08:01 -08:00
Abhishek Chanda cd51ee6430 Handle non 200 codes while getting env metadata 2015-12-22 05:23:32 +00:00
Alex Dadgar b943c6e278 Remove all calls to the default logger 2015-12-11 15:02:13 -08:00
Diptanu Choudhury 7bbf3b6497 Renaming constants 2015-12-11 13:47:35 -08:00
Diptanu Choudhury 6b43aa6753 Renaming constants 2015-12-11 13:18:04 -08:00
Diptanu Choudhury 7d6a687b95 Changed the log line in consul fingerprinter 2015-12-11 09:10:21 -08:00
Diptanu Choudhury 49d9d04025 Print consul debug messages only when the consul agent is available 2015-12-11 09:06:03 -08:00
Chris Hines 53ecc03022 Improve error messages. 2015-12-09 16:34:18 -05:00
Chris Hines 6697980331 Use GetDiskFreeSpaceEx instead of parsing output of fsutil on Windows. 2015-11-30 16:43:25 -05:00
Alex Dadgar b2daa5e2e6 Standardize log messages in fingerprinters to DEBUG 2015-11-24 11:06:51 -08:00
Alex Dadgar 6e59527a97 Clear consul attributes if consul couldn't be fingerprinted 2015-11-22 13:51:10 -08:00
Alex Dadgar 77d86d5bb0 Fingerprinter only creates one consul client 2015-11-22 13:42:06 -08:00
Alex Dadgar a352c0fc69 Fix network fingerprint test 2015-11-19 09:22:32 -08:00
Alex Dadgar cf4a44352f Allow no interfaces in network fingerprinter 2015-11-18 19:01:39 -08:00
Guillaume Jacquet 4a3e709eef Fix AWS metadata url 
Fix URL. It was printing an error message on startup:
```
2015/11/13 15:49:21 [ERR] fingerprint.env_aws: Error querying AWS Metadata URL, skipping
```

By the way is it safe to use latest? Is there a chance that Amazon decides to change the format of the metadata? It could be safer to use something like `http://169.254.169.254/2014-11-05/meta-data`
 2015-11-13 11:03:05 -05:00
Alex Dadgar 7c6b80a8fd Merge pull request #400 from nak3/fingerprint-test 
Update fingerprint test codes
 2015-11-10 17:06:16 -08:00
Kenjiro Nakayama 658118ed5f Update fingerprint test codes 2015-11-07 23:16:43 +09:00
Chris Bednarski 2719c4ffd7 Merge branch 'master' into f-windows-networking 2015-11-06 13:32:28 -08:00
Alex Dadgar 5f1b6e70b0 Client handles periodic fingerprinters 2015-11-05 13:46:17 -08:00
Alex Dadgar 5f84878ffb Add consul fingerprinter to builtin list; sort list; fix bad error in consul fingerprinter 2015-11-05 13:46:16 -08:00
Alex Dadgar f9fd83c696 Merge fix 2015-11-05 13:46:02 -08:00
Kenjiro Nakayama 21f537339e Use const value for AWS metadata URL 2015-11-04 00:06:14 +09:00
Chris Bednarski a67a78a2d9 Use the same network fingerprinter for *nix and windows 2015-10-30 16:48:02 -07:00
Diptanu Choudhury e55bbe5f2c Refactored code 2015-10-29 16:16:10 -07:00
Diptanu Choudhury 17d430e46c Renamed BasicNetworkInterfaceDetector to DefaultNetworkInterfaceDetector 2015-10-29 15:55:49 -07:00
Diptanu Choudhury 1360f8c6fa Fixed a debug message 2015-10-29 15:17:40 -07:00
Diptanu Choudhury d31f567484 Fixed style of a debug log 2015-10-29 15:15:44 -07:00
Diptanu Choudhury 08302d43ee Added a test to ensure we are not selecting devices which are not marked as UP or loopback and have no IP addresses 2015-10-29 15:14:13 -07:00
Diptanu Choudhury b048919b63 Added some coments 2015-10-29 11:05:58 -07:00
Diptanu Choudhury 6400273cf8 Added tests for testing detecting default device 2015-10-29 11:01:15 -07:00
Diptanu Choudhury b78c00f255 Introduced an interface to detect network devices so that we can mock it for tests 2015-10-28 15:58:40 -07:00
Diptanu Choudhury f0e445d8f5 Refactored the findInterfaces method to make it more clear 2015-10-28 15:48:08 -07:00
Diptanu Choudhury a3d79ec589 Some coding style changes 2015-10-28 15:03:11 -07:00
Diptanu Choudhury 9b86b2313f We don't want lo and PPP in production 2015-10-28 14:44:46 -07:00
Diptanu Choudhury df5efd1e24 Added more information to log 2015-10-28 14:41:13 -07:00
Diptanu Choudhury 58bf6047a7 Adding more information to errors 2015-10-28 14:32:13 -07:00
Diptanu Choudhury 2464789b73 Added more comments 2015-10-28 14:03:33 -07:00
Diptanu Choudhury 915be5e86c Selecting the ipv4 address if there are ipv4 and ipv6 addresses configured for an interface 2015-10-28 13:38:28 -07:00
Diptanu Choudhury fffb0ed5ae Exctracted a method to detect network interface 2015-10-28 11:11:13 -07:00
Diptanu Choudhury 7a15cf0e45 Fix merge conflicts 2015-10-28 08:54:29 -07:00
Kenjiro Nakayama a79ae3c546 Assign IP when network device is specified 2015-10-28 21:19:06 +09:00
Diptanu Choudhury 809f360743 Using the Go stdlib APIs to detect IP Address of a device 2015-10-27 14:14:25 -07:00
Kenjiro Nakayama e688dc3826 Set default network interface rather than eth0 for non-dev mode, and use loopback device for dev mode by default 2015-10-28 02:03:48 +09:00
Kenjiro Nakayama fa005d75fa Set default network interface rather than eth0 2015-10-25 23:32:54 +09:00
Jeff Mitchell 959c175ca1 Update the location of cleanhttp 2015-10-22 14:21:07 -04:00
Jeff Mitchell cea5fd9081 Use cleanhttp for truly clean clients and transports. 2015-10-22 10:58:23 -04:00
Alex Dadgar 1d15f6eda6 Bind alloc dir and task local dir to docker containers and parse args correctly 2015-10-15 16:40:07 -07:00