luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
4465 commits 1 branch 0 tags 311 MiB
Commit graph

1506 commits

Author SHA1 Message Date
Sean Chittenden bcbec34937
Only actively test Consul when env CONSUL_HTTP_ADDR is set 2016-06-10 15:50:11 -04:00
Sean Chittenden 04c697c610
Update godoc for newServer to reflect DNS and IP-based inputs 
Requested by: alex
 2016-06-10 15:50:11 -04:00
Sean Chittenden 27413076fb
Pick the right DefaultConfig from the right package. 
Overly zealous search && replace at work here.
 2016-06-10 15:50:11 -04:00
Sean Chittenden 7167b7a357
Add a quick set of client/rpcproxy.ServerEndpoint equality tests 2016-06-10 15:50:11 -04:00
Sean Chittenden 837b387dcb
Fix building tests that used DefaultConfig() but didn't pickup the package move. 2016-06-10 15:50:11 -04:00
Sean Chittenden 3b5db4e390
Fix the client/rpcproxy unit tests. 2016-06-10 15:50:11 -04:00
Sean Chittenden 39fb0f2469
Change the endpoint for /v1/agent/servers and fix tests. 
When an agent is running a server, the list of servers includes the
Raft peers.  When the agent is running a client (which is always the
case?), include a list of the servers found in the Client's RpcProxy.
Dedupe and provide a unique list back to the caller.
 2016-06-10 15:50:11 -04:00
Sean Chittenden 6fdf9135cb
Provide a default ConsulConfig for client/config.DefaultConfig() 
Change the unit test to only test if the consul link exists, not the
value of the link.  The old test was hostname specific and therefore
would always be different based on the environment running the tests.
 2016-06-10 15:50:11 -04:00
Sean Chittenden cb80e93a6b
Move client.DefaultConfig() to client/config.DefaultConfig() 
Resolves an import cycle in testing and is more appropriate because
the default should reside next to its struct definition.
 2016-06-10 15:50:11 -04:00
Sean Chittenden d87085e040
Fix a comment to be more correct 2016-06-10 15:50:11 -04:00
Sean Chittenden c7e1879c4d
Unused code wasn't as unused as I thought. Restore. 2016-06-10 15:50:11 -04:00
Sean Chittenden bff57a0dce
Reconcile, clean up, and centralize API version numbers (major and minor). 
Reduce future confusion by introducing a minor version that is gossiped out
via the `mvn` Serf tag (Minor Version Number, `vsn` is already being used for
to communicate `Major Version Number`).

Background: hashicorp/consul/issues/1346#issuecomment-151663152
 2016-06-10 15:50:11 -04:00
Sean Chittenden b3fd455b1f
Register the serf service with the Nomad server service. 
This will be unused in this PR.
 2016-06-10 15:50:11 -04:00
Sean Chittenden 82d537fbd9
Update the nomad_server_service default from nomad-server to just nomad. 2016-06-10 15:50:11 -04:00
Sean Chittenden 6d9ec364e5
Change the constants used to match the struct definitions 2016-06-10 15:50:11 -04:00
Sean Chittenden c013957d49
When clearing the backup servers, set the length to zero. 2016-06-10 15:50:11 -04:00
Sean Chittenden 36340c654d
Improve language re: fingerprinting 2016-06-10 15:50:11 -04:00
Sean Chittenden bff82e4890
Remove unused function. 2016-06-10 15:50:11 -04:00
Sean Chittenden 498f21cdec
Clear the backup server list when a Nomad heartbeat arives with servers 
If Nomad is heartbeating during a transition from using backup servers
to Nomad servers, make Nomad the canonical source of servers and flush
the list of servers populated from Consul.
 2016-06-10 15:50:11 -04:00
Sean Chittenden 1fe979a5e4
Remove types.ShutdownChannel and replace with chan struct{} 2016-06-10 15:50:11 -04:00
Sean Chittenden 2395aa481c
Fix unit tests 2016-06-10 15:50:11 -04:00
Sean Chittenden 438becb28b
Pass the datacenter name in the heartbeat 
Servers that are part of a different datacenter are added as backup
servers instead of primary servers.
 2016-06-10 15:50:11 -04:00
Sean Chittenden d914f262f9
Consolidate all consul sync periodic go routines to handlers. 
Only one pump and periodic loop now.
 2016-06-10 15:50:11 -04:00
Sean Chittenden 9fb0104def
Teach Client to reuse an Agent's consulSyncer. 
"There can be only one."
 2016-06-10 15:50:11 -04:00
Sean Chittenden 47891fb559
Register two services each for clients and servers, http and rpc. 
In order to give clients a fighting chance to talk to the right port,
differentiate RPC services from HTTP services by registering two
services with different tags.  This yields
`rpc.nomad-server.service.consul` and
`http.nomad-server.service.consul` which is immensely more useful to
clients attempting to bootstrap their world.
 2016-06-10 15:50:11 -04:00
Sean Chittenden d2f9848348
Bump the cluster test minimums to 10min. 
These ranges aren't too useful with the default 600s rebalance, but
will be useful if that default ever changes in the future.
 2016-06-10 15:50:11 -04:00
Sean Chittenden 861f900225
s/RpcVersion/RPCVersion/g 2016-06-10 15:50:11 -04:00
Sean Chittenden bd0588ffa8
Bump shuffle interval per suggestion from Alex 2016-06-10 15:50:11 -04:00
Sean Chittenden 9f8d6c9f67
Move struct member to reduce diff context 2016-06-10 15:50:11 -04:00
Sean Chittenden 5570aa98e9
Fix package name in comments 2016-06-10 15:50:11 -04:00
Sean Chittenden 410d85cc78
Rename the package from client/rpc_proxy to client/rpcproxy 
Also rename `NewRpcProxy()` to just `New()` to avoid package stutter.
 2016-06-10 15:50:11 -04:00
Sean Chittenden be679c161c
Rename manager_test.go to rpc_proxy_test.go 2016-06-10 15:50:11 -04:00
Sean Chittenden 677f149b16
Rename RpcProxy's internal filename to match 2016-06-10 15:50:11 -04:00
Sean Chittenden 11d53fda00
Remove unused code 2016-06-10 15:50:11 -04:00
Sean Chittenden 10d1b6ba9b
Reduce all forms of ConsulConfig down to a single struct 
nomad/structs/config/consul.go's ConsulConfig is the canonical definition
for all things Consul now.
 2016-06-10 15:50:11 -04:00
Sean Chittenden 0dbd16aa5d
Rebalance Nomad client RPCs among different Nomad servers. 
Implement client/rpc_proxy.RpcProxy.
 2016-06-10 15:50:11 -04:00
Sean Chittenden a13cbe73c3
Rename ConsulConfig to ConsulAgentConfig 2016-06-10 15:50:11 -04:00
Sean Chittenden e727fd8c3c
Centralize the creation of a consul/api.Config struct. 
While documented, the consul.timeout parameter wasn't ever set
except one-off in the Consul fingerprinter.
 2016-06-10 15:50:11 -04:00
Sean Chittenden af96cb23a5
Rename mergedNomadMap to mergedPrimaryMap. Gratuitous, but more correct. 2016-06-10 15:50:11 -04:00
Sean Chittenden 9d5cdf4e68
Correct a mismerge 
Somewhere along the lines when rebasing I mis-merged a patch.
 2016-06-10 15:50:11 -04:00
Sean Chittenden d58632e93f
Rename RpcProxy's internal filename to match 2016-06-10 15:50:11 -04:00
Sean Chittenden 108d2ca981
index on f-dyn-server-list: bd38cb4 Remove unused code 2016-06-10 15:50:11 -04:00
Sean Chittenden cbe77754df
Remove unused code 2016-06-10 15:50:11 -04:00
Sean Chittenden ec7c78f533
Only poll Consul for servers when Nomad heartbeats begin to fail 
When a deadline timer of 2x Server's last requested TTL expires,
begin polling Consul for Nomad Servers.
 2016-06-10 15:50:11 -04:00
Sean Chittenden a4b32c62e2
s/availble/runChecks/g 2016-06-10 15:50:11 -04:00
Sean Chittenden d1442dc317
Reduce all forms of ConsulConfig down to a single struct 
nomad/structs/config/consul.go's ConsulConfig is the canonical definition
for all things Consul now.
 2016-06-10 15:50:11 -04:00
Sean Chittenden e60580b279
Define a type for the PeriodicCallback handlers and ShutdownChannel 2016-06-10 15:50:11 -04:00
Sean Chittenden 17116fc5a7
Rebalance Nomad client RPCs among different Nomad servers. 
Implement client/rpc_proxy.RpcProxy.
 2016-06-10 15:50:11 -04:00
Sean Chittenden 88f3422d7c
Rename NewConsulService to NewSyncer 2016-06-10 15:49:37 -04:00
Sean Chittenden b9789e0cf7
Rename client/consul/sync.PeriodicSync to Run 2016-06-10 15:49:37 -04:00
Sean Chittenden 768aab015d
Rename client/consul/sync.ConsulService to client/consul/sync.Syncer 
Syncer describes the responsibility and actions of the type.
 2016-06-10 15:49:37 -04:00
Sean Chittenden bfa4711df5
Rename ConsulConfig to ConsulAgentConfig 2016-06-10 15:48:36 -04:00
Sean Chittenden 73e173673e
Rename client/config/config's ConsulConfig to ConsulAgentConfig 
A follow up commit to the previous rename.  More to come.
 2016-06-10 15:48:36 -04:00
Sean Chittenden e97652bbfb
Rename consul.ConsulConfig to consul.AgentConfig 
There were two `ConsulConfig` structs running around, one of them
needed to go away.  Rely on the package's path to provide context
for the type of AgentConfig.
 2016-06-10 15:48:36 -04:00
Sean Chittenden e36686a17d
Use consul/lib's RandomStagger 
Removes four redundant copies of the method in the process.
 2016-06-10 15:48:36 -04:00
Alex Dadgar 26c9110e82 fix types 2016-06-10 11:52:15 -07:00
Alex Dadgar a3d41f2312 use time.Second instead of our own const 2016-06-10 11:38:59 -07:00
Alex Dadgar 3cf74e7fd8 Alloc-status only shows measured statistics and fixes to CPU calculations 2016-06-10 10:38:29 -07:00
Diptanu Choudhury 287e0eff22 Fixed merge conflicts 2016-06-10 04:47:34 +01:00
Diptanu Choudhury a69307cff2 Using named variables 2016-06-10 04:45:16 +01:00
Diptanu Choudhury 9056be6d86 Fixed calculation of cpu usage percent in user and sys mode for pids 2016-06-10 04:06:32 +01:00
Alex Dadgar 98d7547014 Annotate what is measured 2016-06-09 19:45:41 -07:00
Alex Dadgar b7e3a45fef fix channel being nil on restore 2016-06-07 15:03:08 -07:00
Diptanu Choudhury 6bf38bb213 Fixed the svc id generation scheme while pruning services 2016-06-07 08:14:11 -07:00
Diptanu Choudhury 4027ff56f0 Returning an error if stats collection hasn't started yet for a new docker container 2016-06-06 18:52:54 -07:00
Diptanu Choudhury c21d606ebb Getting inodes used percent back 2016-06-06 16:10:34 -07:00
Diptanu Choudhury d928d02d85 Removing inode percent since it doesn't work in some cases 2016-06-06 16:10:34 -07:00
Alex Dadgar ba1a92eb8c Handle errors during stats collection 2016-06-03 14:23:18 -07:00
Diptanu Choudhury 667b478f3f Merge pull request #1226 from hashicorp/f-push-stats 
Push Resource Usage stats to remote sinks
 2016-06-02 23:14:59 +02:00
Diptanu Choudhury 35e31c1b81 Enqueing metrics only if they are not nil 2016-06-02 17:14:15 -04:00
Diptanu Choudhury 7efde782fa Sending metrics for tasks as well 2016-06-01 16:42:16 +02:00
Alex Dadgar 4e15611339 fix wait result being nil and some panics in the cli 2016-05-31 23:09:05 +00:00
Diptanu Choudhury 7f172f314f Pushing host stats 2016-05-31 04:05:47 +02:00
Diptanu Choudhury f95b1d00c3 Renamed error message in alloc endpoint 2016-05-28 20:03:52 -07:00
Diptanu Choudhury ae87177eed Added a test to make sure task runner is collecting stats 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 11d1dd113b Added a test for docker 2016-05-28 19:59:20 -07:00
Diptanu Choudhury c0dc6cfbf2 Changing the api of the stats endpoints 2016-05-28 19:59:20 -07:00
Diptanu Choudhury fa9b0dd7e8 Implemented the resource usage ts since a time 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 02ceb6c697 Initializing the ring buffer with no cells 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 77ac2dd624 Initializing the ring buffer with no cells 2016-05-28 19:59:20 -07:00
Diptanu Choudhury a0c279f3b2 comments 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 4c85f02072 Stopping the metrics collector timers using defer and starting to collect host stats right away 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 40e60ce920 Making the call to Stats on a go-routine 2016-05-28 19:59:20 -07:00
Diptanu Choudhury c8f5527158 Added a test to validate we are collecting stats 2016-05-28 19:59:20 -07:00
Diptanu Choudhury b2e8495971 Fixing the alloc runner tests 2016-05-28 19:59:20 -07:00
Diptanu Choudhury a64062d6a6 Fixed the compilation on linux 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 2b37f08d49 creating the host cpu percent calculator lazily 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 2ff43788a0 simplified the stats method in basic executor 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 097bab52a9 Fixed the logic of scanpids 2016-05-28 19:59:20 -07:00
Diptanu Choudhury f93820f40c Added comments 2016-05-28 19:59:20 -07:00
Diptanu Choudhury c2da19bf11 Refactored the api for NewHostStatsCollector 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 0b0d0764e4 Changed signature of Allocation Stats Reporter 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 75c7f84490 Simplified the docker stats collection 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 98016ec066 Incorporated review comments for executor 2016-05-28 19:59:20 -07:00
Diptanu Choudhury c46400597e Making the stats collection interval and number of data points to keep in memory configurable 2016-05-28 19:59:20 -07:00
Diptanu Choudhury b6e5227e7a Renamed monitorUsage method 2016-05-28 19:59:20 -07:00
Diptanu Choudhury d2021e2953 Changed the signature of ResourceUsageTS 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 666b419dba Acquiring locks before iterating allocations and tasks 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 05c221186b Added disk usage to node status 2016-05-28 19:59:20 -07:00
Diptanu Choudhury cf247c1309 Added uptime to node stats 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 73fa700a88 Showing host resource usage stats 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 313c61b9db Added pidstats in task resource usage struct 2016-05-28 19:59:20 -07:00
Diptanu Choudhury cfd3016747 Renamed CpuUsage to CpuStats 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 77357ccf80 Added a test for calculating cpu stats 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 84cd943c48 Stopping stats collection of tasks which has been destroyed 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 193bbc10f8 Fixed the percentage calculation for cgroups 2016-05-28 19:59:20 -07:00
Diptanu Choudhury 7744a41358 Added some docs 2016-05-28 19:59:03 -07:00
Diptanu Choudhury b884253d39 Added locks to RingBuf 2016-05-28 19:59:03 -07:00
Diptanu Choudhury 22c3f6fee0 Implemented nomad cpu percentage calculator 2016-05-28 19:59:03 -07:00
Diptanu Choudhury e37a108b4c Implemented cpu stats 2016-05-28 19:51:22 -07:00
Diptanu Choudhury 1789dbd1ff Fixed implementation of the docker stats 2016-05-28 19:51:22 -07:00
Diptanu Choudhury 9790516a22 Added the PidStats method on the executor 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 63166d0e46 Adding a query param to return time series of stats 2016-05-28 19:42:34 -07:00
Diptanu Choudhury c347b27e6a Implemented stats for raw_exec 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 9a96dddc07 Added some docs to resource stats endpoint 2016-05-28 19:42:34 -07:00
Diptanu Choudhury b9feae89ce Making the conversion to Stats simpler 2016-05-28 19:42:34 -07:00
Diptanu Choudhury f84f93c817 Returning nil if peek is called before any value is enqueued 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 91d2cf319e Added some documentation 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 7569b1af2e Collecting host stats 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 15d867d5f1 Updated gopsutil 2016-05-28 19:42:34 -07:00
Diptanu Choudhury f3d0aecafe Reporting time series of stats 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 8abc594da1 Added a ringbuff datastructure 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 13825f434e Added the nomad stats command 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 2b86bb3396 Added cpu stats 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 0fb0e0237f Added a client API to display resource usage of an allocation 2016-05-28 19:42:34 -07:00
Diptanu Choudhury 6c1b60ff07 Add the Stats api to driverhandle 2016-05-28 19:42:34 -07:00
Daniel Kerwin df7f5175a8 Add support for shm_size to docker driver. Required update of go-dockerclient 
to more recent version (ba4298ecf4cfa216f1cce2befbf653359f62c722).
 2016-05-27 12:30:04 +02:00
Diptanu Choudhury 76509eb230 Fixed docker test 2016-05-22 02:49:33 -07:00
Sean Chittenden dc28ab0cb5
Speling police 2016-05-15 09:41:34 -07:00
Diptanu Choudhury 2e2e2e500e Using a helper method to create service identifiers 2016-05-14 00:43:25 -07:00
Diptanu Choudhury ecd995b283 Added some docs 2016-05-14 00:36:26 -07:00
Diptanu Choudhury 2a99a2cfb6 Removing addition of the client service while reconciling services 2016-05-13 10:34:21 -07:00
Diptanu Choudhury df9d256132 Making Nomad register services with consul in dev mode 2016-05-13 10:19:49 -07:00
Diptanu Choudhury 439d7bf326 Fixed an agent test 2016-05-11 17:26:53 -07:00
Diptanu Choudhury 5dc507dbe0 Fixing the client test 2016-05-11 16:58:14 -07:00
Diptanu Choudhury 347cb890d2 Removed allocID and task name from consul service 2016-05-11 16:26:41 -07:00
Diptanu Choudhury 7287376eca Using consul config from client config instead of reading from client options 2016-05-11 16:10:57 -07:00
Diptanu Choudhury 83fed62a0a Implemented registering client and server services 2016-05-11 16:07:02 -07:00
Diptanu Choudhury cec6c03b79 Fixed the consul tests 2016-05-11 15:22:58 -07:00
Diptanu Choudhury 2f8a3532ad Refactored the signature of NewConsulService 2016-05-11 15:22:58 -07:00
Sean Chittenden 514f22c4a8 Merge pull request #1160 from hashicorp/f-freebsd 
Stubbed out raw_exec support for *NIX platforms

OOB LGTM: @diptanu
 2016-05-11 12:54:27 -07:00
Diptanu Choudhury 2f15842f3e Merge pull request #1164 from hashicorp/fix-unmount 
Removing directories only if we could successfully unmount them
 2016-05-11 12:51:34 -07:00
Diptanu Choudhury 5b12aebd31 Removing directories only if we could successfully unmount them 2016-05-10 14:57:16 -07:00
Sean Chittenden 9fd4b82390
Establish a floor of one core for the number of cores. 
In most cases the upstream library [shirou/gopsutil](https://github.com/shirou/gopsutil)
needs to be fixed.
 2016-05-09 12:22:40 -07:00
Sean Chittenden 53d4681b61
Emit various debugging information with the results of the fingerprinter 2016-05-09 12:21:51 -07:00
Sean Chittenden a2b71983c0
cgroups are a decidedly Linux-only feature. Build and test as such. 2016-05-09 11:56:35 -07:00
Sean Chittenden 8e88b11596
Decrease the log-level from WARN to DEBUG when fingerprinting GCE 
This brings the logging here inline with the remaining fingerprinting code
 2016-05-09 11:56:35 -07:00
Sean Chittenden 2ff8b10e05
Rename cgroup_universal.go to cgroup_default.go 
Universal implies it is applicable to Linux, whereas default implies
the fallthrough for the non-Linux build targets.
 2016-05-09 11:56:35 -07:00
Sean Chittenden f8e9a759d3
Move network detection code to Linux-specific build 
Non-Linux build targets get the current default of 0.  This will allow
platform specific fingerprinting for network capabilities.
 2016-05-09 11:56:35 -07:00
Sean Chittenden f2e01f0eab
Stub out FreeBSD support for Nomad 
Compiles, but is not functional (yet).
 2016-05-09 11:56:35 -07:00