52b7fb5361
Separate client and server rpc advertise addresses
2018-03-16 16:47:08 -07:00
1044bc0feb
Merge pull request #3984 from hashicorp/f-loosen-consul-skipverify
...
Replace Consul TLSSkipVerify handling
2018-03-16 11:21:28 -07:00
0971114f0c
Replace Consul TLSSkipVerify handling
...
Instead of checking Consul's version on startup to see if it supports
TLSSkipVerify, assume that it does and only log in the job service
handler if we discover Consul does not support TLSSkipVerify.
The old code would break TLSSkipVerify support if Nomad started before
Consul (such as on system boot) as TLSSkipVerify would default to false
if Consul wasn't running. Since TLSSkipVerify has been supported since
Consul 0.7.2, it's safe to relax our handling.
2018-03-14 17:43:06 -07:00
92cb552ff6
Always add core scheduler and detect invalid schedulers
2018-03-14 10:53:27 -07:00
2ccf565bf6
Refactor redundancy_zone/upgrade_version out of client meta
2018-01-29 20:03:38 -08:00
d09cc2a69f
Merge pull request #3492 from hashicorp/f-client-tls-reload
...
Client/Server TLS dynamic reload
2018-01-23 05:51:32 -05:00
6c9f9c8ac3
adding additional test assertions; differentiate reloading agent and http server
2018-01-16 07:34:39 -05:00
0708d34135
call reload on agent, client, and server separately
2018-01-08 09:56:31 -05:00
6a2432659a
code review fixups
2018-01-08 09:21:06 -05:00
4e0dbd23cf
prevent races when reloading, fully shut down raft
2018-01-08 09:21:06 -05:00
c0ad9a4627
add ability to upgrade/downgrade nomad agents tls configurations via sighup
2018-01-08 09:21:06 -05:00
1c07066064
Add autopilot functionality based on Consul's autopilot
2017-12-18 14:29:41 -08:00
b775fc7b33
Added support for v2 raft APIs and -raft-protocol option
2017-12-12 10:17:16 -06:00
2dfda33703
Nomad agent reload TLS configuration on SIGHUP ( #3479 )
...
* Allow server TLS configuration to be reloaded via SIGHUP
* dynamic tls reloading for nomad agents
* code cleanup and refactoring
* ensure keyloader is initialized, add comments
* allow downgrading from TLS
* initalize keyloader if necessary
* integration test for tls reload
* fix up test to assert success on reloaded TLS configuration
* failure in loading a new TLS config should remain at current
Reload only the config if agent is already using TLS
* reload agent configuration before specific server/client
lock keyloader before loading/caching a new certificate
* introduce a get-or-set method for keyloader
* fixups from code review
* fix up linting errors
* fixups from code review
* add lock for config updates; improve copy of tls config
* GetCertificate only reloads certificates dynamically for the server
* config updates/copies should be on agent
* improve http integration test
* simplify agent reloading storing a local copy of config
* reuse the same keyloader when reloading
* Test that server and client get reloaded but keep keyloader
* Keyloader exposes GetClientCertificate as well for outgoing connections
* Fix spelling
* correct changelog style
2017-11-14 17:53:23 -08:00
46bc4280b2
Adding support for tagged metrics
2017-11-01 13:15:06 -07:00
524a1f0712
Publishing metrics for job summary
2017-11-01 13:15:06 -07:00
c53aac9eea
Agent Health Endpoint
2017-10-13 15:37:44 -07:00
4173834231
Enable more linters
2017-09-26 15:26:33 -07:00
e5ec915ac3
sync
2017-09-19 10:08:23 -05:00
1238efc2a8
improve documentation
...
move metrics to telemetry; copy to client config
2017-09-06 21:38:06 +00:00
66fa05405a
tagged metrics config options should be on telemetry config
...
better api example, add telemetry documentation
2017-09-06 15:25:36 +00:00
751fc5324e
add http endpoint for in memory metrics
...
prevent against flaky test due to timing/initialization issues
2017-09-06 13:51:19 +00:00
3e46094cee
Passthrough replication token for token/policy replication
2017-09-04 13:05:53 -07:00
792f176a44
agent: thread ACL config to client
2017-09-04 13:04:45 -07:00
7d4aa1975f
agent: thread through ACL config to Server
2017-09-04 13:04:45 -07:00
1a86aecf55
Add version package
...
This PR adds a version package and consolidates version strings into a
Version struct.
2017-08-16 15:44:21 -07:00
4b83eba599
Use seen more conservatively
2017-07-24 16:48:40 -07:00
809724ad8d
Track whether Consul has ever been seen
...
Need a way to squelch Consul operation errors on shutdown. If it's never
been seen don't log errors about deregs failing.
2017-07-24 12:12:02 -07:00
747d67eb3f
Allow tuning of heartbeat ttls
...
This PR allows tuning of heartbeat TTLs. An example of very aggressive
settings is as follows:
```
server {
heartbeat_grace = "1s"
min_heartbeat_ttl = "1s"
max_heartbeats_per_second = 200.0
}
```
2017-07-19 09:38:35 -07:00
c643e6b0d1
Add config options
2017-07-07 12:05:56 -07:00
b9c9e6e557
Fix no_host_uuid parsing
...
Need to pointerify it to default to true since we can't tell false from
unset if it's not a pointer.
2017-07-03 17:41:20 -07:00
95a00cbef1
Fix path used by Nomad Server HTTP Check
...
Fixes #2701
2017-06-21 10:41:28 -07:00
0453c2709c
Add new gc_max_allocs tuneable
...
More than gc_max_allocs may be running on a node, but terminal allocs
will be garbage collected to try to keep the total number below the
limit.
2017-05-11 17:18:02 -07:00
c4ab8211b6
Skip https health check if verify_https_client is true
2017-05-03 12:19:02 -07:00
66aea59650
Fix error check for consul skip tls verify support
2017-05-02 17:38:18 -07:00
e997ae44a5
Skip checks with TLSSkipVerify if it's unsupported
...
Fixes #2218
2017-04-19 12:45:34 -07:00
cdb7d2ebb6
Use go-version instead of manual version parsing
2017-04-19 12:42:48 -07:00
947e31e9c2
Only register HTTPS agent check when Consul>=0.7.2
...
Support for TLSSkipVerify in other checks coming soon!
2017-04-19 12:42:48 -07:00
e204a287ed
Refactor Consul Syncer into new ServiceClient
...
Fixes #2478 #2474 #1995 #2294
The new client only handles agent and task service advertisement. Server
discovery is mostly unchanged.
The Nomad client agent now handles all Consul operations instead of the
executor handling task related operations. When upgrading from an
earlier version of Nomad existing executors will be told to deregister
from Consul so that the Nomad agent can re-register the task's services
and checks.
Drivers - other than qemu - now support an Exec method for executing
abritrary commands in a task's environment. This is used to implement
script checks.
Interfaces are used extensively to avoid interacting with Consul in
tests that don't assert any Consul related behavior.
2017-04-19 12:42:47 -07:00
a1a7941dec
Various fixes
...
This PR:
* Uses Go 1.8 executable lookup
* Stores any err message from stats init method
* Allows overriding of Cpu Compute for hosts where it can't be detected
2017-03-14 12:56:31 -07:00
70e4feb045
Limit parallelism during garbage collection
...
This PR introduces a parallelism limit during garbage collection. This
is used to avoid large resource usage spikes if garbage collecting many
allocations at once.
2017-03-10 16:27:00 -08:00
6910678c21
Allow random UUID
2017-02-27 13:42:37 -08:00
6afcba9e22
Allow specification of eval/job gc threshold
2017-02-27 11:58:10 -08:00
15ffdff497
Vault Client on Server handles SIGHUP
...
This PR allows the Vault client on the server to handle a SIGHUP. This
allows updating the Vault token and any other configuration without
downtime.
2017-02-01 14:24:10 -08:00
11d7cb1230
Making the GC related fields tunable
2017-01-31 15:51:20 -08:00
1999b7eebb
Merge pull request #2159 from hashicorp/b-consul-config
...
Fixed merging consul config
2017-01-18 16:14:54 -08:00
e927de02d2
Moved functions to helper from structs
2017-01-18 15:55:14 -08:00
b1d0078db5
Filter executor log messages
2017-01-12 11:54:19 -08:00
6a4efbf3b2
Fix TestRetryJoin
2016-11-15 15:28:21 -08:00
a75e5b5803
Addresses are just addresses - no ports
...
Store address+port in an unexported field for ease-of-use
2016-11-09 11:49:55 -08:00
290f71f62e
Restore sync/atomic use for consistency
2016-11-08 16:21:52 -08:00
1e3a2ae04d
Match old error messages
2016-11-08 15:25:29 -08:00
c735589f41
Choose safer default advertise address
...
* -dev mode defaults bind & advertise to localhost
* Normal mode defaults bind to 0.0.0.0 & advertise to the resolved
hostname. If the hostname resolves to localhost it will refuse to
start and advertise must be manually set.
2016-11-08 11:17:16 -08:00
98bd0e2100
Disable http checks if TLS is enabled
2016-11-03 17:33:58 -07:00
4a1fa67f26
Separate agent configuration into its own pages
...
I apologize in advance for the rather long PR, but unfortunately there
is not an easy way to break this up into smaller chunks. This separates
the agent configuration into smaller, more consumable pieces just like
the job specification.
2016-11-02 18:30:00 -04:00
75696f4c70
Resolve
2016-11-01 13:23:44 -07:00
376651bcb2
Allow advertise to not specify port
2016-10-31 17:12:29 -07:00
ddf5fb82b5
Small cleanups
2016-10-27 10:51:11 -07:00
59489e4e4f
Never return 0.0.0.0 as a non-bind address
2016-10-27 10:51:11 -07:00
7511490c12
Add more address selector tests
2016-10-27 10:51:11 -07:00
af3de22d4f
Remove old address advertise config code
2016-10-27 10:51:11 -07:00
de33949df8
Add address selector methods to the agent
2016-10-27 10:51:11 -07:00
06909d2465
Use bind address for consul healtchecks in nomad client too
2016-10-27 10:47:45 -07:00
c0ff732513
Use bind address for consul service checks
2016-10-27 10:47:45 -07:00
a051d87c5d
Making the nomad agent checks tcp
2016-10-26 16:30:29 -07:00
067fcda3fe
Making the cli use TLS if the client has enabled TLS
2016-10-26 11:13:53 -07:00
cf35aeac84
Moving the TLSConfig to structs
2016-10-25 15:57:38 -07:00
e03927bb5c
Changed the way TLS config is parsed
2016-10-24 13:56:19 -07:00
2e3118e69c
Implemented TLS support for http and rpc
2016-10-23 22:22:00 -07:00
0f6e0d10b6
Enable serf encryption ( #1791 )
...
* Added the keygen command
* Added support for gossip encryption
* Changed the URL for keyring management
* Fixed the cli
* Added some tests
* Added tests for keyring operations
* Added a test for removal of keys
* Added some docs
* Fixed some docs
* Added general options
2016-10-17 10:48:04 -07:00
4e8576516f
Increase the timeout to be greater than the RPC hold threshold
2016-08-26 11:13:20 -07:00
7d899b6c60
Pass Vault config to client
2016-08-17 16:23:29 -07:00
c913e4396f
Add Vault config to server
2016-08-17 16:23:29 -07:00
9a75052d2c
Merge pull request #1518 from pubnub/feature/chroot-map-rebase
...
Add config field to specify chroot mapping for exec driver
2016-08-10 17:00:03 -07:00
7df692226a
Add config field to specify chroot mapping for exec driver
...
- Same format as used by the internal chroot mapping
- Map: source_path -> dest_path
- Example HCL:
client {
chroot_env {
"/etc" = "/etc"
"/lib" = "/lib"
"/opt/projects/foo/bin" = "/usr/bin"
}
}
2016-08-03 17:17:17 -07:00
41b540fbc8
Allow operators to opt into publishing node and alloc metrics
2016-08-01 19:52:20 -07:00
c96f3ccb28
Use net.JoinHostPort
2016-07-25 23:34:08 +02:00
a04907ad91
Fix IPv6 advertise addresses
2016-07-25 22:52:18 +02:00
f3bc85c20f
use microsecond granularity
2016-06-22 15:20:56 -07:00
9406739ba8
remove resolving client rpc addr
2016-06-17 14:55:37 -07:00
46e2d54acf
Provide `nomad.Config` with a default `LogOutput` of `os.StdErr`
2016-06-17 06:44:10 -07:00
9a60999100
Pass a logger arg to `NewClient` and `NewServer`
2016-06-16 23:29:23 -07:00
c2f96ee6ba
Remove the shutdownCh as an argument, it was already present in the receiver
2016-06-16 23:15:41 -07:00
e58c2a2ed4
Remove client registration of RPC services, only servers have an RPC interface
2016-06-16 18:14:59 -07:00
2a2b35d1bc
Use `/v1/agent/servers` (vs `/v1/agent/self`).
2016-06-16 17:40:47 -07:00
26cc057751
Automatically register service checks for HTTP, RPC, and Serf.
2016-06-16 17:34:25 -07:00
af55b74114
Merge pull request #1276 from hashicorp/f-consul-server-autojoin
...
Teach Nomad servers how to fall back to Consul.
2016-06-16 14:40:45 -07:00
14747848f7
Removed the stats block from client configuration
2016-06-16 21:30:29 +01:00
b0fecbefc1
Define `BootstrapExepct` as an `int32` so it can be manipulated atomically.
2016-06-16 12:00:15 -07:00
f05514335b
Teach Nomad servers how to fall back to Consul.
2016-06-15 12:40:51 -07:00
fd80e6ed0f
Added a warn message for deprecated consul options
2016-06-15 02:31:57 +02:00
6e22b680ce
Disambiguate `auto_join` from `auto_register`, rename reg to `auto_advertise`.
...
Provide an option that describes the value to the user vs the
operation performed by the software. Momentarily introducing
`auto_join`
2016-06-14 12:11:38 -07:00
4b04e503f3
address comments
2016-06-13 17:32:18 -07:00
8bbf4a55e5
Fix IDs and domain scoping
2016-06-13 16:30:58 -07:00
d019d8ef8e
implemented reconciliation of unwanted services
2016-06-13 14:52:26 +02:00
753d50f3d4
agent just calls run
2016-06-12 22:30:52 -07:00
8e231fa382
Rename ConsulService back to Service
2016-06-12 16:36:49 -07:00
fdda90229f
only support latest and remove ring buffer
2016-06-12 09:32:38 -07:00
917766a3df
Prefer `%+q` over `%q` in log messages.
2016-06-11 18:17:20 -04:00
bbd8dfa798
goling(1) compliance pass (e.g. Rpc* -> RPC)
2016-06-10 23:38:28 -04:00
c92306f020
Stash client and server registration behind consul.auto_register
2016-06-10 15:54:39 -04:00
a1a0c313de
Skip nil check for agent's consulSyncer is always not nil
2016-06-10 15:54:39 -04:00
c54539b860
Initialize Consul for the Nomad Agent in a more uniform way.
...
Decompose Client and Server registration into `setupClient()` and
`setupServer()`, respectively.
2016-06-10 15:54:39 -04:00
d307ca2c28
Begin leveraging the Agent-level consul.Syncer
2016-06-10 15:54:39 -04:00
8c813630e6
Move package client/consul/sync to command/agent/consul.
...
This has been done to allow the Server and Client to reuse the same
Syncer because the Agent may be running Client, Server, or both
simultaneously and we only want one Syncer object alive in the agent.
2016-06-10 15:54:39 -04:00
6b126ce488
Change the API signature of Syncer.SyncServices().
...
SyncServices() immediately attempts to sync whatever information
the process has with Consul. Previously this method would take an
argument of the exclusive list of services that should exist,
however this is not condusive to having a Nomad Client and Nomad
Server share the same consul.Syncer.
2016-06-10 15:54:39 -04:00
e24de53ead
Refine Nomad's Consul `port` handling.
...
Previously this would immediately default to '127.0.0.1' if the
config was set to `:some-port-number`. Now it uses the BindAddr
if available. Also, if the `port` option is set to just a port`
number (e.g. '1234'), attempt to parse the port number by itself
to allow statically configured ports to work, even when no host is
specified.
2016-06-10 15:54:39 -04:00
555f4fe135
Change client/consul.NewSyncer() to accept a shutdown channel
...
In addition to the API changing, consul.Syncer can now be signaled
to shutdown via the Shutdown() method, which will call the Run()'ing
sync task to exit gracefully.
2016-06-10 15:54:39 -04:00
e2a561204a
Fix typo in comment
2016-06-10 15:50:11 -04:00
7fe1944bc5
Fix tests for client.TestAgent_ServerConfig
...
Add similar logic in Agent `serverConfig()` to set up the
`serverSerfAddr` the same as `serverHttpAddr` and `serverRpcAddr`.
2016-06-10 15:50:11 -04:00
cb80e93a6b
Move client.DefaultConfig() to client/config.DefaultConfig()
...
Resolves an import cycle in testing and is more appropriate because
the default should reside next to its struct definition.
2016-06-10 15:50:11 -04:00
b3fd455b1f
Register the serf service with the Nomad server service.
...
This will be unused in this PR.
2016-06-10 15:50:11 -04:00
6cd01d7567
Nuke a.consulConfig in favor of a.consul.Config
2016-06-10 15:50:11 -04:00
1fe979a5e4
Remove types.ShutdownChannel and replace with `chan struct{}`
2016-06-10 15:50:11 -04:00
9fb0104def
Teach Client to reuse an Agent's consulSyncer.
...
"There can be only one."
2016-06-10 15:50:11 -04:00
47891fb559
Register two services each for clients and servers, http and rpc.
...
In order to give clients a fighting chance to talk to the right port,
differentiate RPC services from HTTP services by registering two
services with different tags. This yields
`rpc.nomad-server.service.consul` and
`http.nomad-server.service.consul` which is immensely more useful to
clients attempting to bootstrap their world.
2016-06-10 15:50:11 -04:00
ac7881226c
Advertise the server's RPC endpoint, not its HTTP endpoint.
...
Rename c.serverRpcAddr to serverRpcAddr. This will be broken out
into in additional set of services in a subsequent commit.
2016-06-10 15:50:11 -04:00
10d1b6ba9b
Reduce all forms of ConsulConfig down to a single struct
...
nomad/structs/config/consul.go's ConsulConfig is the canonical definition
for all things Consul now.
2016-06-10 15:50:11 -04:00
520fcb99de
Rename consulConfig to consulAgentConfig
2016-06-10 15:50:11 -04:00
e6c78274c8
Rename client/config/config's ConsulConfig to ConsulAgentConfig
...
A follow up commit to the previous rename. More to come.
2016-06-10 15:50:11 -04:00
882364f4d9
An Agent's consulSyncer is always not nil now.
2016-06-10 15:50:11 -04:00
98cea9ba07
Revise Agent.syncAgentServicesWithConsul()'s interface
...
Reduce down to its lowest common helper function denominator.
2016-06-10 15:50:11 -04:00
d1442dc317
Reduce all forms of ConsulConfig down to a single struct
...
nomad/structs/config/consul.go's ConsulConfig is the canonical definition
for all things Consul now.
2016-06-10 15:50:11 -04:00
e60580b279
Define a type for the PeriodicCallback handlers and ShutdownChannel
2016-06-10 15:50:11 -04:00
17116fc5a7
Rebalance Nomad client RPCs among different Nomad servers.
...
Implement client/rpc_proxy.RpcProxy.
2016-06-10 15:50:11 -04:00
88f3422d7c
Rename NewConsulService to NewSyncer
2016-06-10 15:49:37 -04:00
b9789e0cf7
Rename client/consul/sync.PeriodicSync to Run
2016-06-10 15:49:37 -04:00
768aab015d
Rename client/consul/sync.ConsulService to client/consul/sync.Syncer
...
Syncer describes the responsibility and actions of the type.
2016-06-10 15:49:37 -04:00
840b49a720
Distill config.Config.ConsulConfig down to config.Config.Consul
...
The enclosed struct provides the necessary context
2016-06-10 15:48:36 -04:00
c7a680c28f
Rename consulConfig to consulAgentConfig
2016-06-10 15:48:36 -04:00
73e173673e
Rename client/config/config's ConsulConfig to ConsulAgentConfig
...
A follow up commit to the previous rename. More to come.
2016-06-10 15:48:36 -04:00
e97652bbfb
Rename consul.ConsulConfig to consul.AgentConfig
...
There were two `ConsulConfig` structs running around, one of them
needed to go away. Rely on the package's path to provide context
for the type of AgentConfig.
2016-06-10 15:48:36 -04:00
c46400597e
Making the stats collection interval and number of data points to keep in memory configurable
2016-05-28 19:59:20 -07:00
bf5b78b77e
Added a test for generation of the server addr
2016-05-24 21:29:13 -07:00
227477533c
Using the bind addr as server's IP if advertise addr and addresses.HTTP is not specified
2016-05-24 17:27:09 -07:00
c07770fd4d
Using the auto-register flag
2016-05-16 18:23:01 +02:00
1b3356375d
Using advertise addr for the http address
2016-05-14 01:09:05 -07:00
ecd995b283
Added some docs
2016-05-14 00:36:26 -07:00
df9d256132
Making Nomad register services with consul in dev mode
2016-05-13 10:19:49 -07:00
347cb890d2
Removed allocID and task name from consul service
2016-05-11 16:26:41 -07:00
83fed62a0a
Implemented registering client and server services
2016-05-11 16:07:02 -07:00
891b201eef
removing interfaces from config
2016-04-07 13:25:38 -07:00
aefa4b9207
add nomad fingerprinter
2016-03-22 17:12:30 -07:00
acd016c7c0
Overriding the device configuration if addresses is present
2016-03-20 23:41:16 -07:00
cfb9ae91a1
Using the interfaces for configuring bind addrs for nomad servers
2016-03-19 00:54:19 -07:00
6e6f2940f0
Setting the ip from the network interface to Addresses
2016-03-18 22:26:13 -07:00
4f689d1a38
Using the interface name to find bind address for network services
2016-03-18 21:44:22 -07:00
601008706d
Added an interfaces config
2016-03-18 13:50:29 -07:00
7d4c19ed99
reserve resources on the node
2016-03-13 19:05:41 -07:00
b18cbaa416
Using unix domain sockets for docker syslog collector
2016-03-09 23:28:26 -08:00
Alex Dadgar