luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
17501 commits 1 branch 0 tags 311 MiB
Commit graph

1542 commits

Author SHA1 Message Date
Chelsea Holland Komlo e8379c9059 skip update checking if DisableUpdateCheck is set to true 2018-08-10 13:08:13 -04:00
Chelsea Holland Komlo b92098fd08 change function signature to take entire tls config object 2018-08-10 12:37:21 -04:00
Chelsea Holland Komlo 75d631a1c8 fix reload issue for tls certificates in dev mode 2018-07-05 17:08:31 -04:00
Alex Dadgar b61051b3cd
Merge pull request #4409 from hashicorp/r-client-packages 
Refactor client packages
 2018-06-13 17:32:25 -07:00
Alex Dadgar 300b1a7a15 Tests only use testlog package logger 2018-06-13 15:40:56 -07:00
Chelsea Komlo 03075b603a
Merge pull request #4399 from hashicorp/r-reload-refactor 
Refactor logic for dynamic reloading
 2018-06-13 13:35:12 -04:00
Alex Dadgar 90c2108bfb Fix gc tests + parallel destroy + small test fixes 2018-06-12 10:23:45 -07:00
Alex Dadgar af5753d2cd bump version + generated files 2018-06-11 13:39:42 -07:00
Chelsea Holland Komlo 3b5d5c7be8 remove logic to reload RPC connections from agent 2018-06-08 13:14:40 -04:00
Alex Dadgar 8efe9696ad move log line 2018-06-07 15:12:51 -07:00
Chelsea Komlo d738976234
Merge pull request #4395 from hashicorp/b-vault-second 
Fix for dynamically reloading vault
 2018-06-07 18:03:00 -04:00
Chelsea Holland Komlo dcc9cdfeb7 fixup! comment and move to always log server reload operation 2018-06-07 17:12:36 -04:00
Chelsea Holland Komlo 9f6bd7bf3a move logic for testing equality for vault config 2018-06-07 16:23:50 -04:00
Chelsea Holland Komlo 282f37b1ee fix for dynamically reloading vault 2018-06-07 15:34:18 -04:00
Michael Schurter f8e12e6ee7 agent: global logger should use the same flags 
Prior to this change logs from the global logger only used seconds:

```
2018/06/06 18:25:58 http: TLS handshake error from ...
```

After this change they properly use the microseconds flag:

```
2018/06/06 18:39:50.702447 http: TLS handshake error ...
```

They still lack a log level unfortunately.
 2018-06-06 11:40:08 -07:00
Preetha Appan 82837839eb
Fix bug with determining when agent is a client 
This fixes a bug introduced in commit e27caadca6 that sets a boolean flag
when the agent is a client. It incorrectly checked state before initializing
the client. This leads to Nomad clients not deregistering any services registered
in Consul after allocs are destroyed
 2018-06-05 19:19:52 -05:00
Alex Dadgar c0386819b3 bump version/lint/generated files 2018-06-01 15:23:10 -07:00
Alex Dadgar 247f1edb11 spelling 2018-06-01 14:53:08 -07:00
Preetha Appan ce6d4a8d7a
Fix tests and move isClient to constructor 2018-06-01 15:59:53 -05:00
Preetha Appan a5bfaa098c
Fix unnecessary deregistration in consul sync 
This commit fixes an issue where if a nomad client and server shared the same consul instance, the server would deregister any services and checks registered by clients for running tasks.
 2018-06-01 14:48:25 -05:00
Alex Dadgar 40fec81315
Merge pull request #4277 from hashicorp/f-retry-join-clients 
Add go-discover support to Nomad clients
 2018-06-01 16:57:40 +00:00
Alex Dadgar aca8d5cece Actually disable the schedulers 2018-05-31 13:11:11 -07:00
Alex Dadgar d098885b79 Disable schedulers for TestHTTP_AllocSnapshot_Atomic 2018-05-31 12:05:44 -07:00
Alex Dadgar 4765b62284 Improve validation/defaulting, handle start-join 
This commit:
* Improves how we combine the old retry-* fields and the new stanza and
how it is validated
* Handles the new stanza setting start_join
* Fixes integration test to not bind to the standard port and instead be
randomized.
* Simplifies parsing of the old retry_interval
* Fixes the errors from retry join being masked
* Flags get parsed into new server_join stanza
 2018-05-31 10:53:26 -07:00
Alex Dadgar e1bf8780b5 validation errors 2018-05-31 10:53:26 -07:00
Alex Dadgar a02fbe3e0f indentation 2018-05-31 10:53:26 -07:00
Chelsea Holland Komlo 2bf2af4378 ensure default value of 30s is set for server_join stanza 2018-05-31 10:50:04 -07:00
Chelsea Holland Komlo 307458d4a3 ignore default values for retry interval 
add additional validation case
 2018-05-31 10:50:04 -07:00
Chelsea Holland Komlo ebc758aa0e add stronger protections for nil pointers in server join merge 2018-05-31 10:50:04 -07:00
Chelsea Holland Komlo 10aff14509 update config parse test 
documentation fixes
 2018-05-31 10:50:04 -07:00
Chelsea Holland Komlo ac1411ce95 RetryInterval should be a time.Duration 2018-05-31 10:50:04 -07:00
Chelsea Holland Komlo e79bc29e1a set retryInterval and other code feedback 2018-05-31 10:50:04 -07:00
Chelsea Holland Komlo de03c884bc add further configuration validation for server_join 2018-05-31 10:50:04 -07:00
Chelsea Holland Komlo df7539b9d0 update documentation for server_join 2018-05-31 10:50:04 -07:00
Chelsea Holland Komlo a4e514e07f update server_join naming and improve logging 2018-05-31 10:50:03 -07:00
Chelsea Holland Komlo 064b5481e0 add server join info to server and client 2018-05-31 10:50:03 -07:00
Preetha Appan 7414395daa
Use constant in test 2018-05-30 17:27:04 -05:00
Preetha Appan 6cbd25945c
Add unit test to verify compatibility code for node drains 2018-05-30 17:14:53 -05:00
Preetha Appan 4f835790d7
Set node eligibility to true when old client calls disable 2018-05-30 16:54:07 -05:00
Preetha Appan 34db410b74
Fix failing test TestClientStatusRequest 2018-05-30 15:11:54 -05:00
Preetha Appan 2752204f26
Fix failing test TestHTTP_AllocAllGC 2018-05-30 15:11:54 -05:00
Chelsea Holland Komlo 19e4a5489b add support for tls PreferServerCipherSuites 
add further tests for tls configuration
 2018-05-25 13:20:00 -04:00
Chelsea Komlo af15dda45a
Merge pull request #4328 from hashicorp/r-single-tls-config-constructor 
Refactor to prefer using NewTLSConfiguration constructor
 2018-05-24 13:46:29 -04:00
Alex Dadgar b1de61e012
Merge pull request #4321 from hashicorp/f-network-info 
Display bind/advertise addresses on agent startup
 2018-05-24 17:30:56 +00:00
Chelsea Holland Komlo 38f611a7f2 refactor NewTLSConfiguration to pass in verifyIncoming/verifyOutgoing 
add missing fields to TLS merge method
 2018-05-23 18:35:30 -04:00
Alex Dadgar 51e67daf69 Use Tags when CanaryTags isn't specified 
This PR fixes a bug where we weren't defaulting to `tags` when
`canary_tags` was empty and adds documentation.
 2018-05-23 13:07:47 -07:00
Alex Dadgar dd52ec402c Display bind/advertise addresses on agent startup 
Sample outputs from demo/vagrant/(server/client1).hcl and `nomad agent -dev` mode

Server:

```
==> Nomad agent configuration:

       Advertise Addrs: HTTP: 192.168.1.75:4646; RPC: 192.168.1.75:4647; Serf: 192.168.1.75:4648
            Bind Addrs: HTTP: 0.0.0.0:4646; RPC: 0.0.0.0:4647; Serf: 0.0.0.0:4648
                Client: false
             Log Level: DEBUG
                Region: global (DC: dc1)
                Server: true
               Version: 0.8.4-dev
```

Client:

```
==> Nomad agent configuration:

       Advertise Addrs: HTTP: 192.168.1.75:5656
            Bind Addrs: HTTP: 0.0.0.0:5656
                Client: true
             Log Level: DEBUG
                Region: global (DC: dc1)
                Server: false
               Version: 0.8.4-dev
```

Dev:

```
==> Nomad agent configuration:

       Advertise Addrs: HTTP: 127.0.0.1:4646; RPC: 127.0.0.1:4647; Serf: 127.0.0.1:4648
            Bind Addrs: HTTP: 127.0.0.1:4646; RPC: 127.0.0.1:4647; Serf: 127.0.0.1:4648
                Client: true
             Log Level: DEBUG
                Region: global (DC: dc1)
                Server: true
               Version: 0.8.4-dev
```
 2018-05-22 15:14:33 -07:00
Alex Dadgar 44697efd9a safety guard 2018-05-22 14:45:34 -07:00
Alex Dadgar 586895965c Unit test for dev agent 2018-05-22 14:45:34 -07:00
Alex Dadgar 58d2a4c7c2 Do not bypass normal RPC codepath when running both client and server at once 2018-05-22 14:45:34 -07:00
Alex Dadgar 21c5ed850d Register events 2018-05-22 14:06:33 -07:00
Preetha 159888a856
Merge pull request #4274 from hashicorp/f-force-rescheduling 
Add CLI and API support for forcing rescheduling of failed allocs
 2018-05-21 16:24:22 -07:00
Preetha Appan bfa0937bbb
Code review feedback 2018-05-10 14:42:24 -05:00
Chelsea Holland Komlo 44f536f18e add support for configurable TLS minimum version 2018-05-09 18:07:12 -04:00
Chelsea Holland Komlo 796bae6f1b allow configurable cipher suites 
disallow 3DES and RC4 ciphers

add documentation for tls_cipher_suites
 2018-05-09 17:15:31 -04:00
Preetha Appan b12df3c64b
Added CLI for evaluating job given ID, and modified client API for evaluate to take a request payload 2018-05-09 15:04:27 -05:00
Preetha Appan c1b92c284e
Work in progress - force rescheduling of failed allocs 2018-05-08 17:26:57 -05:00
Preetha e7ae6e98d9
Merge pull request #4259 from hashicorp/f-deployment-improvements 2018-05-08 16:37:10 -05:00
Chelsea Holland Komlo 136635f04d only write error log line on error 2018-05-07 16:57:07 -04:00
Chelsea Holland Komlo 30584639b5 remove log line for empty addresses which could confuse on initalization 2018-05-07 16:57:07 -04:00
Chelsea Holland Komlo 24ff40df01 retry until all options are exhausted 2018-05-07 16:57:07 -04:00
Chelsea Holland Komlo ec4be4f871 ensure provider= is always the string prefix 2018-05-07 16:57:07 -04:00
Chelsea Holland Komlo 5422b1b088 update test for more realistic IP address from go-discover 2018-05-07 16:57:07 -04:00
Chelsea Holland Komlo 7e4d4f8088 comments and other fixups 2018-05-07 16:57:06 -04:00
Chelsea Holland Komlo 8f584f6474 add go-discover 2018-05-07 16:57:06 -04:00
Chelsea Holland Komlo 25ad6eaf96 refactor to retryJoiner interface 2018-05-07 16:57:06 -04:00
Michael Schurter f1d13683e6
consul: remove services with/without canary tags 
Guard against Canary being set to false at the same time as an
allocation is being stopped: this could cause RemoveTask to be called
with the wrong Canary value and leaking a service.

Deleting both Canary values is the safest route.
 2018-05-07 14:55:01 -05:00
Michael Schurter 50e04c976e
consul: support canary tags for services 
Also refactor Consul ServiceClient to take a struct instead of a massive
set of arguments. Meant updating a lot of code but it should be far
easier to extend in the future as you will only need to update a single
struct instead of every single call site.

Adds an e2e test for canary tags.
 2018-05-07 14:55:01 -05:00
Alex Dadgar f4af30fbb5
Canary tags structs 2018-05-07 14:50:01 -05:00
Alex Dadgar ee50789c22
Initial implementation 2018-05-07 14:50:01 -05:00
Michael Schurter 0d534d30d6
Merge pull request #4251 from hashicorp/f-grpc-checks 
Support Consul gRPC Health Checks
 2018-05-04 14:55:16 -07:00
Michael Schurter f6a4713141 consul: make grpc checks more like http checks 2018-05-04 11:08:11 -07:00
Michael Schurter 382caec1e1 consul: initial grpc implementation 
Needs to be more like http.
 2018-05-04 11:08:11 -07:00
Michael Schurter 526af6a246 framer: fix early exit/truncation in framer 2018-05-02 10:46:16 -07:00
Michael Schurter 5d6bf2938f client: use a bytes.Reader for reading a []byte 2018-05-02 10:46:16 -07:00
Michael Schurter e00e265da5 client: ensure cancel is always called when func exits 2018-05-02 10:46:16 -07:00
Michael Schurter d21b749c68 client: give pipe conns meaningful names 2018-05-02 10:46:16 -07:00
Michael Schurter 5ef0a82e6e client: reset encoders between uses 
According to go/codec's docs, Reset(...) should be called on
Decoders/Encoders before reuse:

https://godoc.org/github.com/ugorji/go/codec

I could find no evidence that *not* calling Reset() caused bugs, but
might as well do what the docs say?
 2018-05-02 10:46:16 -07:00
Alex Dadgar 43192cefae generated files 2018-04-26 16:28:58 -07:00
Michael Schurter cfcbb9fa21 consul: periodically reconcile services/checks 
Periodically sync services and checks from Nomad to Consul. This is
mostly useful when testing with the Consul dev agent which does not
persist state across restarts. However, this is a reasonable safety
measure to prevent skew between Consul's state and Nomad's
services+checks.

Also modernized the test suite a bit.
 2018-04-19 15:45:42 -07:00
Nick Ethier 2e6c95f511
Merge pull request #4138 from hashicorp/i-hcl-json-endpoint 
HCL to JSON api endpoint
 2018-04-19 14:18:34 -04:00
Michael Schurter d3650fb2cd test: build with mock_driver by default 
`make release` and `make prerelease` set a `release` tag to disable
enabling the `mock_driver`
 2018-04-18 14:45:33 -07:00
Nick Ethier 555b036369
command/agent: fix url in jobs parse ep test 2018-04-16 19:21:09 -04:00
Nick Ethier 0baf9ba385
command/agent: fix ptr ref in job endpoint test 2018-04-16 19:21:09 -04:00
Nick Ethier de4176606d
command/agent: add Canonicalize option to parse args 2018-04-16 19:21:09 -04:00
Nick Ethier 31da01856a
command/agent: add HCL mock for parse endpoint 2018-04-16 19:21:09 -04:00
Nick Ethier f2db03e56c
command/agent: add /v1/jobs/parse endpoint 
The parse endpoint accepts a hcl jobspec body within a json object
and returns the parsed json object for the job. This allows users to
register jobs with the nomad json api without specifically needing
a nomad binary to parse their hcl encoded jobspec file.
 2018-04-16 19:21:06 -04:00
Alex Dadgar 663c4d0433 Version bump and generated files 2018-04-12 16:21:50 -07:00
Preetha Appan f04c9d2b19
minor code review fix 2018-04-10 15:33:01 -05:00
Preetha Appan 6b6e97e50c
Dev mode should never persist nodeid 2018-04-10 12:34:14 -05:00
Preetha Appan e1c3c7ab6c
Lint fixes 2018-04-10 11:22:16 -05:00
Preetha Appan 6d0e1c9fea
Use preconfigured nodeID if there isn't a persisted node ID, and persist it if its not persisted. 2018-04-10 08:47:33 -05:00
Alex Dadgar d179a09b83 WIP: Not setting node id properlperly 2018-04-09 18:01:28 -07:00
Alex Dadgar 6871a068cb Fix prefix output of agent metadata 2018-03-30 16:27:18 -07:00
Alex Dadgar af81349dbe Generated files 2018-03-30 16:14:40 -07:00
Michael Schurter 7199a2b960 cli: differentiate normal output vs info 2018-03-30 11:42:11 -07:00
Chelsea Komlo 607e631714
Merge pull request #4046 from hashicorp/tls-same-file-reload 
Check file contents when determining if agent should reload TLS confi…
 2018-03-29 10:51:32 -04:00
Chelsea Holland Komlo be15c07110 make check fix 2018-03-28 19:11:51 -04:00
Chelsea Holland Komlo 58ada9bc42 return error when setting checksum; don't reload 2018-03-28 18:15:50 -04:00
Chelsea Holland Komlo 38c7a38549 output warning for error in creating TLS checksum 2018-03-28 17:44:23 -04:00
Chelsea Holland Komlo 770ac13bc4 check for nil, remove unnecessary set checksum call 2018-03-28 13:29:53 -04:00
Preetha 177d2d6010
Merge pull request #4052 from hashicorp/f-specify-total-memory 
Allow to specify total memory on agent configuration
 2018-03-28 12:28:41 -05:00
Chelsea Holland Komlo d3d686b58a fix up test for file content changes 2018-03-28 13:18:13 -04:00
Preetha Appan 5c2fc6d46b
Fix compilation fail in test after rebase 2018-03-28 11:54:34 -05:00
Chelsea Holland Komlo 2d5af7ff4d set TLS checksum when parsing config 
Refactor checksum comparison, always set checksum if it is empty
 2018-03-28 09:56:11 -04:00
Alex Dadgar de4b3772f1 Create evals for system jobs when drain is unset 
This PR creates evals for system jobs when:

* Drain is unset and mark eligible is true
* Eligibility is restored to the node
 2018-03-27 15:53:24 -07:00
Mildred Ki'Lya 1017cbe8ab
Allow to specify total memory on agent configuration 
Allow to set the total memory of an agent in its configuration file. This
can be used in case the automatic detection doesn't work or in specific
environments when memory overcommit (using swap for example) can be
desirable.
 2018-03-27 15:46:18 -05:00
Chelsea Holland Komlo 6e6d6b7e33 check file contents when determining if agent should reload TLS configuration 2018-03-27 15:42:20 -04:00
Alex Dadgar 432784dae3 Fix alloc watcher snapshot streaming 2018-03-27 11:14:53 -07:00
Chelsea Komlo 57e2cd04bd
Merge pull request #4025 from hashicorp/reload-http-tls 
Allow TLS configurations for HTTP and RPC connections to be reloaded …
 2018-03-26 18:00:30 -04:00
Preetha Appan 33e170c15d
s/linear/constant/g 2018-03-26 14:45:09 -05:00
Chelsea Holland Komlo 96df419fff code review feedback 2018-03-26 10:55:22 -04:00
Alex Dadgar 34211f00a7 Allow separate enterprise config overlay 2018-03-22 13:53:08 -07:00
Michael Schurter 0e0b04afec test: fix by using mock.BatchJob 2018-03-21 16:51:45 -07:00
Michael Schurter 39cef16c73 test: don't call t.Fatal from within a goroutine 2018-03-21 16:51:45 -07:00
Michael Schurter cb61a4bdc7 Fix linting errors 2018-03-21 16:51:45 -07:00
Alex Dadgar 7b2bad8c5e Toggle Drain allows resetting eligibility 
This PR allows marking a node as eligible for scheduling while toggling
drain. By default the `nomad node drain -disable` commmand will mark it
as eligible but the drainer will maintain in-eligibility.
 2018-03-21 16:51:44 -07:00
Alex Dadgar 02019f216a Correct defaulting 2018-03-21 16:51:44 -07:00
Alex Dadgar 78c7c36e65 code review 2018-03-21 16:51:44 -07:00
Alex Dadgar 8289cc3c6f HTTP and API 2018-03-21 16:51:44 -07:00
Alex Dadgar b3d2346419 Upgrade path 2018-03-21 16:51:43 -07:00
Alex Dadgar 010228577e Drain cli, api, http 2018-03-21 16:51:43 -07:00
Chelsea Holland Komlo 66e44cdb73 Allow TLS configurations for HTTP and RPC connections to be reloaded separately 2018-03-21 17:51:08 -04:00
Michael Schurter 70c370c6fe
Merge pull request #4003 from jrasell/f_gh_3988 
Allow Nomads Consul health check names to be configurable.
 2018-03-20 16:44:08 -07:00
James Rasell 121c3bc997 Update Consul check params from using health-check to check. 2018-03-20 16:03:58 +01:00
Michael Schurter 86ccdb9115 Fix generating static assets 
Broke due to a change in go-bindata-assetfs
 2018-03-19 15:52:38 -07:00
James Rasell 15afef9b77 Allow Nomads Consul health checks to be configurable. 
This change allows the client HTTP and the server HTTP, Serf and
RPC health check names within Consul to be configurable with the
defaults as previous. The configuration can be done via either a
config file or using CLI flags.

Closes #3988
 2018-03-19 19:37:56 +01:00
Preetha 6df57c177c
Merge pull request #4002 from hashicorp/b-reschedule-systemjob-panic 
Fix incorrect initialization of reschedule policy for system jobs.
 2018-03-19 13:06:55 -05:00
Preetha Appan 161bc66355
Fix incorrect initialization of reschedule policy for system jobs. 2018-03-19 12:16:13 -05:00
Alex Dadgar 9e05c9a50e
Merge pull request #3997 from hashicorp/b-serf-addr 
RPC Advertise used exclusively for Clients
 2018-03-19 09:30:20 -07:00
Alex Dadgar 9ef23ff277 enable server in test 2018-03-16 16:52:37 -07:00
Alex Dadgar b8607ad6d6 Heartbeat uses client rpc advertise and server defaults server rpc advertise addr 2018-03-16 16:47:08 -07:00
Alex Dadgar 52b7fb5361 Separate client and server rpc advertise addresses 2018-03-16 16:47:08 -07:00
Michael Schurter 86f562be3a Remove unnecessary conversions 2018-03-16 16:32:59 -07:00
Michael Schurter c3e8f6319c gofmt -s (simplify) files 2018-03-16 16:31:16 -07:00
Michael Schurter 1044bc0feb
Merge pull request #3984 from hashicorp/f-loosen-consul-skipverify 
Replace Consul TLSSkipVerify handling
 2018-03-16 11:21:28 -07:00
Michael Schurter 0971114f0c Replace Consul TLSSkipVerify handling 
Instead of checking Consul's version on startup to see if it supports
TLSSkipVerify, assume that it does and only log in the job service
handler if we discover Consul does not support TLSSkipVerify.

The old code would break TLSSkipVerify support if Nomad started before
Consul (such as on system boot) as TLSSkipVerify would default to false
if Consul wasn't running. Since TLSSkipVerify has been supported since
Consul 0.7.2, it's safe to relax our handling.
 2018-03-14 17:43:06 -07:00
Alex Dadgar 3537c73289
Merge pull request #3978 from hashicorp/b-core-sched 
Always add core scheduler
 2018-03-14 16:13:15 -07:00
Preetha Appan e75630f8e8
Fix formatting 2018-03-14 16:10:32 -05:00
Preetha Appan 9a5e6edf1f
Rename DelayCeiling to MaxDelay 2018-03-14 16:10:32 -05:00
Preetha Appan 5f50c3d618
Add new reschedule options to API layer and unit tests 2018-03-14 16:10:32 -05:00
Alex Dadgar 92cb552ff6 Always add core scheduler and detect invalid schedulers 2018-03-14 10:53:27 -07:00
Alex Dadgar 63e14b7d63 nodeevents -> events 2018-03-13 18:08:22 -07:00
Chelsea Holland Komlo 1488b076d1 code review feedback 2018-03-13 18:08:21 -07:00
Chelsea Holland Komlo 00d9923454 Ensure node updates don't strip node events 
Add node events to CLI
 2018-03-13 18:05:40 -07:00
Michael Schurter ec381ee705 Revert spelling corrections in generated code 2018-03-12 11:19:29 -07:00
Josh Soref 1359fd2c3d spelling: unexpected 2018-03-11 19:08:07 +00:00
Josh Soref 42d7f19861 spelling: supports 2018-03-11 19:00:11 +00:00
Josh Soref c808dc3095 spelling: submitted 2018-03-11 18:59:27 +00:00
Josh Soref 6e1244b6c1 spelling: significantly 2018-03-11 18:56:45 +00:00
Josh Soref 8978caea28 spelling: shutdown 2018-03-11 18:55:49 +00:00
Josh Soref 05305afcd9 spelling: services 2018-03-11 18:53:58 +00:00
Josh Soref ad55e85e73 spelling: registrations 2018-03-11 18:40:53 +00:00
Josh Soref 6fa892a463 spelling: propagated 2018-03-11 18:39:26 +00:00
Josh Soref d208d26b6e spelling: preemptively 2018-03-11 17:58:48 +00:00
Josh Soref 8abf038f4d spelling: output 2018-03-11 18:35:30 +00:00
Josh Soref 3c1ce6d16d spelling: otherwise 2018-03-11 18:34:27 +00:00
Josh Soref 3e2f500cf9 spelling: largely 2018-03-11 18:21:52 +00:00
Josh Soref 85fabc63c8 spelling: expected 2018-03-11 17:57:01 +00:00
Josh Soref 680bbd6d4f spelling: encountered 2018-03-11 17:58:59 +00:00
Josh Soref 1f927cd343 spelling: each other 2018-03-11 17:56:50 +00:00
Josh Soref 7a6dfa4b1a spelling: current 2018-03-11 17:52:32 +00:00
Josh Soref 6f48e31c00 spelling: convenience 2018-03-11 17:50:48 +00:00
Josh Soref 76cf178933 spelling: cleanup 2018-03-11 17:47:09 +00:00
Josh Soref 3ec4ebc7b1 spelling: canonical 2018-03-11 17:46:01 +00:00
Josh Soref 1d58ae8899 spelling: bootstrap 2018-03-11 17:43:19 +00:00
Josh Soref 455eb1aeb3 spelling: authoritative 2018-03-11 17:42:05 +00:00
Josh Soref 5f87691df1 spelling: asynchronously 2018-03-11 17:41:50 +00:00
Josh Soref 58b794875f spelling: artifact 2018-03-11 17:41:02 +00:00
Josh Soref c74245279b spelling: arbitrary 2018-03-11 17:40:03 +00:00
Alex Dadgar 483e011720
Merge pull request #3892 from hashicorp/f-tunnel 
Client RPC Endpoints, Server Routing and Streaming RPCs
 2018-02-20 16:35:42 -08:00
Alex Dadgar 8b86e64fd8 Show HTTP request method 2018-02-16 15:55:26 -08:00
Alex Dadgar aa98f8ba7b Enhance API pkg to utilize Server's Client Tunnel 
This PR enhances the API package by having client only RPCs route
through the server when they are low cost and for filesystem access to
first attempt a direct connection to the node and then falling back to
a server routed request.
 2018-02-15 13:59:03 -08:00
Alex Dadgar 38b695b69c feedback and rebasing 2018-02-15 13:59:03 -08:00
Alex Dadgar c5e9ebb656 Use helper for forwarding 2018-02-15 13:59:03 -08:00
Alex Dadgar 9117ef4650 HTTP agent 2018-02-15 13:59:03 -08:00
Alex Dadgar d7029965ca Server side impl + touch ups 2018-02-15 13:59:02 -08:00
Alex Dadgar 14845bb918 Refactor determining the handler for a node id call 2018-02-15 13:59:02 -08:00
Alex Dadgar e685211892 Code review feedback 2018-02-15 13:59:02 -08:00
Alex Dadgar f5f43218f5 HTTP and tests 2018-02-15 13:59:02 -08:00
Alex Dadgar 9a5569678c Client Stat/List impl 2018-02-15 13:59:02 -08:00
Alex Dadgar 8854b35b34 Agent logs 2018-02-15 13:59:02 -08:00
Alex Dadgar 69def2ff22 Server tests of logs 2018-02-15 13:59:02 -08:00
Alex Dadgar ddd67f5f11 Server streaming 2018-02-15 13:59:01 -08:00
Alex Dadgar ca9379be09 Logs over RPC w/ lots to touch up 2018-02-15 13:59:01 -08:00
Alex Dadgar 993727c28f Use in-mem rpc 2018-02-15 13:59:01 -08:00
Alex Dadgar d5a834b801 fix lint 2018-02-15 13:59:01 -08:00
Alex Dadgar 9bc75f0ad4 Fix manager tests and make testagent recover from port conflicts 2018-02-15 13:59:01 -08:00
Alex Dadgar 8dcda29c21 Use nomad UUID 2018-02-15 13:59:00 -08:00
Alex Dadgar 71029b6329 Test http 2018-02-15 13:59:00 -08:00
Alex Dadgar 6dd1c9f49d Refactor 2018-02-15 13:59:00 -08:00
Alex Dadgar ad7bc0c6bd Server can forward ClientStats.Stats 2018-02-15 13:59:00 -08:00
Alex Dadgar 1472b943d6 Stats Endpoint 2018-02-15 13:59:00 -08:00
Kyle Havlovitz 54b691f538
Merge pull request #3852 from hashicorp/autopilot-cleanup 
Clean up some leftover autopilot differences from Consul
 2018-02-14 10:42:32 -08:00
Preetha df6400222b
Merge pull request #3868 from hashicorp/f-server-side-restarts 
server side rescheduling
 2018-02-13 20:09:51 -06:00
Kyle Havlovitz 709b693d39 Clean up some leftover autopilot differences from Consul 2018-02-08 10:27:26 -08:00
Mahmood Ali bebafb5234 Add tags option to datadog telemetry 
Expose an global tags option in telemetry config for dogstatsd, for
purposes of distinguishing between multiple nomad cluster metrics.
 2018-02-06 12:08:37 -05:00
Preetha Appan 4a78c5bc84
Fix unit test 2018-01-31 09:56:53 -06:00
Preetha Appan 1f834d1a31
Add reschedule policy to API, and HCL parsing support. 2018-01-31 09:56:53 -06:00
Kyle Havlovitz 0eb0acacdc Fix remaining issues with autopilot change 2018-01-30 15:21:28 -08:00
Kyle Havlovitz 2ccf565bf6 Refactor redundancy_zone/upgrade_version out of client meta 2018-01-29 20:03:38 -08:00
Chelsea Komlo d09cc2a69f
Merge pull request #3492 from hashicorp/f-client-tls-reload 
Client/Server TLS dynamic reload
 2018-01-23 05:51:32 -05:00
Michael Schurter 694b547a6b
Merge pull request #3682 from hashicorp/b-3681-always-set-driver-ip 
Always advertise driver IP when in driver mode
 2018-01-22 16:41:34 -08:00
Chelsea Holland Komlo 7d3c240871 swap raft layer tls wrapper 2018-01-19 17:00:15 -05:00
Michael Schurter 8a0cf66822 Improve invalid port error message for services 
Related to #3681

If a user specifies an invalid port *label* when using
address_mode=driver they'll get an error message about the label being
an invalid number which is very confusing.

I also added a bunch of testing around Service.AddressMode validation
since I was concerned by the linked issue that there were cases I was
missing. Unfortunately when address_mode=driver is used there's only so
much validation that can be done as structs/structs.go validation never
peeks into the driver config which would be needed to verify the port
labels/map.
 2018-01-18 15:35:24 -08:00
Michael Schurter 447dc5bbd3 Fix test 2018-01-18 15:35:24 -08:00
Michael Schurter 583e17fad5 Always advertise driver IP when in driver mode 
Fixes #3681

When in drive address mode Nomad should always advertise the driver's IP
in Consul even when no network exists. This matches the 0.6 behavior.

When in host address mode Nomad advertises the alloc's network's IP if
one exists. Otherwise it lets Consul determine the IP.

I also added some much needed logging around Docker's network discovery.
 2018-01-18 15:35:24 -08:00
Kyle Havlovitz 8d41f4ad40 Formatting/test adjustments 2018-01-18 15:03:35 -08:00
Kyle Havlovitz 12ff22ea70 Merge branch 'master' into autopilot 2018-01-18 13:29:25 -08:00
Chelsea Holland Komlo 35466a331a fixing up raft reload tests 
close second goroutine in raft-net
 2018-01-17 10:29:15 -05:00
Michael Schurter 57eb128dcf
Merge pull request #3718 from hashicorp/b-3713-fix-check-restart 
Fix service.check_restart stanza propagation
 2018-01-16 16:39:42 -08:00
Kyle Havlovitz 7b980c42d8 Add raft remove by id endpoint/command 2018-01-16 13:35:32 -08:00
Chelsea Holland Komlo 6c9f9c8ac3 adding additional test assertions; differentiate reloading agent and http server 2018-01-16 07:34:39 -05:00
Alex Dadgar 54124a8478 Test listener uses freeport instead of static ports 2018-01-12 15:10:26 -08:00
Michael Schurter 9f179e9fab Fix HTTP code for permission denied errors 
Fixes #3697

The existing code and test case only covered the leader behavior. When
querying against non-leaders the error has an "rpc error: " prefix.

To provide consistency in HTTP error response I also strip the "rpc
error: " prefix for 403 responses as they offer no beneficial additional
information (and in theory disclose a tiny bit of data to unauthorized
users, but it would be a pretty weird bit of data to use in a malicious
way).
 2018-01-09 15:25:53 -08:00
Michael Schurter 7c282f174b Fix service.check_restart stanza propagation 
There was a bug in jobspec parsing, a bug in CheckRestart merging, and a
bug in CheckRestart canonicalization. All are now tested.
 2018-01-09 15:15:36 -08:00
Chelsea Holland Komlo 214d128eb9 reload raft transport layer 
fix up linting
 2018-01-08 14:52:28 -05:00
Chelsea Holland Komlo 0708d34135 call reload on agent, client, and server separately 2018-01-08 09:56:31 -05:00
Chelsea Holland Komlo d9ec538d6a don't ignore error in http reloading 
code review feedback
 2018-01-08 09:21:06 -05:00
Chelsea Holland Komlo 6a2432659a code review fixups 2018-01-08 09:21:06 -05:00
Chelsea Holland Komlo 4e0dbd23cf prevent races when reloading, fully shut down raft 2018-01-08 09:21:06 -05:00
Chelsea Holland Komlo ae7fc4695e fixups from code review 
Revert "close raft long-lived connections"

This reverts commit 3ffda28206fcb3d63ad117fd1d27ae6f832b6625.

reload raft connections on changing tls
 2018-01-08 09:21:06 -05:00
Chelsea Holland Komlo c0ad9a4627 add ability to upgrade/downgrade nomad agents tls configurations via sighup 2018-01-08 09:21:06 -05:00
Preetha 1712b03705
Merge branch 'master' into 0.8 2018-01-03 16:06:38 -06:00
Alex Dadgar bfc62ae41c bump version and remove generated structs 2017-12-19 17:10:52 -08:00
Alex Dadgar f0127afd93 generated files 2017-12-19 16:57:34 -08:00
Michael Schurter 714eb0b266 Services should not require a port 
Fixes #3673
 2017-12-19 15:50:23 -08:00
Kyle Havlovitz 1c07066064 Add autopilot functionality based on Consul's autopilot 2017-12-18 14:29:41 -08:00
Kyle Havlovitz b775fc7b33
Added support for v2 raft APIs and -raft-protocol option 2017-12-12 10:17:16 -06:00
Alex Dadgar d61ade8f02 remove generated structs 2017-12-11 17:51:41 -08:00
Alex Dadgar 8e63d545c4 generated assets 2017-12-11 17:30:37 -08:00
Michael Schurter cdcefd0908 Use the Service.Hash() method in agent service ids 
The allocID and taskName parameters are useless for agents, but it's
still nice to reuse the same hash method for agent and task services.
This brings in the lowercase mode for the agent hash as well.
 2017-12-11 16:50:15 -08:00
Michael Schurter 4f1002c1a8 Be more defensive in port checks 2017-12-08 12:27:57 -08:00
Michael Schurter d613e0aaf5 Move service hash logic to Service.Hash method 2017-12-08 12:03:43 -08:00
Michael Schurter b71edf846f Hash fields used in task service IDs 
Fixes #3620

Previously we concatenated tags into task service IDs. This could break
deregistration of tag names that contained double //s like some Fabio
tags.

This change breaks service ID backward compatibility so on upgrade all
users services and checks will be removed and re-added with new IDs.

This change has the side effect of including all service fields in the
ID's hash, so we no longer have to track PortLabel and AddressMode
changes independently.
 2017-12-08 12:03:43 -08:00
Michael Schurter 91282315d1 Prevent using port 0 with address_mode=driver 2017-12-08 12:03:43 -08:00
Michael Schurter 4b20441eef Validate port label for host address mode 
Also skip getting an address for script checks which don't use them.

Fixed a weird invalid reserved port in a TaskRunner test helper as well
as a problem with our mock Alloc/Job. Hopefully the latter doesn't cause
other tests to fail, but we were referencing an invalid PortLabel and
just not catching it before.
 2017-12-08 12:03:43 -08:00
Michael Schurter 4347026f83 Test Consul from TaskRunner thoroughly 
Rely less on the mockConsulServiceClient because the real
consul.ServiceClient needs all the testing it can get!
 2017-12-08 12:03:00 -08:00
Michael Schurter 4ae115dc59 Allow custom ports for services and checks 
Fixes #3380

Adds address_mode to checks (but no auto) and allows services and checks
to set literal port numbers when using address_mode=driver.

This allows SDNs, overlays, etc to advertise internal and host addresses
as well as do checks against either.
 2017-12-08 12:03:00 -08:00
Michael Schurter 1dd5b3822c
Merge pull request #3608 from hashicorp/b-3342-windows-log-leak 
Fix bug in log framer only affecting Windows
 2017-12-08 10:59:26 -08:00
Chelsea Holland Komlo 2ea8e43214 code review fixups 2017-12-06 16:37:47 -05:00
Chelsea Holland Komlo a010db084b fix up basic test 
add conversion for KillSignal for api/struct representation of task
 2017-12-06 14:36:45 -05:00
Michael Schurter b66aa5b7f6
Merge pull request #3563 from hashicorp/b-snapshot-atomic 
Atomic Snapshotting / Sticky Volume Migration
 2017-12-05 09:16:33 -08:00
Alex Dadgar 0bec137561
Merge pull request #3555 from PagerDuty/fix-loop-on-sigpipe 
Do not emit logs on SIGPIPE since logging service could be unavailable
 2017-12-04 14:11:05 -08:00
Alex Dadgar ab67a98c13 Emit hostname as a label 2017-12-04 10:42:31 -08:00
Jens Herrmann 5680fcccc2 Fix typos in metric names. #3610 2017-12-01 15:24:14 +01:00
Michael Schurter 2cbde16b9b Add check for Windows ECONNRESET 2017-11-30 21:30:20 -08:00
Michael Schurter 3e8e3aac70 Add defensive check to safeguard from future #3342s 
I hate adding "this should never happen" checks, but causing a tight
loop that OOMs Nomad is just too easy in this code otherwise.
 2017-11-30 20:37:13 -08:00
Michael Schurter 29d86eb348 Fix race in framer and improperly returned err 
Fixes #3342

Two bugs were fixed:

* Closing the StreamFramer's exitCh before setting the error means other
  goroutines blocked on exitCh closing could see the error as nil. This
  was *not* observered.
* parseFramerError on Windows would fall through and return an
  improperly captured nil err variable. There's no need for
  parseFramerError to be a closure which fixes the confusion.
 2017-11-30 17:42:53 -08:00
Michael Schurter 5e975bbd0f Add comment and normalize err check ordering 
as per PR comments
 2017-11-29 17:26:11 -08:00
Michael Schurter d996c3a231 Check for error file when receiving snapshots 2017-11-29 17:26:11 -08:00
Michael Schurter ca946679f6 Destroy partially migrated alloc dirs 
Test that snapshot errors don't return a valid tar currently fails.
 2017-11-29 17:26:11 -08:00
Michael Lange 96403746b1 Add CORS headers to client fs endpoints 2017-11-21 11:22:42 -08:00
Preetha Appan 3592635ede Populate DisplayMessage in various http endpoints that return allocations, plus unit tests. 2017-11-17 14:53:26 -06:00
Alex Dadgar 05b1588cea Only publish metric when the task is running and dev mode publishes metrics 2017-11-15 13:21:06 -08:00
Max Timchenko 8b7e61d055 Do not emit logs on SIGPIPE since logging service could be unavailable 
This should fix https://github.com/hashicorp/nomad/issues/3554
 2017-11-15 18:01:41 +02:00
Chelsea Komlo 2dfda33703 Nomad agent reload TLS configuration on SIGHUP (#3479) 
* Allow server TLS configuration to be reloaded via SIGHUP

* dynamic tls reloading for nomad agents

* code cleanup and refactoring

* ensure keyloader is initialized, add comments

* allow downgrading from TLS

* initalize keyloader if necessary

* integration test for tls reload

* fix up test to assert success on reloaded TLS configuration

* failure in loading a new TLS config should remain at current

Reload only the config if agent is already using TLS

* reload agent configuration before specific server/client

lock keyloader before loading/caching a new certificate

* introduce a get-or-set method for keyloader

* fixups from code review

* fix up linting errors

* fixups from code review

* add lock for config updates; improve copy of tls config

* GetCertificate only reloads certificates dynamically for the server

* config updates/copies should be on agent

* improve http integration test

* simplify agent reloading storing a local copy of config

* reuse the same keyloader when reloading

* Test that server and client get reloaded but keep keyloader

* Keyloader exposes GetClientCertificate as well for outgoing connections

* Fix spelling

* correct changelog style
 2017-11-14 17:53:23 -08:00
Michael Lange eb42e6d219 generated UI routes 2017-11-10 13:29:17 -08:00
Michael Lange 157abf2c76 Remove the connect-src self restriction for the UI 2017-11-10 13:28:11 -08:00
Alex Dadgar 8324ef3bc4 format 2017-11-10 12:47:59 -08:00
James Powis 4bf9e6a3ab [3503] adding consul-address flag to command.go 2017-11-04 18:05:20 -06:00
Alex Dadgar 701f462d33 remove atlas 2017-11-02 11:27:21 -07:00
Preetha 2f67e839c1
Merge pull request #3484 from hashicorp/b-nomad-0.7.1 
merge nomad 0.7.1 branch
 2017-11-01 16:50:37 -05:00
Chelsea Holland Komlo afe9f9a714 add rpc_upgrade_mode as config option for tls upgrades 2017-11-01 15:19:52 -05:00
Michael Schurter 1769db98b7 Fix regression by returning error on unknown alloc 2017-11-01 15:16:38 -05:00
Michael Schurter 2a81160dcd Fix GC'd alloc tracking 
The Client.allocs map now contains all AllocRunners again, not just
un-GC'd AllocRunners. Client.allocs is only pruned when the server GCs
allocs.

Also stops logging "marked for GC" twice.
 2017-11-01 15:16:38 -05:00
Diptanu Choudhury 46bc4280b2 Adding support for tagged metrics 2017-11-01 13:15:06 -07:00
Diptanu Choudhury 524a1f0712 Publishing metrics for job summary 2017-11-01 13:15:06 -07:00
Alex Dadgar 51c87ec858 bump version and remove generated 2017-11-01 10:02:25 -07:00
Alex Dadgar 11c24e90a1 generated 2017-11-01 09:42:18 -07:00
Alex Dadgar 51f869040f remove generated structs 2017-10-31 13:50:16 -07:00
Alex Dadgar a539483a35 changelog + PUT 2017-10-27 10:43:57 -07:00
Diptanu Choudhury 9b18737d15 Added the purge API on node endpoints 2017-10-25 23:51:53 -07:00
Alex Dadgar 68c0d8a77a add static ui assets 2017-10-25 17:01:26 -07:00
Alex Dadgar 74e612fe1e metrics test 2017-10-23 18:38:36 -07:00
Alex Dadgar cb0d0ef009 move to consul freeport implementation 2017-10-23 16:51:40 -07:00
Alex Dadgar dbc014b360 Standardize retrieving a free port into a helper package 2017-10-23 16:48:20 -07:00
Alex Dadgar 2d43e527d1 fix namespaces 2017-10-23 16:07:38 -07:00
Alex Dadgar fa55f3bc21 Fix on ipv6 2017-10-19 16:49:58 -07:00
Alex Dadgar e7299676f6 generated 2017-10-19 15:20:39 -07:00
Michael Schurter c9a73ac76e Support CORS for client endpoints 
Added to /v1/client/stats and /v1/client/allocation/
 2017-10-18 17:32:36 -07:00
Michael Schurter 0aace3d749 Don't set Interval on TTL health checks 2017-10-16 17:35:47 -07:00
Michael Schurter 916682f8a0 Merge pull request #3387 from hashicorp/f-aclless-health 
Agent Health Endpoint
 2017-10-16 11:05:09 -07:00
Alex Dadgar c3f06b2134 Merge pull request #3384 from hashicorp/f-self-policies 
Ability to introspect self token
 2017-10-13 17:11:22 -07:00
Michael Schurter 9d3f5a043e Better client health check and error handling 2017-10-13 16:25:25 -07:00
Michael Schurter 799a928812 Merge pull request #3388 from hashicorp/b-migration-fixes 
Test fixes from #3383
 2017-10-13 16:03:12 -07:00
Michael Schurter b5fd075d74 Test fixes from #3383 2017-10-13 15:45:35 -07:00
Michael Schurter c53aac9eea Agent Health Endpoint 2017-10-13 15:37:44 -07:00
Alex Dadgar c1cc51dbee sync 2017-10-13 14:36:02 -07:00
Alex Dadgar dd3920d681 Self token lookup 2017-10-13 13:12:20 -07:00
Alex Dadgar 44927ca449 remove bad test 2017-10-13 12:40:21 -07:00
Michael Schurter 15b3df0b80 Merge pull request #3374 from hashicorp/f-auth-token 
SecretID -> AuthToken
 2017-10-12 16:57:49 -07:00
Alex Dadgar d6b970eec9 Handle invalid token as well 2017-10-12 15:39:05 -07:00
Michael Schurter 84d8a51be1 SecretID -> AuthToken 2017-10-12 15:16:33 -07:00
Alex Dadgar 0b538ded83 403 instead of 500 for permission denied 2017-10-12 14:10:20 -07:00
Alex Dadgar b5fc557253 ACL command options 2017-10-12 13:51:39 -07:00
Alex Dadgar 53f2ea88a5 Small fixes 
This commit:

* Fixes the error checking in migration tests now that we are using the
canonical ErrPermissionDenied error
* Guard against NPE when looking up objects to generate the migration
token
* Handle an additional case in ShouldMigrate()
 2017-10-11 17:13:50 -07:00
Chelsea Holland Komlo c67bfc2ee4 fixups from code review 
change creation of a migrate token to be for a previous allocation
 2017-10-11 17:13:50 -07:00
Chelsea Holland Komlo e1c4701a43 fix up build warnings 2017-10-11 17:11:57 -07:00
Chelsea Holland Komlo 00a1f5fc9a adding migration token validation for gc endpoint 2017-10-11 17:11:57 -07:00
Chelsea Holland Komlo b018ca4d46 fixing up code review comments 2017-10-11 17:09:20 -07:00
Chelsea Holland Komlo 1b90a2eef0 adding valid case test for http endpoint 2017-10-11 17:09:20 -07:00
Chelsea Holland Komlo a77e462465 add tests for functionality 2017-10-11 17:09:20 -07:00
Chelsea Holland Komlo 410adaf726 Add functionality for authenticated volumes 2017-10-11 17:09:20 -07:00
Michael Schurter de767ffa04 Status.Members ACL enforcement 
Was incorrectly checked on the HTTP API before. Moved to RPC endpoint.
 2017-10-10 10:36:54 -07:00
Michael Schurter 180cbc4f5a Fix AgentSelf, AgentMembers, and KeyringOperations 2017-10-09 16:12:28 -07:00
Michael Schurter e50acae1a9 ForceLeave endpoint must use Server.ResolveToken 
The ForceLeaveRequest endpoint may only be called on servers, but the
code was using a Client to resolve tokens. This would cause a panic when
an agent wasn't both a Server and a Client.
 2017-10-09 15:49:04 -07:00
Michael Schurter 492c861419 /v1/client/agent/* ACL enforcement 2017-10-09 12:18:54 -07:00
Michael Schurter 04adc987ed /v1/client/allocation/./{stats,gc} ACL enforcement 2017-10-09 12:13:52 -07:00
Michael Schurter 84d82d79bb /v1/client/stats ACL enforcement 2017-10-09 12:02:32 -07:00
Michael Schurter 8a125759ae Refactor as there's no need for an ACL helper 2017-10-09 11:19:25 -07:00
Michael Schurter 260b66bf52 Test all fs endpoints in a loop 2017-10-09 11:19:14 -07:00
Michael Schurter a7450c8633 FS HTTP API ACL enforcement 
ACL enforcement for the filesystem HTTP APIs on clients.
 2017-10-09 11:06:34 -07:00
Michael Schurter 69bcfe78ba /v1/client/gc ACL enforcement 2017-10-06 14:50:16 -07:00
Michael Schurter fd9d09e290 Rename TestAgent.Token to TestAgent.RootToken 2017-10-06 14:35:14 -07:00
James Rasell 4628637b54
Update agent CLI to include Consul config option flags. 
This update introduces command line flags for all Consul options
thus allowing users to both use a config file and CLI options to
configure Consul.

Website documentation has also been updated.

Closes #3304
 2017-10-05 14:26:26 +01:00
Alex Dadgar 4173834231 Enable more linters 2017-09-26 15:26:33 -07:00
Alex Dadgar fb1ceabad6 generated code 2017-09-19 12:17:16 -05:00
Alex Dadgar f16167b5e1 always gzip 2017-09-19 10:37:49 -05:00
Alex Dadgar e5ec915ac3 sync 2017-09-19 10:08:23 -05:00
Michael Schurter a844fba8d2 Fix comments: task -> check 2017-09-15 15:19:53 -07:00
Michael Schurter bd3f517f2f Test converting CheckRestart from api->structs 2017-09-15 15:01:56 -07:00
Michael Schurter 0f2a3dcec9 Test check watch updates 2017-09-14 16:48:39 -07:00
Michael Schurter 847fe080f6 Rename unhealthy var and fix test indeterminism 2017-09-14 16:48:39 -07:00
Michael Schurter 573a0df03d Watched -> TriggersRestart 
Watched was a silly name
 2017-09-14 16:48:39 -07:00
Michael Schurter 4ea19baa52 Handle multiple failing checks on a single task 
Before this commit if a task had 2 checks cause restarts at the same
time, both would trigger restarts of the task! This change removes all
checks for a task whenever one of them is restarted.
 2017-09-14 16:48:39 -07:00
Michael Schurter 73fb71ca10 RestartDelay isn't needed as checks are re-added on restarts 
@dadgar made the excellent observation in #3105 that TaskRunner removes
and re-registers checks on restarts. This means checkWatcher doesn't
need to do *any* internal restart tracking. Individual checks can just
remove themselves and be re-added when the task restarts.
 2017-09-14 16:48:39 -07:00
Michael Schurter 448ad3945f Simplify from 2 select loops to one 2017-09-14 16:48:39 -07:00
Michael Schurter 550e631eea Wrap check watch updates in a struct 
Reusing checkRestart for both adds/removes and the main check restarting
logic was confusing.
 2017-09-14 16:48:39 -07:00
Michael Schurter d299d42089 Canonicalize and Merge CheckRestart in api 2017-09-14 16:48:39 -07:00
Michael Schurter 72e5c0c0aa Fix whitespace 2017-09-14 16:47:41 -07:00
Michael Schurter ade29ecbed Improve check watcher logging and add tests 
Also expose a mock Consul Agent to allow testing ServiceClient and
checkWatcher from TaskRunner without actually talking to a real Consul.
 2017-09-14 16:47:41 -07:00
Michael Schurter 99f4aa999a Default grace period to 1s 2017-09-14 16:46:54 -07:00
Michael Schurter a137676358 Add comments and move delay calc to TaskRunner 2017-09-14 16:46:54 -07:00
Michael Schurter a180c00fc3 on_warning=false -> ignore_warnings=false 
Treat warnings as unhealthy by default
 2017-09-14 16:46:54 -07:00
Michael Schurter 8a87475498 Use existing restart policy infrastructure 2017-09-14 16:46:54 -07:00
Michael Schurter 22690c5f4c Add check watcher for restarting unhealthy tasks 2017-09-14 16:46:54 -07:00
Michael Schurter b35d208428 Nest restart fields in CheckRestart 2017-09-14 16:46:54 -07:00
Michael Schurter bf34505509 Add restart fields 2017-09-14 16:46:54 -07:00
Alex Dadgar 01180fec58 use assert 2017-09-14 14:20:22 -07:00
Alex Dadgar c55b7ce4d6 Sort /v1/agent/servers output 
This PR sorts the output of the endpoint since its results are used as
part of Consul checks to avoid the value changing unnecessarily.

Fixes https://github.com/hashicorp/nomad/issues/3211
 2017-09-14 14:20:22 -07:00
Chelsea Holland Komlo c014cb014f fix bug to return prometheus sink 2017-09-13 19:21:21 +00:00
Chelsea Holland Komlo fcc3071e52 fixups from code review 2017-09-13 19:21:21 +00:00
Chelsea Holland Komlo faabf61c6d updated documentation 2017-09-13 19:21:21 +00:00
Chelsea Holland Komlo f5975dceb7 refactoring prometheus endpoint 2017-09-13 19:21:21 +00:00
Chelsea Holland Komlo 035fab0c8d enabling prometheus metrics should be a config option 2017-09-13 19:21:21 +00:00
Chelsea Holland Komlo d8e9f2fef7 add endpoint for prometheus 2017-09-13 19:21:21 +00:00
Alex Dadgar 5aa8f1a82e pass in uid to codecgen 2017-09-11 15:40:27 -07:00
Armon Dadgar c234e09d9e website: document ACL bootstrap reset process 2017-09-10 16:18:39 -07:00
Alex Dadgar 84d06f6abe Sync namespace changes 2017-09-07 17:04:21 -07:00
Chelsea Holland Komlo 1238efc2a8 improve documentation 
move metrics to telemetry; copy to client config
 2017-09-06 21:38:06 +00:00
Chelsea Holland Komlo 66fa05405a tagged metrics config options should be on telemetry config 
better api example, add telemetry documentation
 2017-09-06 15:25:36 +00:00
Chelsea Holland Komlo 751fc5324e add http endpoint for in memory metrics 
prevent against flaky test due to timing/initialization issues
 2017-09-06 13:51:19 +00:00
Chelsea Holland Komlo 0ef43c3c5f final code review fixups 2017-09-05 18:47:44 +00:00
Chelsea Holland Komlo a8cbd0b559 fixups from code review 2017-09-05 14:13:34 +00:00
Chelsea Holland Komlo b4d21869ca update comments 2017-09-05 14:13:34 +00:00
Chelsea Holland Komlo ba4abbe09c remove prints during test 2017-09-05 14:13:34 +00:00
Chelsea Holland Komlo 5c10a9325e parse config for metrics fields 2017-09-05 14:13:34 +00:00
Armon Dadgar 0dabcb8659 agent: fix routing for token-specific request 2017-09-04 13:07:44 -07:00
Armon Dadgar af9caef4b9 Adding a comment to address @dadgar 2017-09-04 13:05:53 -07:00
Armon Dadgar f1c4143a62 agent: thread ACLs for Job Register 2017-09-04 13:05:53 -07:00
Armon Dadgar 5c94e7e99f agent: thread through token for ACL endpoint tests 2017-09-04 13:05:53 -07:00
Armon Dadgar 3e46094cee Passthrough replication token for token/policy replication 2017-09-04 13:05:53 -07:00
Armon Dadgar 4107335cb2 agent: Adding X-Nomad-Token header parsing 2017-09-04 13:05:53 -07:00
Armon Dadgar 4bda2fa9e9 nomad: ACL endpoints check support enabled and redirect to authority 2017-09-04 13:05:53 -07:00
Armon Dadgar 866fe5e216 nomad: adding ACL bootstrapping endpoint 2017-09-04 13:05:53 -07:00
Armon Dadgar 792f176a44 agent: thread ACL config to client 2017-09-04 13:04:45 -07:00
Armon Dadgar 3efdf1f7d9 Address @chelseakomlo comments 2017-09-04 13:04:45 -07:00
Armon Dadgar 018973aea8 Address @dadgar feedback 2017-09-04 13:04:45 -07:00
Armon Dadgar 5b3d6c0a52 Fixing intent 2017-09-04 13:04:45 -07:00
Armon Dadgar 7d4aa1975f agent: thread through ACL config to Server 2017-09-04 13:04:45 -07:00
Armon Dadgar 9cd8ac832f agent: Adding ACL block configuration 2017-09-04 13:04:45 -07:00
Armon Dadgar a45bd0d2a2 agent: fixing ACL endpoint, return updated tokens 2017-09-04 13:04:45 -07:00
Armon Dadgar bd2db18c80 agent: Adding HTTP endpoints for ACL tokens 2017-09-04 13:04:45 -07:00
Armon Dadgar 18e6053b58 agent: Adding ACL Policy endpoints 2017-09-04 13:03:15 -07:00
Michael Schurter 28e9fa73f6 Welcome to the brave new world 
Committing generated code to help ensure reproducible builds.
 2017-08-28 15:51:18 -07:00
Chelsea Holland Komlo ffe95a1a62 adds any resource autocomplete 
defaults to listing jobs if no id is provided
 2017-08-25 16:42:11 +00:00
Chelsea Komlo 49542712f9 Merge pull request #3068 from hashicorp/f-add-deployments-search-api 
Add deployments to search api
 2017-08-21 16:34:00 -04:00
Chelsea Holland Komlo 50c20b1801 add deployments to search api 2017-08-21 15:20:55 +00:00
Michael Schurter 7f6e1f3a9c Initializing embedded structs is weird 2017-08-17 16:49:14 -07:00
Michael Schurter 0634eef12a Test createCheckReg 2017-08-17 16:49:14 -07:00
Michael Schurter bb8d5689d8 Add Header and Method support for HTTP checks 2017-08-17 16:44:21 -07:00
Michael Schurter b4813747d0 Merge pull request #3043 from hashicorp/f-2441-shutdown-delay 
Add optional shutdown delay to tasks
 2017-08-17 14:37:48 -07:00
Michael Schurter d529b422b2 Add optional shutdown delay to tasks 
Fixes #2441

Defaults to 0 (no delay) for backward compat and because this feature
should be opt-in.
 2017-08-16 17:59:46 -07:00
Alex Dadgar d6187cd3e8 Fix tests 2017-08-16 16:26:52 -07:00
Alex Dadgar 1a86aecf55 Add version package 
This PR adds a version package and consolidates version strings into a
Version struct.
 2017-08-16 15:44:21 -07:00
Alex Dadgar 828b9bfae4 fix http test 2017-08-16 11:01:12 -07:00
Chelsea Komlo b9f189495a Merge pull request #2995 from hashicorp/f-allocation-autocomplete 
Allocation autocomplete, client api
 2017-08-14 15:26:39 -04:00
Chelsea Holland Komlo e658d4daaa fixups from code review 2017-08-14 17:56:03 +00:00
Alex Dadgar 43dff0a11d Fix integration test 2017-08-14 10:52:49 -07:00
Chelsea Holland Komlo 69c2b7e308 context constants should be in both api and structs pkg 2017-08-14 17:38:10 +00:00
Chelsea Holland Komlo 9bc48a01a6 using contexts constants in replace of string contexts 
fix up endpoints
 2017-08-14 17:38:10 +00:00
Chelsea Holland Komlo 76827c98f7 use constants for contexts 2017-08-14 17:38:10 +00:00
Chelsea Holland Komlo 465c4d7082 change endpoint to /v1/search 2017-08-14 17:38:10 +00:00
Chelsea Holland Komlo 5ee58a391b rename to cluster search 
comment updates
 2017-08-14 17:36:14 +00:00
James Nugent ae85c548ac build: Apply goimports formatting to various files 
The following files were not formatted as the output of goimports(1)
would create, causing the goimports linter to fail:

command/agent/resources_endpoint.go
nomad/resources_endpoint.go
nomad/resources_endpoint_test.go
 2017-08-14 11:51:47 +08:00
Alex Dadgar 6e20acb503 Merge pull request #2984 from hashicorp/b-tags 
Fix alloc health with checks using interpolation
 2017-08-10 13:07:25 -07:00
Alex Dadgar c8f74ac43b Address comments 2017-08-10 13:07:08 -07:00
Alex Dadgar c8367ed01b Merge pull request #2988 from jvoorhis/redact-vault-token 
Redact Vault.Token from AgentSelf response.
 2017-08-08 11:04:48 -07:00
Jeremy Voorhis a19043e6c7 Handle nil values when redacting vault token. 2017-08-08 10:54:11 -07:00
Alex Dadgar aba107be99 Merge pull request #2979 from lfarnell/cleanup 
Code cleanup
 2017-08-08 10:21:15 -07:00
Jeremy Voorhis 554f6efe02 Redact Vault.Token from AgentSelf response. 
If Config.Vault.Token is defined, /v1/agent/self will return the string
`<redacted>`. If the token is not set, This endpoint will continue to
return the empty string.
 2017-08-07 21:34:43 -07:00
Alex Dadgar 79d25b7db9 Merge pull request #2947 from hashicorp/f-vault-grace 
Allow template to set Vault grace
 2017-08-07 16:29:53 -07:00
Alex Dadgar d86b3977b9 Fix alloc health with checks using interpolation 
Fixes an issue in which the allocation health watcher was checking for
allocations health based on un-interpolated services and checks. Change
the interface for retrieving check information from Consul to retrieving
all registered services and checks by allocation. In the future this
will allow us to output nicer messages.

Fixes https://github.com/hashicorp/nomad/issues/2969
 2017-08-07 16:27:08 -07:00
Luke Farnell f0ced87b95 fixed all spelling mistakes for goreport 2017-08-07 17:13:05 -04:00
Chelsea Komlo 5537516bfd Merge pull request #2964 from hashicorp/f-autocomplete-api 
autocomplete api
 2017-08-07 12:40:09 -07:00
Chelsea Holland Komlo 1b77f9a216 further refactoring 2017-08-04 22:50:41 +00:00
Chelsea Holland Komlo 424e475333 if no context is specified, set maximum index for available contexts 2017-08-04 20:16:49 +00:00
Chelsea Holland Komlo 0b38704392 fix up tests to intantiate assertion test helper 
add http tests for remaining contexts
 2017-08-04 19:48:20 +00:00
Michael Schurter 5794e5ece7 Use int32 for atomic ops to avoid alignment issues 
From https://golang.org/pkg/sync/atomic/#pkg-note-BUG :

On both ARM and x86-32, it is the caller's responsibility to arrange for
64-bit alignment of 64-bit words accessed atomically. The first word in
a global variable or in an allocated struct or slice can be relied upon
to be 64-bit aligned.
 2017-08-04 10:14:16 -07:00
Chelsea Holland Komlo 111019642b resources are expected by state store to be plural 2017-08-04 15:07:34 +00:00
Chelsea Holland Komlo b2b4c5d7af add documentation 
extract magic number into variable
 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo b6f459a7a9 refactor rpc endpoint and tests 
add test for when no prefixes are matched

add test for no context at HTTP api
 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo 7b7a80d6ce resources list endpoint accepts http POST and PUT 
set the index for a resources response
 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo 312bb19e1c refactor and add error handling for invalid context type 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo ef5a62aaa1 use upsert as a test helper 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo 0026a308ec add truncation boolean to response 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo 06f10d2e5f adds evaluations 
makes context singular
 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo 92cc40f1de change resources endpoint from http get to post 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo 64a83edd9f remove resourceliststub, no need for another layer of abstraction 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo 36285a4b23 test resources endpoint will return matching prefixes 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo ecd0d85a4e adding test validation that received resources matches requested 2017-08-04 14:34:25 +00:00
Chelsea Holland Komlo 4dd6b46198 Retrieve job information for resources endpoint 
requires further refactoring and logic for more contexts
 2017-08-04 14:34:25 +00:00
Alex Dadgar 067a638478 Allow template to set Vault grace 
This PR allows a template to specify the Vault grace duration.

Fixes https://github.com/hashicorp/nomad/issues/2922
 2017-08-01 14:14:08 -07:00
James Nugent bf13a0cd90 meta: Fix goimports for command/agent/syslog.go 2017-07-30 08:56:40 -05:00
Michael Schurter d2f8fdcad5 Fix comment 2017-07-25 12:13:05 -07:00
Michael Schurter 3e6231842d Forgot to setcmdenv 
This would leak a consul agent
 2017-07-25 12:09:57 -07:00
Michael Schurter 4b83eba599 Use seen more conservatively 2017-07-24 16:48:40 -07:00
Michael Schurter cdf138eb27 Always increment failures... 
...as it's used in calculating the backoff
 2017-07-24 15:37:53 -07:00
Michael Schurter 809724ad8d Track whether Consul has ever been seen 
Need a way to squelch Consul operation errors on shutdown. If it's never
been seen don't log errors about deregs failing.
 2017-07-24 12:12:02 -07:00
Michael Schurter edbe62a879 Synchronously deregister agent on shutdown 
Fixes #2891

Previously the agent services and checks were being asynchrously
deregistered on shutdown, so it was a race between the sync goroutine
deregistering them and Nomad shutting down.

This switches to synchronously deregister agent serivces and checks
which doesn't really have a downside since the sync goroutines retry
behavior doesn't help on shutdown anyway.
 2017-07-24 11:40:37 -07:00
Alex Dadgar 553bc91725 Parallel client tests (#2890) 
* alloc_runner

* Random tests

* parallel task_runner and no exec compatible check

* Parallel client

* Fail fast and use random ports

* Fix docker port mapping

* Make concurrent pull less timing dependant

* up parallel

* Fixes

* don't build chroots in parallel on travis

* Reduce parallelism on travis with lxc/rkt

* make java test app not run forever

* drop parallelism a little

* use docker ports that are out of the os's ephemeral port range

* Limit even more on travis

* rkt deadline
 2017-07-22 19:04:36 -07:00
Alex Dadgar 4dd5d943c7 remove root requirement on consul integration check 2017-07-21 19:32:41 -07:00
Alex Dadgar 56f9cf86df Speed up client startup 2017-07-20 22:34:24 -07:00
Alex Dadgar c106df9215 Switch to in-process agent 2017-07-20 21:07:32 -07:00
Alex Dadgar d019eed363 Merge pull request #2874 from hashicorp/f-command-agent-tests 
Parallelize the command/agent tests and add new test agent
 2017-07-20 20:27:49 -07:00
Alex Dadgar 5df9be0ccb Fix bootstrapping and waiting 2017-07-20 20:15:37 -07:00
Alex Dadgar 4e90d56098 More parallel 2017-07-20 09:36:34 -07:00
Alex Dadgar 9037693436 New test agent 2017-07-19 22:14:36 -07:00
Alex Dadgar 9a2a5af608 Don't print atlas 2017-07-19 20:25:06 -07:00
Alex Dadgar 18c7043a30 Merge pull request #2866 from hashicorp/f-autocomplete-agent 
Agent command autocompletes to hcl/json files
 2017-07-19 13:18:32 -07:00
Michael Schurter c9e4c041b3 Too lazy to remember the right formatter for floats 2017-07-19 11:53:18 -07:00
Alex Dadgar b4b50b636f Fix predictor 2017-07-19 11:51:01 -07:00
Alex Dadgar 871cdcb932 Agent command autocompletes to hcl/json files 2017-07-19 11:28:16 -07:00
Michael Schurter 40c2d4e5eb Merge pull request #2858 from hashicorp/b-2849-deploy-json 
Implement -json for job deployments
 2017-07-19 10:15:01 -07:00
Michael Schurter 125a3fb2f9 Error -> Errof 2017-07-19 10:00:57 -07:00
Alex Dadgar 747d67eb3f Allow tuning of heartbeat ttls 
This PR allows tuning of heartbeat TTLs. An example of very aggressive
settings is as follows:

```
server {
  heartbeat_grace = "1s"
  min_heartbeat_ttl = "1s"
  max_heartbeats_per_second = 200.0
}
```
 2017-07-19 09:38:35 -07:00
Michael Schurter 99d1486f32 Never remove unknown agent services 
Fixes #2827

This is a tradeoff. The pro is that you can run separate client and
server agents on the same node and advertise both. The con is that if a
Nomad agent crashes and isn't restarted on that node in the same mode
its entry will not be cleaned up.

That con scenario seems far less likely to occur than the scenario on
the pro side, and even if we do leak an agent entry the checks will be
failing so nothing should attempt to use it.
 2017-07-18 13:23:01 -07:00
Alex Dadgar 45712c6ca3 test fixes 2017-07-07 14:11:27 -07:00
Alex Dadgar bf2dafb8e9 check id method name changed 2017-07-07 12:15:09 -07:00