Commit graph

2991 commits

Author SHA1 Message Date
Drew Bailey d9e41d2880
docs for shutdown delay
update docs, address pr comments

ensure pointer is not nil

use pointer for diff tests, set vs unset
2019-12-16 11:38:35 -05:00
Drew Bailey ae145c9a37
allow only positive shutdown delay
more explicit test case, remove select statement
2019-12-16 11:38:30 -05:00
Drew Bailey 24929776a2
shutdown delay for task groups
copy struct values

ensure groupserviceHook implements RunnerPreKillhook

run deregister first

test that shutdown times are delayed

move magic number into variable
2019-12-16 11:38:16 -05:00
Seth Hoenig 270233e23d tests: remove trace statements from nodeDrainWatcher.watch
Avoid logging in the `watch` function as much as possible, since
it is not waited on during a server shutdown. When the logger
logs after a test passes, it may or may not cause the testing
framework to panic.

More info in:
https://github.com/golang/go/issues/29388#issuecomment-453648436
2019-12-16 07:08:11 -06:00
Michael Schurter 95fd2643d7 connect: canonicalize before adding sidecar
Fixes #6853

Canonicalize jobs first before adding any sidecars. This fixes a bug
where sidecar tasks were added without interpolated names and broke
validation. Sidecar tasks must be canonicalized independently.

Also adds a group network to the mock connect job because it wasn't a
valid connect job before!
2019-12-12 20:55:56 -08:00
Seth Hoenig d45dec1ca8 tests: parallelize state store tests
It has been decided we're going to live in a many core world.
Let's take advantage of that and parallelize these state store
tests which all run in memory and are largely CPU bound.

An unscientific benchmark demonstrating the improvement:

[mp state (master)] $ go test
PASS
ok  	github.com/hashicorp/nomad/nomad/state	5.162s

[mp state (f-parallelize-state-store-tests)] $ go test
PASS
ok  	github.com/hashicorp/nomad/nomad/state	1.527s
2019-12-11 09:36:37 -06:00
Seth Hoenig f0c3dca49c tests: swap lib/freeport for tweaked helper/freeport
Copy the updated version of freeport (sdk/freeport), and tweak it for use
in Nomad tests. This means staying below port 10000 to avoid conflicts with
the lib/freeport that is still transitively used by the old version of
consul that we vendor. Also provide implementations to find ephemeral ports
of macOS and Windows environments.

Ports acquired through freeport are supposed to be returned to freeport,
which this change now also introduces. Many tests are modified to include
calls to a cleanup function for Server objects.

This should help quite a bit with some flakey tests, but not all of them.
Our port problems will not go away completely until we upgrade our vendor
version of consul. With Go modules, we'll probably do a 'replace' to swap
out other copies of freeport with the one now in 'nomad/helper/freeport'.
2019-12-09 08:37:32 -06:00
Danielle Lancashire d2075ebae9 spellcheck: Fix spelling of retrieve 2019-12-05 18:59:47 -06:00
Mahmood Ali b3e557cae3 address feedback review
apply `s/requestAuthToken/requestACLToken/g`
2019-11-26 08:39:04 -05:00
Mahmood Ali 02e20c720b acl_endpoint: permission denied for unauthenticated requests
If ACL Request is unauthenticated, we should honor the anonymous token.
This PR makes few changes:

* `GetPolicy` endpoints may return policy if anonymous policy allows it,
or return permission denied otherwise.
* `ListPolicies` returns an empty policy list, or one with anonymous
policy if one exists.

Without this PR, the we return an incomprehensible error.

Before:
```
$ curl http://localhost:4646/v1/acl/policy/doesntexist; echo
acl token lookup failed: index error: UUID must be 36 characters
$ curl http://localhost:4646/v1/acl/policies; echo
acl token lookup failed: index error: UUID must be 36 characters
```

After:
```
$ curl http://localhost:4646/v1/acl/policy/doesntexist; echo
Permission denied
$ curl http://localhost:4646/v1/acl/policies; echo
[]
```
2019-11-22 08:43:09 -05:00
Michael Schurter 4b6762511d
Merge pull request #6021 from hashicorp/f-anonymous-policy-access
api: Update policy endpoint to permit anonymous access
2019-11-20 15:33:45 -08:00
Buck Doyle 5fcc00d0f9 Add gofmt changes 2019-11-20 12:47:01 -06:00
Buck Doyle dc9c0d5ead
Add explanatory comment 2019-11-20 11:45:44 -06:00
Buck Doyle bea9837510
Remove extraneous else block 2019-11-20 11:37:45 -06:00
Buck Doyle d6a3e571bd
Remove extraneous whitespace 2019-11-20 11:37:01 -06:00
Buck Doyle db77a24ed3
Merge branch 'master' into f-policy-json 2019-11-20 11:20:07 -06:00
Mahmood Ali 7fb4c35831 comments and casing 2019-11-19 16:03:55 -05:00
Mahmood Ali 97d0fd009d 404 if token isn't found 2019-11-19 15:52:53 -05:00
Mahmood Ali 6f8bb5e90b api: acl bootstrap errors aren't 500
Noticed that ACL endpoints return 500 status code for user errors.  This
is confusing and can lead to false monitoring alerts.

Here, I introduce a concept of RPCCoded errors to be returned by RPC
that signal a code in addition to error message.  Codes for now match
HTTP codes to ease reasoning.

```
$ nomad acl bootstrap
Error bootstrapping: Unexpected response code: 500 (ACL bootstrap already done (reset index: 9))

$ nomad acl bootstrap
Error bootstrapping: Unexpected response code: 400 (ACL bootstrap already done (reset index: 9))
```
2019-11-19 15:51:57 -05:00
Michael Schurter 796758b8a5 core: add semver constraint
The existing version constraint uses logic optimized for package
managers, not schedulers, when checking prereleases:

- 1.3.0-beta1 will *not* satisfy ">= 0.6.1"
- 1.7.0-rc1 will *not* satisfy ">= 1.6.0-beta1"

This is due to package managers wishing to favor final releases over
prereleases.

In a scheduler versions more often represent the earliest release all
required features/APIs are available in a system. Whether the constraint
or the version being evaluated are prereleases has no impact on
ordering.

This commit adds a new constraint - `semver` - which will use Semver
v2.0 ordering when evaluating constraints. Given the above examples:

- 1.3.0-beta1 satisfies ">= 0.6.1" using `semver`
- 1.7.0-rc1 satisfies ">= 1.6.0-beta1" using `semver`

Since existing jobspecs may rely on the old behavior, a new constraint
was added and the implicit Consul Connect and Vault constraints were
updated to use it.
2019-11-19 08:40:19 -08:00
Nick Ethier bd454a4c6f
client: improve group service stanza interpolation and check_re… (#6586)
* client: improve group service stanza interpolation and check_restart support

Interpolation can now be done on group service stanzas. Note that some task runtime specific information
that was previously available when the service was registered poststart of a task is no longer available.

The check_restart stanza for checks defined on group services will now properly restart the allocation upon
check failures if configured.
2019-11-18 13:04:01 -05:00
Luiz Aoqui e499c5bddc
Merge pull request #6698 from hashicorp/f-add-drain-start-time
api: add `StartedAt` in `Node.DrainStrategy`
2019-11-15 15:38:38 -05:00
Luiz Aoqui e862b61daa
api: use the same initial time for all drain properties 2019-11-14 16:06:09 -05:00
Drew Bailey 9b63828658
serverID to target remote leader or server
handle the case where we request a server-id which is this current server

update docs, error on node and server id params

more accurate names for tests

use shared no leader err, formatting

rm bad comment

remove redundant variable
2019-11-14 10:07:35 -05:00
Drew Bailey b644e1f47d
add server-id to monitor specific server 2019-11-14 09:53:41 -05:00
Drew Bailey 2185c1a89e
Allows monitor to target leader server
Allows user to pass in node-id=leader to forward monitor request to
remote a remote leader.
2019-11-14 09:53:40 -05:00
Luiz Aoqui 5bd7cdd5c3
api: add StartedAt in Node.DrainStrategy 2019-11-13 17:54:40 -05:00
Lars Lehtonen 22a3c21dd0
nomad: fix dropped test error 2019-11-13 12:49:41 -08:00
Michael Schurter 08afb7d605 vault: allow overriding implicit vault constraint
There's a bug in version parsing that breaks this constraint when using
a prerelease enterprise version of Vault (eg 1.3.0-beta1+ent). While
this does not fix the underlying bug it does provide a workaround for
future issues related to the implicit constraint. Like the implicit
Connect constraint: *all* implicit constraints should be overridable to
allow users to workaround bugs or other factors should the need arise.
2019-11-12 12:26:36 -08:00
Mahmood Ali c4c37cb42e vault: check token_explicit_max_ttl as well
Vault 1.2.0 deprecated `explicit_max_ttl` in favor of
`token_explicit_max_ttl`.
2019-11-12 08:47:23 -05:00
Lars Lehtonen adbab29228 nomad: TestEvalBroker_Dequeue_Empty_Timeout() proper goroutine error handling (#6657) 2019-11-08 14:35:06 -05:00
Drew Bailey 7420446458
Merge pull request #6639 from hashicorp/return-after-forward
return after request has been forwarded
2019-11-08 09:48:35 -05:00
Lars Lehtonen 39b68e0b88 TestEvalBroker_Dequeue_Blocked() proper goroutine error handling (#6651)
TestEvalBroker_Dequeue_Blocked() improve test readability
2019-11-08 08:52:23 -05:00
Nick Ethier e947aaed4f
nomad: fix bug that didn't allow for multiple connect services in same tg 2019-11-08 04:33:39 -05:00
Lars Lehtonen 6deae70e35 TestEvalBroker_PauseResumeNackTimeout() proper goroutine error handling (#6649)
TestEvalBroker_PauseResumeNackTimeout() improve test readability
2019-11-07 16:04:59 -05:00
Lars Lehtonen 2638cbb31d nomad: TestEvalBroker_EnqueueAll_Dequeue_Fair() proper goroutine error handling (#6636)
nomad: TestEvalBroker_EnqueueAll_Dequeue_Fair() improve test readability
2019-11-07 10:39:29 -05:00
Drew Bailey a5e2e1805f
return after request has been forwarded 2019-11-07 08:33:53 -05:00
Lars Lehtonen e64f98837c nomad: fix dropped error in TestJobEndpoint_Deregister_ACL (#6602) 2019-11-06 16:40:45 -05:00
Drew Bailey f4a7e3dc75
coordinate closing of doneCh, use interface to simplify callers
comments
2019-11-05 11:44:26 -05:00
Drew Bailey fe542680dc
log-json -> json
fix typo command/agent/monitor/monitor.go

Co-Authored-By: Chris Baker <1675087+cgbaker@users.noreply.github.com>

Update command/agent/monitor/monitor.go

Co-Authored-By: Chris Baker <1675087+cgbaker@users.noreply.github.com>

address feedback, lock to prevent send on closed channel

fix lock/unlock for dropped messages
2019-11-05 09:51:59 -05:00
Drew Bailey 298b8358a9
move forwarded monitor request into helper 2019-11-05 09:51:56 -05:00
Drew Bailey 8726b685de
address feedback 2019-11-05 09:51:56 -05:00
Drew Bailey 0e759c401c
moving endpoints over to frames 2019-11-05 09:51:54 -05:00
Drew Bailey 17d876d5ef
rename function, initialize log level better
underscores instead of dashes for query params
2019-11-05 09:51:53 -05:00
Drew Bailey 8178beecf0
address feedback, use agent_endpoint instead of monitor 2019-11-05 09:51:53 -05:00
Drew Bailey db65b1f4a5
agent:read acl policy for monitor 2019-11-05 09:51:52 -05:00
Drew Bailey 2533617888
rpc acl tests for both monitor endpoints 2019-11-05 09:51:51 -05:00
Drew Bailey 3c33747e1f
client monitor endpoint tests 2019-11-05 09:51:50 -05:00
Drew Bailey 4bc68855d0
use intercepting loggers for rpchandlers 2019-11-05 09:51:50 -05:00
Drew Bailey 3b9c33a5f0
new hclog with standardlogger intercept 2019-11-05 09:51:49 -05:00