Commit graph

16933 commits

Author SHA1 Message Date
Tim Gross 1e600d573d
e2e: improve reusability of provisioning scripts (#6942)
This changeset is part of the work to improve our E2E provisioning
process to allow our upgrade tests:

* Move more of the setup into the AMI image creation so it's a little
 more obvious to provisioning config authors which bits are essential
 to deploying a specific version of Nomad.

* Make the service file update do a systemd daemon-reload so that we
  can update an already-running cluster with the same script we use to
  deploy it initially.
2020-01-16 09:29:36 -05:00
Mahmood Ali f36cc54efd actually always canonicalize alloc.Job
alloc.Job may be stale as well and need to migrate it.  It does cost
extra cycles but should be negligible.
2020-01-15 09:02:48 -05:00
Mahmood Ali b1b714691c address review comments 2020-01-15 08:57:05 -05:00
Michael Schurter ffbfb60f40 test: restore e2e-test target and use -integration 2020-01-14 13:47:51 -08:00
Michael Schurter da4645e9a4 test: download Vault binaries for e2e test
Modernize Vault integration/e2e test a bit:

- Download from releases.hashicorp.com instead of using a hardcoded list
- Remove old unused make target e2e-test
- Use NOMAD_E2E env var instead of -integration flag
- Add a README

On my machine with ~250 Mbps internet it takes ~400s to download all
Vault binaries.
2020-01-14 11:02:02 -08:00
Charlie Voiselle a4aeea0058
Upgrade -> Update 2020-01-13 17:36:49 -05:00
Danielle 5fd52171aa
cli: add system command and subcmds to interact with system API. (#6924)
cli: add system command and subcmds to interact with system API.
2020-01-13 16:16:08 +01:00
James Rasell 09519bb03f
system CLI documentation fixes based on feedback from @angrycub 2020-01-13 15:45:14 +01:00
Mahmood Ali 1ab682f622 scheduler: allow configuring default preemption for system scheduler
Some operators want a greater control over when preemption is enabled,
especially during an upgrade to limit potential side-effects.
2020-01-13 08:30:49 -05:00
James Rasell c452f32414
docs: add documentation for system command. 2020-01-13 11:35:24 +01:00
James Rasell 4e48217a4e
cli: add system command and subcmds to interact with system API.
The system command includes gc and reconcile-summaries subcommands
which covers all currently available system API calls. The help
information is largely pulled from the current Nomad website API
documentation.
2020-01-13 11:34:46 +01:00
Drew Bailey ff4bfb8809
Merge pull request #6841 from hashicorp/f-agent-pprof-acl
Remote agent pprof endpoints
2020-01-10 14:52:39 -05:00
Tim Gross 6ad9cc186d
dev: create directories for cluster script (#6927) 2020-01-10 14:25:57 -05:00
Lang Martin 4f6a8263ef GNUmakefile: report failed path on hclfmt error 2020-01-10 13:17:14 -05:00
Mahmood Ali bfa33cf471 canonicalize allocs from plan results too 2020-01-10 10:41:12 -05:00
Nick Ethier 1f28633954
Merge pull request #6816 from hashicorp/b-multiple-envoy
connect: configure envoy to support multiple sidecars in the same alloc
2020-01-09 23:25:39 -05:00
Drew Bailey f97d2e96c1
refactor api profile methods
comment why we ignore errors parsing params
2020-01-09 15:15:12 -05:00
Drew Bailey b702dede49
adds qc param, address pr feedback 2020-01-09 15:15:11 -05:00
Drew Bailey 085659f6ff
condense table test 2020-01-09 15:15:10 -05:00
Drew Bailey 45210ed901
Rename profile package to pprof
Address pr feedback, rename profile package to pprof to more accurately
describe its purpose. Adds gc param for heap lookup profiles.
2020-01-09 15:15:10 -05:00
Drew Bailey 1b8af920f3
address pr feedback 2020-01-09 15:15:09 -05:00
Drew Bailey 92469ffcb3
comments for api usage of agent profile 2020-01-09 15:15:09 -05:00
Drew Bailey 4ced73875b
leave acl checking to rpc endpoints
fix test expectation

test wrapNonJSON
2020-01-09 15:15:08 -05:00
Drew Bailey 279512c7f8
provide helpful error, cleanup logic 2020-01-09 15:15:08 -05:00
Drew Bailey 7bbba613a5
prevent doubly wrapping with rpc error 2020-01-09 15:15:07 -05:00
Drew Bailey fd42020ad6
RPC server EnableDebug option
Passes in agent enable_debug config to nomad server and client configs.
This allows for rpc endpoints to have more granular control if they
should be enabled or not in combination with ACLs.

enable debug on client test
2020-01-09 15:15:07 -05:00
Drew Bailey fc67175141
api docs for agent/pprof 2020-01-09 15:15:06 -05:00
Drew Bailey 31c0aca10a
rename forward func, add comment for why we forward 2020-01-09 15:15:06 -05:00
Drew Bailey 9a80938fb1
region forwarding; prevent recursive forwards for impossible requests
prevent region forwarding loop, backfill tests

fix failing test
2020-01-09 15:15:06 -05:00
Drew Bailey 46121fe3fd
move shared structs out of client and into nomad 2020-01-09 15:15:05 -05:00
Drew Bailey aec81a0b99
api agent endpoints
helper func to return serverPart based off of serverID
2020-01-09 15:15:05 -05:00
Drew Bailey 3672414888
test pprof headers and profile methods
tidy up, add comments

clean up seconds param assignment
2020-01-09 15:15:04 -05:00
Drew Bailey fc37448683
warn when enabled debug is on when registering
m -> a receiver name

return codederrors, fix query
2020-01-09 15:15:04 -05:00
Drew Bailey 62eb2d76a6
acl and debug test table
rename implementation method
2020-01-09 15:15:03 -05:00
Drew Bailey 50288461c9
Server request forwarding for Agent.Profile
Return rpc errors for profile requests, set up remote forwarding to
target leader or server id for profile requests.

server forwarding, endpoint tests
2020-01-09 15:15:03 -05:00
Drew Bailey 901f362858
test for known pprof endpoints 2020-01-09 15:15:02 -05:00
Drew Bailey 49ad5fbc85
agent pprof endpoints
wip, agent endpoint and client endpoint for pprof profiles

agent endpoint test
2020-01-09 15:15:02 -05:00
Tim Gross b5bcfb533b
upgrade CNI plugins to 0.8.4 (#6921)
When multiple Connect-enabled task groups start on the same client
node, a race condition in the CNI plugins for creating iptables chains
causes one of the tasks to fail. We upstreamed a patch to CNI plugins
to make iptables chain creation idempotent.

This changeset updates end-to-end testing, development tooling, and
documentation to use 0.8.4 which includes our patch.
2020-01-09 10:57:07 -05:00
Mahmood Ali 4e5d867644 client: stop using alloc.TaskResources
Now that alloc.Canonicalize() is called in all alloc sources in the
client (i.e. on state restore and RPC fetching), we no longer need to
check alloc.TaskResources.

alloc.AllocatedResources is always non-nil through alloc runner.
Though, early on, we check for alloc validity, so NewTaskRunner and
TaskEnv must still check.  `TestClient_AddAllocError` test validates
that behavior.
2020-01-09 09:25:07 -05:00
Mahmood Ali 7c153e1a64 client: canonicalize alloc runner on RPC 2020-01-09 08:46:50 -05:00
Mahmood Ali d740d347ce Migrate old alloc structs on read
This commit ensures that Alloc.AllocatedResources is properly populated
when read from persistence stores (namely Raft and client state store).
The alloc struct may have been written previously by an arbitrary old
version that may only populate Alloc.TaskResources.
2020-01-09 08:46:50 -05:00
Tim Gross c11cc60674 commit a hclfmt to eliminate diffs after 'make dev' 2020-01-09 08:18:51 -05:00
Tim Gross dca69e3ad9 changelog for GH-6916 2020-01-09 08:16:41 -05:00
Tim Gross fa4da93578
interpolate environment for services in script checks (#6916)
In 0.10.2 (specifically 387b016) we added interpolation to group
service blocks and centralized the logic for task environment
interpolation. This wasn't also added to script checks, which caused a
regression where the IDs for script checks for services w/
interpolated fields (ex. the service name) didn't match the service ID
that was registered with Consul.

This changeset calls the same taskenv interpolation logic during
`script_check` configuration, and adds tests to reduce the risk of
future regressions by comparing the IDs of service hook and the check hook.
2020-01-09 08:12:54 -05:00
Mahmood Ali a2e181dd45 CLI: protect against AllocatedResources being nil 2020-01-08 17:22:05 -05:00
Mahmood Ali 8e78fb15bf Update go-discover library
To pick up go.net change so we can cross compile to arm64
2020-01-08 14:49:04 -05:00
Nick Ethier 9c3cc63cd1 tr: initialize envoybootstrap prestart hook response.Env field 2020-01-08 13:41:38 -05:00
Charlie Voiselle d1b12781ea
Merge pull request #6915 from hashicorp/d-consul-warning
[docs] Change consul agent note to callout
2020-01-08 13:40:27 -05:00
Charlie Voiselle 52dfb3de79 Change consul agent note to callout
Because this is such a common misconfig, it is probably worth
highlighting
2020-01-08 13:08:54 -05:00
Charlie Voiselle 5298fee5d6
Typo fix
Synopsis needs to start with uppercase to match other commands
2020-01-08 10:44:00 -05:00