Commit graph

2720 commits

Author SHA1 Message Date
Lang Martin 97c7f2acea
command/operator_debug: mkdir before storing agent-host (#8707)
The api calls were reordered, the new order omits the
`agent-host.json` result by fetching it before the directory is
created.
2020-08-28 11:58:06 -04:00
Lang Martin 7d483f93c0
csi: plugins track jobs in addition to allocations, and use job information to set expected counts (#8699)
* nomad/structs/csi: add explicit job support
* nomad/state/state_store: capture job updates directly
* api/nodes: CSIInfo needs the AllocID
* command/agent/csi_endpoint: AllocID was missing
Co-authored-by: Tim Gross <tgross@hashicorp.com>
2020-08-27 17:20:00 -04:00
Seth Hoenig 9f1f2a5673 Merge branch 'master' into f-cc-ingress 2020-08-26 15:31:05 -05:00
Seth Hoenig dfe179abc5 consul/connect: fixup some comments and context timeout 2020-08-26 13:17:16 -05:00
Tim Gross f9b6c8153c
csi: fix panic in serializing nil allocs in volume API (#8735)
- fix panic in serializing nil allocs in volume API
- prevent potential panic in serializing plugin allocs
2020-08-25 10:13:05 -04:00
Seth Hoenig 26e77623e5 consul/connect: fixup tests to use new consul sdk 2020-08-24 12:02:41 -05:00
Seth Hoenig c4fa644315 consul/connect: remove envoy dns option from gateway proxy config 2020-08-24 09:11:55 -05:00
Yoan Blanc 327d17e0dc
fixup! vendor: consul/api, consul/sdk v1.6.0
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-08-24 08:59:03 +02:00
Seth Hoenig 5b072029f2 consul/connect: add initial support for ingress gateways
This PR adds initial support for running Consul Connect Ingress Gateways (CIGs) in Nomad. These gateways are declared as part of a task group level service definition within the connect stanza.

```hcl
service {
  connect {
    gateway {
      proxy {
        // envoy proxy configuration
      }
      ingress {
        // ingress-gateway configuration entry
      }
    }
  }
}
```

A gateway can be run in `bridge` or `host` networking mode, with the caveat that host networking necessitates manually specifying the Envoy admin listener (which cannot be disabled) via the service port value.

Currently Envoy is the only supported gateway implementation in Consul, and Nomad only supports running Envoy as a gateway using the docker driver.

Aims to address #8294 and tangentially #8647
2020-08-21 16:21:54 -05:00
Nick Ethier 3cd5f46613
Update UI to use new allocated ports fields (#8631)
* nomad: canonicalize alloc shared resources to populate ports

* ui: network ports

* ui: remove unused task network references and update tests with new shared ports model

* ui: lint

* ui: revert auto formatting

* ui: remove unused page objects

* structs: remove unrelated test from bad conflict resolution

* ui: formatting
2020-08-20 11:07:13 -04:00
Tim Gross 22e77bb03c
mrd: remove redundant validation in HTTP endpoint (#8685)
The `regionForJob` function in the HTTP job endpoint overrides the region for
multiregion jobs to `global`, which is used as a sentinel value in the
server's job endpoint to avoid re-registration loops. This changeset removes
an extraneous check that results in errors in the web UI and makes
round-tripping through the HTTP API cumbersome for all consumers.
2020-08-18 16:48:09 -04:00
Tim Gross 38ec70eb8d
multiregion: validation should always return error for OSS (#8687) 2020-08-18 15:35:38 -04:00
Lang Martin 6d8165c410
command/agent/csi_endpoint: explicit allocations (#8669) 2020-08-13 15:48:08 -04:00
Tim Gross 7dca72acbe
csi: fix panic from assignment to nil map in plugin API (#8666) 2020-08-13 11:36:41 -04:00
Tim Gross 3faa138732
fix panic converting structs to API in CSI endpoint (#8659) 2020-08-12 15:59:10 -04:00
Nomad Release bot 1ea9d4eb22 Generate files for 0.12.2 release 2020-08-12 00:50:49 +00:00
Lang Martin 07ea822c6a
nomad debug renamed to nomad operator debug (#8602)
* renamed: command/debug.go -> command/operator_debug.go
* website: rename debug -> operator debug
* website/pages/api-docs/agent: name in api docs
2020-08-11 15:39:44 -04:00
Lang Martin 1d7998f39f
debug command archive content changes (#8462)
* command/debug: print interval data so the operator knows its waiting

* command/debug: use the Consul/Vault env for queries

* command/debug: capture the operator endpoints

* command/debug: capture API errors in the archive bundle
2020-08-11 13:14:28 -04:00
Lang Martin c82b2a2454
CSI: volume and plugin allocations in the API (#8590)
* command/agent/csi_endpoint: explicitly convert to API structs, and convert allocs for single object get endpoints
2020-08-11 12:24:41 -04:00
Tim Gross 443fdaa86b
csi: nomad volume detach command (#8584)
The soundness guarantees of the CSI specification leave a little to be desired
in our ability to provide a 100% reliable automated solution for managing
volumes. This changeset provides a new command to bridge this gap by providing
the operator the ability to intervene.

The command doesn't take an allocation ID so that the operator doesn't have to
keep track of alloc IDs that may have been GC'd. Handle this case in the
unpublish RPC by sending the client RPC for all the terminal/nil allocs on the
selected node.
2020-08-11 10:18:54 -04:00
Seth Hoenig fd4804bf26 consul: able to set pass/fail thresholds on consul service checks
This change adds the ability to set the fields `success_before_passing` and
`failures_before_critical` on Consul service check definitions. This is a
feature added to Consul v1.7.0 and later.
  https://www.consul.io/docs/agent/checks#success-failures-before-passing-critical

Nomad doesn't do much besides pass the fields through to Consul.

Fixes #6913
2020-08-10 14:08:09 -05:00
Drew Bailey c06a84e4a2
ignore VAULT_NAMESPACE (#8581)
VAULT_NAMESPACE in 0.12.1 and previous versions is already ignored. \n revert change that used it as a default since it will break oss users
2020-07-31 10:33:21 -04:00
Drew Bailey b296558b8e
oss compoments for multi-vault namespaces
adds in oss components to support enterprise multi-vault namespace feature

upgrade specific doc on vault multi-namespaces

vault docs

update test to reflect new error
2020-07-24 10:14:59 -04:00
Mahmood Ali b800a4f80e
Merge pull request #8514 from sashaaKr/bugfix/cli_ui
change url to client
2020-07-24 09:54:39 -04:00
James Rasell 95db43eaf0
Merge pull request #8491 from hashicorp/b-gh-8481
api: task groups in system jobs do not support scaling stanzas.
2020-07-24 14:20:26 +02:00
Tim Gross 43d2052c99
csi: avoid panic in CLI for failed node attachment cleanup (#8525)
If the node API returns an attached volume that doesn't belong to an
alloc (because it's failed to clean up properly), `nomad node status`
will panic when rendering the response.

Also, avoid empty volumes output in node status
2020-07-24 08:17:27 -04:00
sasha f09a65227d remove test file 2020-07-23 18:44:10 +03:00
sasha 2980010e63 change url to client 2020-07-23 18:41:38 +03:00
Nomad Release bot f2f50bf48e Generate files for 0.12.1 release 2020-07-23 13:17:59 +00:00
Mahmood Ali fc38cd21c4 mrd: only output evalID if found
If the multi-region job is a periodic/dispatch job, stopping them
returns an empty EvalID. This removes some unexpected empty lines.
2020-07-22 16:43:03 -04:00
Lars Lehtonen e26ea30b7e
command/agent: fix dropped test error (#8504) 2020-07-22 15:06:35 -04:00
Drew Bailey 744cf9b2e8
remove duplicate license info (#8496) 2020-07-22 10:21:56 -04:00
James Rasell 2da8bd8f58
agent: task groups in system jobs do not support scaling stanzas. 2020-07-22 11:10:59 +02:00
Mahmood Ali c29dec2ebd
format job init hcl (#8483) 2020-07-21 11:49:02 -04:00
Mahmood Ali a483dde8b9 minor tweaks from Ent 2020-07-20 09:25:09 -04:00
Mahmood Ali 72ac33e4e7 Refactor setupLoggers 2020-07-17 11:05:57 -04:00
Mahmood Ali ad2d484974 Set AgentShutdown 2020-07-17 11:04:57 -04:00
Mahmood Ali ec9a12e54e Fix pro tags 2020-07-17 11:02:00 -04:00
Tim Gross bd457343de
MRD: all regions should start pending (#8433)
Deployments should wait until kicked off by `Job.Register` so that we can
assert that all regions have a scheduled deployment before starting any
region. This changeset includes the OSS fixes to support the ENT work.

`IsMultiregionStarter` has no more callers in OSS, so remove it here.
2020-07-14 10:57:37 -04:00
Chris Baker f8478b6f82 Merge branch 'master' of github.com:hashicorp/nomad into release-0.12.0 2020-07-08 21:16:31 +00:00
Nick Ethier 119ece09a0
docs: add CNI and host_network docs (#8391)
Co-authored-by: Seth Hoenig <shoenig@hashicorp.com>
2020-07-08 15:45:04 -04:00
Tim Gross 1098ca6ef1
fix multiregion plan output flags (#8375)
The call to render the output diff swapped the `diff` and `verbose` bool
parameters, resulting in dropping the diff output in multi-region plans but
not single-region plans.
2020-07-08 10:10:08 -04:00
Nomad Release bot 549e766eab Generate files for 0.12.0-rc1 release 2020-07-07 03:17:05 +00:00
Nick Ethier e0fb634309
ar: support opting into binding host ports to default network IP (#8321)
* ar: support opting into binding host ports to default network IP

* fix config plumbing

* plumb node address into network resource

* struct: only handle network resource upgrade path once
2020-07-06 18:51:46 -04:00
Tim Gross 18250f71fd
fix region flag vs job region handling in plan/submit (#8347) 2020-07-06 15:46:09 -04:00
Chris Baker 9100b6b7c0 changes to make sure that Max is present and valid, to improve error messages
* made api.Scaling.Max a pointer, so we can detect (and complain) when it is neglected
* added checks to HCL parsing that it is present
* when Scaling.Max is absent/invalid, don't return extraneous error messages during validation
* tweak to multiregion handling to ensure that the count is valid on the interpolated regional jobs

resolves #8355
2020-07-04 19:05:50 +00:00
Mahmood Ali 329969b97e tests: make testagent shutdown idempotent
Avoid double freeing ports if an agent.Shutdown() is called multiple
times.
2020-07-03 09:16:01 -04:00
Lang Martin 1e7560d621
command/debug: use the correct env vars for Consul token (#8332) 2020-07-02 10:04:22 -04:00
Lang Martin 6c22cd587d
api: nomad debug new /agent/host (#8325)
* command/agent/host: collect host data, multi platform

* nomad/structs/structs: new HostDataRequest/Response

* client/agent_endpoint: add RPC endpoint

* command/agent/agent_endpoint: add Host

* api/agent: add the Host endpoint

* nomad/client_agent_endpoint: add Agent Host with forwarding

* nomad/client_agent_endpoint: use findClientConn

This changes forwardMonitorClient and forwardProfileClient to use
findClientConn, which was cribbed from the common parts of those
funcs.

* command/debug: call agent hosts

* command/agent/host: eliminate calling external programs
2020-07-02 09:51:25 -04:00
Mahmood Ali 1917989a1f document namespace option in CLI docs 2020-07-01 15:31:41 -04:00