Commit Graph

552 Commits

Author SHA1 Message Date
Drew Bailey 1c95bdb616
upgrade go metrics to 0.3.4 (#8637) 2020-08-11 13:26:45 -04:00
Tim Gross 443fdaa86b
csi: nomad volume detach command (#8584)
The soundness guarantees of the CSI specification leave a little to be desired
in our ability to provide a 100% reliable automated solution for managing
volumes. This changeset provides a new command to bridge this gap by providing
the operator the ability to intervene.

The command doesn't take an allocation ID so that the operator doesn't have to
keep track of alloc IDs that may have been GC'd. Handle this case in the
unpublish RPC by sending the client RPC for all the terminal/nil allocs on the
selected node.
2020-08-11 10:18:54 -04:00
Seth Hoenig fd4804bf26 consul: able to set pass/fail thresholds on consul service checks
This change adds the ability to set the fields `success_before_passing` and
`failures_before_critical` on Consul service check definitions. This is a
feature added to Consul v1.7.0 and later.
  https://www.consul.io/docs/agent/checks#success-failures-before-passing-critical

Nomad doesn't do much besides pass the fields through to Consul.

Fixes #6913
2020-08-10 14:08:09 -05:00
Seth Hoenig fb1c85a956 deps: upgrade import of consul/api
Upgrade our consul/api import to the equivelent of consul@v1.8.1 which includes
a bug fix necessary for #6913. If consul would publish a proper api/ submodule tag
we could reference that.
2020-08-06 21:02:33 -05:00
Drew Bailey b296558b8e
oss compoments for multi-vault namespaces
adds in oss components to support enterprise multi-vault namespace feature

upgrade specific doc on vault multi-namespaces

vault docs

update test to reflect new error
2020-07-24 10:14:59 -04:00
Mahmood Ali e9bf3a42f5
Merge pull request #8333 from hashicorp/b-test-tweak-20200701
tests: avoid os.Exit in a test
2020-07-10 11:18:28 -04:00
Tim Gross b6d9e546aa
fix volume deregister -force params in API (#8380)
The CSI `volume deregister -force` flag was using the documented parameter
`force` everywhere except in the API, where it was incorrectly passing `purge`
as the query parameter.
2020-07-08 10:08:22 -04:00
Chris Baker 9100b6b7c0 changes to make sure that Max is present and valid, to improve error messages
* made api.Scaling.Max a pointer, so we can detect (and complain) when it is neglected
* added checks to HCL parsing that it is present
* when Scaling.Max is absent/invalid, don't return extraneous error messages during validation
* tweak to multiregion handling to ensure that the count is valid on the interpolated regional jobs

resolves #8355
2020-07-04 19:05:50 +00:00
Lang Martin 6c22cd587d
api: `nomad debug` new /agent/host (#8325)
* command/agent/host: collect host data, multi platform

* nomad/structs/structs: new HostDataRequest/Response

* client/agent_endpoint: add RPC endpoint

* command/agent/agent_endpoint: add Host

* api/agent: add the Host endpoint

* nomad/client_agent_endpoint: add Agent Host with forwarding

* nomad/client_agent_endpoint: use findClientConn

This changes forwardMonitorClient and forwardProfileClient to use
findClientConn, which was cribbed from the common parts of those
funcs.

* command/debug: call agent hosts

* command/agent/host: eliminate calling external programs
2020-07-02 09:51:25 -04:00
Mahmood Ali 026d8c6eed tests: avoid os.Exit in a test 2020-07-01 15:25:13 -04:00
Tim Gross 23be116da0
csi: add -force flag to volume deregister (#8295)
The `nomad volume deregister` command currently returns an error if the volume
has any claims, but in cases where the claims can't be dropped because of
plugin errors, providing a `-force` flag gives the operator an escape hatch.

If the volume has no allocations or if they are all terminal, this flag
deletes the volume from the state store, immediately and implicitly dropping
all claims without further CSI RPCs. Note that this will not also
unmount/detach the volume, which we'll make the responsibility of a separate
`nomad volume detach` command.
2020-07-01 12:17:51 -04:00
Nick Ethier 89118016fc
command: correctly show host IP in ports output /w multi-host networks (#8289) 2020-06-25 15:16:01 -04:00
Seth Hoenig 3872b493e5
Merge pull request #8011 from hashicorp/f-cnative-host
consul/connect: implement initial support for connect native
2020-06-24 10:33:12 -05:00
Seth Hoenig 6c5ab7f45e consul/connect: split connect native flag and task in service 2020-06-23 10:22:22 -05:00
Mahmood Ali e7a3d6e358 Update github.com/syndtr/gocapability
runc requires a more recent version
2020-06-22 17:10:41 -04:00
Mahmood Ali 7423f38771 Upgrade libcontainer dependency
Upgrade libcontainer dependency to pick up fix for CVE-2019-19921 .
2020-06-22 16:56:54 -04:00
Seth Hoenig 4d71f22a11 consul/connect: add support for running connect native tasks
This PR adds the capability of running Connect Native Tasks on Nomad,
particularly when TLS and ACLs are enabled on Consul.

The `connect` stanza now includes a `native` parameter, which can be
set to the name of task that backs the Connect Native Consul service.

There is a new Client configuration parameter for the `consul` stanza
called `share_ssl`. Like `allow_unauthenticated` the default value is
true, but recommended to be disabled in production environments. When
enabled, the Nomad Client's Consul TLS information is shared with
Connect Native tasks through the normal Consul environment variables.
This does NOT include auth or token information.

If Consul ACLs are enabled, Service Identity Tokens are automatically
and injected into the Connect Native task through the CONSUL_HTTP_TOKEN
environment variable.

Any of the automatically set environment variables can be overridden by
the Connect Native task using the `env` stanza.

Fixes #6083
2020-06-22 14:07:44 -05:00
Chris Baker b79e25efb9 `make sync` 2020-06-19 23:27:00 +00:00
Michael Schurter 562704124d
Merge pull request #8208 from hashicorp/f-multi-network
multi-interface network support
2020-06-19 15:46:48 -07:00
Mahmood Ali 3824e0362c
Revert "client: defensive against getting stale alloc updates" 2020-06-19 15:39:44 -04:00
Nick Ethier 6e3313c672
vendor: sync api changes 2020-06-19 11:31:39 -04:00
Drew Bailey c2d7b61939
allow raw body instead of JSON encoded string (#8211) 2020-06-19 10:57:09 -04:00
Tim Gross 8a354f828f
store ACL Accessor ID from Job.Register with Job (#8204)
In multiregion deployments when ACLs are enabled, the deploymentwatcher needs
an appropriately scoped ACL token with the same `submit-job` rights as the
user who submitted it. The token will already be replicated, so store the
accessor ID so that it can be retrieved by the leader.
2020-06-19 07:53:29 -04:00
Mahmood Ali 38a01c050e
Merge pull request #8192 from hashicorp/f-status-allnamespaces-2
CLI Allow querying all namespaces for jobs and allocations - Try 2
2020-06-18 20:16:52 -04:00
Michael Schurter 0ed023c61a deps: updated libnetwork dep
Adding missing vendor files as well.
2020-06-18 11:05:31 -07:00
Nick Ethier 0bc0403cc3 Task DNS Options (#7661)
Co-Authored-By: Tim Gross <tgross@hashicorp.com>
Co-Authored-By: Seth Hoenig <shoenig@hashicorp.com>
2020-06-18 11:01:31 -07:00
Mahmood Ali 7a33a75449 cli: jobs allow querying jobs in all namespaces 2020-06-17 16:31:01 -04:00
Mahmood Ali e784fe331a use '*' to indicate all namespaces
This reverts the introduction of AllNamespaces parameter that was merged
earlier but never got released.
2020-06-17 16:27:43 -04:00
Tim Gross 7b12445f29 multiregion: change AutoRevert to OnFailure 2020-06-17 11:05:45 -04:00
Tim Gross b09b7a2475 Multiregion job registration
Integration points for multiregion jobs to be registered in the enterprise
version of Nomad:
* hook in `Job.Register` for enterprise to send job to peer regions
* remove monitoring from `nomad job run` and `nomad job stop` for multiregion jobs
2020-06-17 11:04:58 -04:00
Tim Gross b93efc16d5 multiregion CLI: nomad deployment unblock 2020-06-17 11:03:44 -04:00
Drew Bailey 9263fcb0d3 Multiregion deploy status and job status CLI 2020-06-17 11:03:34 -04:00
Tim Gross 6851024925 Multiregion structs
Initial struct definitions, jobspec parsing, validation, and conversion
between Nomad structs and API structs for multi-region deployments.
2020-06-17 11:00:14 -04:00
Chris Baker 9fc66bc1aa support in API client and Job.Register RPC for PreserveCounts 2020-06-16 18:45:28 +00:00
Chris Baker 377f881fbd removed api.RegisterJobRequest in favor of api.JobRegisterRequest
modified `job inspect` and `job run -output` to use anonymous struct to keep previous behavior
2020-06-16 18:45:17 +00:00
Chris Baker 1e3563e08c wip: added PreserveCounts to struct.JobRegisterRequest, development test for Job.Register 2020-06-16 18:45:17 +00:00
Mahmood Ali 5cf04b5762 api: add snapshot restore 2020-06-07 15:47:07 -04:00
Mahmood Ali 82c87cf97b
Vendor gopsutil with go mod fix (#8108)
Point to gopsutil fork with shirou/gopsutil#889 changes. I used hashicorp/gopsutil rather than pointing to https://github.com/Lomanic/gopsutil/tree/issue832 directly just in case..
2020-06-03 16:09:27 -04:00
Mahmood Ali ee453f2932
run go mod vendor (#8107) 2020-06-03 15:34:39 -04:00
Seth Hoenig 435c0d9fc8 deps: Switch to Go modules for dependency management
This PR switches the Nomad repository from using govendor to Go modules
for managing dependencies. Aspects of the Nomad workflow remain pretty
much the same. The usual Makefile targets should continue to work as
they always did. The API submodule simply defers to the parent Nomad
version on the repository, keeping the semantics of API versioning that
currently exists.
2020-06-02 14:30:36 -05:00
Yoan Blanc e983dc2cf5
docker pick https://github.com/moby/moby/pull/40021
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-06-01 22:43:58 +02:00
Yoan Blanc 53355b998d
docker/cli v19.03.10
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-06-01 22:43:56 +02:00
Yoan Blanc f48f964161
docker v19.03.10
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-06-01 22:43:31 +02:00
Yoan Blanc 3d60a77b3e
docker v19.03.9
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-06-01 22:43:28 +02:00
Mahmood Ali 1ca6162793
Merge pull request #8043 from greut/distribution-v2.7.1
vendor: pin docker/distribution to v2.7.1
2020-05-31 21:20:54 -04:00
Seth Hoenig 4cca46ae61 deps: update go-getter for redirects fix
Fixes #7854

Nomad requires a version of go-getter that is currently in PR (https://github.com/hashicorp/go-getter/pull/256)
We also require some recent bug fix to go-getter around the handling of URL redirects.

Update our vendor'd copy of go-getter to the newly rebased umask changes so that we can incorporate
the latest changes for go-getter.
2020-05-28 08:24:40 -05:00
Drew Bailey 34871f89be
Oss license support for ent builds (#8054)
* changes necessary to support oss licesning shims

revert nomad fmt changes

update test to work with enterprise changes

update tests to work with new ent enforcements

make check

update cas test to use scheduler algorithm

back out preemption changes

add comments

* remove unused method
2020-05-27 13:46:52 -04:00
Yoan Blanc 27568d83d8
vendor: pin docker/distribution to v2.7.1
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-05-24 09:30:05 +02:00
Mahmood Ali 751f337f1c Update hcl2 vendoring
The hcl2 library has moved from http://github.com/hashicorp/hcl2 to https://github.com/hashicorp/hcl/tree/hcl2.

This updates Nomad's vendoring to start using hcl2 library.  Also
updates some related libraries (e.g. `github.com/zclconf/go-cty/cty` and
`github.com/apparentlymart/go-textseg`).
2020-05-19 15:00:03 -04:00
Mahmood Ali 326793939e vendor: use tagged cronexpr, v1.1.0
Also, update to the version with modification notice
2020-05-12 16:20:00 -04:00