Commit graph

480 commits

Author SHA1 Message Date
Drew Bailey b296558b8e
oss compoments for multi-vault namespaces
adds in oss components to support enterprise multi-vault namespace feature

upgrade specific doc on vault multi-namespaces

vault docs

update test to reflect new error
2020-07-24 10:14:59 -04:00
Tim Gross b6d9e546aa
fix volume deregister -force params in API (#8380)
The CSI `volume deregister -force` flag was using the documented parameter
`force` everywhere except in the API, where it was incorrectly passing `purge`
as the query parameter.
2020-07-08 10:08:22 -04:00
Chris Baker 9100b6b7c0 changes to make sure that Max is present and valid, to improve error messages
* made api.Scaling.Max a pointer, so we can detect (and complain) when it is neglected
* added checks to HCL parsing that it is present
* when Scaling.Max is absent/invalid, don't return extraneous error messages during validation
* tweak to multiregion handling to ensure that the count is valid on the interpolated regional jobs

resolves #8355
2020-07-04 19:05:50 +00:00
Lang Martin 6c22cd587d
api: nomad debug new /agent/host (#8325)
* command/agent/host: collect host data, multi platform

* nomad/structs/structs: new HostDataRequest/Response

* client/agent_endpoint: add RPC endpoint

* command/agent/agent_endpoint: add Host

* api/agent: add the Host endpoint

* nomad/client_agent_endpoint: add Agent Host with forwarding

* nomad/client_agent_endpoint: use findClientConn

This changes forwardMonitorClient and forwardProfileClient to use
findClientConn, which was cribbed from the common parts of those
funcs.

* command/debug: call agent hosts

* command/agent/host: eliminate calling external programs
2020-07-02 09:51:25 -04:00
Tim Gross 23be116da0
csi: add -force flag to volume deregister (#8295)
The `nomad volume deregister` command currently returns an error if the volume
has any claims, but in cases where the claims can't be dropped because of
plugin errors, providing a `-force` flag gives the operator an escape hatch.

If the volume has no allocations or if they are all terminal, this flag
deletes the volume from the state store, immediately and implicitly dropping
all claims without further CSI RPCs. Note that this will not also
unmount/detach the volume, which we'll make the responsibility of a separate
`nomad volume detach` command.
2020-07-01 12:17:51 -04:00
Nick Ethier 89118016fc
command: correctly show host IP in ports output /w multi-host networks (#8289) 2020-06-25 15:16:01 -04:00
Seth Hoenig 3872b493e5
Merge pull request #8011 from hashicorp/f-cnative-host
consul/connect: implement initial support for connect native
2020-06-24 10:33:12 -05:00
Seth Hoenig 6c5ab7f45e consul/connect: split connect native flag and task in service 2020-06-23 10:22:22 -05:00
Mahmood Ali e7a3d6e358 Update github.com/syndtr/gocapability
runc requires a more recent version
2020-06-22 17:10:41 -04:00
Mahmood Ali 7423f38771 Upgrade libcontainer dependency
Upgrade libcontainer dependency to pick up fix for CVE-2019-19921 .
2020-06-22 16:56:54 -04:00
Seth Hoenig 4d71f22a11 consul/connect: add support for running connect native tasks
This PR adds the capability of running Connect Native Tasks on Nomad,
particularly when TLS and ACLs are enabled on Consul.

The `connect` stanza now includes a `native` parameter, which can be
set to the name of task that backs the Connect Native Consul service.

There is a new Client configuration parameter for the `consul` stanza
called `share_ssl`. Like `allow_unauthenticated` the default value is
true, but recommended to be disabled in production environments. When
enabled, the Nomad Client's Consul TLS information is shared with
Connect Native tasks through the normal Consul environment variables.
This does NOT include auth or token information.

If Consul ACLs are enabled, Service Identity Tokens are automatically
and injected into the Connect Native task through the CONSUL_HTTP_TOKEN
environment variable.

Any of the automatically set environment variables can be overridden by
the Connect Native task using the `env` stanza.

Fixes #6083
2020-06-22 14:07:44 -05:00
Michael Schurter 562704124d
Merge pull request #8208 from hashicorp/f-multi-network
multi-interface network support
2020-06-19 15:46:48 -07:00
Nick Ethier 6e3313c672
vendor: sync api changes 2020-06-19 11:31:39 -04:00
Drew Bailey c2d7b61939
allow raw body instead of JSON encoded string (#8211) 2020-06-19 10:57:09 -04:00
Tim Gross 8a354f828f
store ACL Accessor ID from Job.Register with Job (#8204)
In multiregion deployments when ACLs are enabled, the deploymentwatcher needs
an appropriately scoped ACL token with the same `submit-job` rights as the
user who submitted it. The token will already be replicated, so store the
accessor ID so that it can be retrieved by the leader.
2020-06-19 07:53:29 -04:00
Mahmood Ali 38a01c050e
Merge pull request #8192 from hashicorp/f-status-allnamespaces-2
CLI Allow querying all namespaces for jobs and allocations - Try 2
2020-06-18 20:16:52 -04:00
Michael Schurter 0ed023c61a deps: updated libnetwork dep
Adding missing vendor files as well.
2020-06-18 11:05:31 -07:00
Nick Ethier 0bc0403cc3 Task DNS Options (#7661)
Co-Authored-By: Tim Gross <tgross@hashicorp.com>
Co-Authored-By: Seth Hoenig <shoenig@hashicorp.com>
2020-06-18 11:01:31 -07:00
Mahmood Ali 7a33a75449 cli: jobs allow querying jobs in all namespaces 2020-06-17 16:31:01 -04:00
Mahmood Ali e784fe331a use '*' to indicate all namespaces
This reverts the introduction of AllNamespaces parameter that was merged
earlier but never got released.
2020-06-17 16:27:43 -04:00
Tim Gross 7b12445f29 multiregion: change AutoRevert to OnFailure 2020-06-17 11:05:45 -04:00
Tim Gross b09b7a2475 Multiregion job registration
Integration points for multiregion jobs to be registered in the enterprise
version of Nomad:
* hook in `Job.Register` for enterprise to send job to peer regions
* remove monitoring from `nomad job run` and `nomad job stop` for multiregion jobs
2020-06-17 11:04:58 -04:00
Tim Gross b93efc16d5 multiregion CLI: nomad deployment unblock 2020-06-17 11:03:44 -04:00
Drew Bailey 9263fcb0d3 Multiregion deploy status and job status CLI 2020-06-17 11:03:34 -04:00
Tim Gross 6851024925 Multiregion structs
Initial struct definitions, jobspec parsing, validation, and conversion
between Nomad structs and API structs for multi-region deployments.
2020-06-17 11:00:14 -04:00
Chris Baker 9fc66bc1aa support in API client and Job.Register RPC for PreserveCounts 2020-06-16 18:45:28 +00:00
Chris Baker 377f881fbd removed api.RegisterJobRequest in favor of api.JobRegisterRequest
modified `job inspect` and `job run -output` to use anonymous struct to keep previous behavior
2020-06-16 18:45:17 +00:00
Chris Baker 1e3563e08c wip: added PreserveCounts to struct.JobRegisterRequest, development test for Job.Register 2020-06-16 18:45:17 +00:00
Mahmood Ali 5cf04b5762 api: add snapshot restore 2020-06-07 15:47:07 -04:00
Mahmood Ali 82c87cf97b
Vendor gopsutil with go mod fix (#8108)
Point to gopsutil fork with shirou/gopsutil#889 changes. I used hashicorp/gopsutil rather than pointing to https://github.com/Lomanic/gopsutil/tree/issue832 directly just in case..
2020-06-03 16:09:27 -04:00
Mahmood Ali ee453f2932
run go mod vendor (#8107) 2020-06-03 15:34:39 -04:00
Seth Hoenig 435c0d9fc8 deps: Switch to Go modules for dependency management
This PR switches the Nomad repository from using govendor to Go modules
for managing dependencies. Aspects of the Nomad workflow remain pretty
much the same. The usual Makefile targets should continue to work as
they always did. The API submodule simply defers to the parent Nomad
version on the repository, keeping the semantics of API versioning that
currently exists.
2020-06-02 14:30:36 -05:00
Yoan Blanc e983dc2cf5
docker pick https://github.com/moby/moby/pull/40021
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-06-01 22:43:58 +02:00
Yoan Blanc 3d60a77b3e
docker v19.03.9
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-06-01 22:43:28 +02:00
Mahmood Ali 1ca6162793
Merge pull request #8043 from greut/distribution-v2.7.1
vendor: pin docker/distribution to v2.7.1
2020-05-31 21:20:54 -04:00
Seth Hoenig 4cca46ae61 deps: update go-getter for redirects fix
Fixes #7854

Nomad requires a version of go-getter that is currently in PR (https://github.com/hashicorp/go-getter/pull/256)
We also require some recent bug fix to go-getter around the handling of URL redirects.

Update our vendor'd copy of go-getter to the newly rebased umask changes so that we can incorporate
the latest changes for go-getter.
2020-05-28 08:24:40 -05:00
Drew Bailey 34871f89be
Oss license support for ent builds (#8054)
* changes necessary to support oss licesning shims

revert nomad fmt changes

update test to work with enterprise changes

update tests to work with new ent enforcements

make check

update cas test to use scheduler algorithm

back out preemption changes

add comments

* remove unused method
2020-05-27 13:46:52 -04:00
Yoan Blanc 27568d83d8
vendor: pin docker/distribution to v2.7.1
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-05-24 09:30:05 +02:00
Mahmood Ali 751f337f1c Update hcl2 vendoring
The hcl2 library has moved from http://github.com/hashicorp/hcl2 to https://github.com/hashicorp/hcl/tree/hcl2.

This updates Nomad's vendoring to start using hcl2 library.  Also
updates some related libraries (e.g. `github.com/zclconf/go-cty/cty` and
`github.com/apparentlymart/go-textseg`).
2020-05-19 15:00:03 -04:00
Mahmood Ali 326793939e vendor: use tagged cronexpr, v1.1.0
Also, update to the version with modification notice
2020-05-12 16:20:00 -04:00
Mahmood Ali c8fb132956 Update cronexpr to point to hashicorp/cronexpr 2020-05-07 17:50:45 -04:00
ccn a4c36add17 Vendor: update fsouza/go-dockerclient to v1.6.5 2020-04-29 18:54:55 +08:00
Mahmood Ali e4f28e24a3 Vendor: update armon/go-metrics to v0.3.3
To pick up a lock contention fix in prometheus sink:
https://github.com/armon/go-metrics/pull/107 .
2020-04-26 08:54:50 -04:00
Mahmood Ali 14a461d6c4
Merge pull request #7560 from hashicorp/vendor-go-msgpack-v1.1.5
vendor: explicit use of hashicorp/go-msgpack
2020-03-31 10:09:05 -04:00
Yoan Blanc 761d014071 vendor: explicit use of hashicorp/go-msgpack
Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2020-03-31 09:45:21 -04:00
Mahmood Ali a6edc29b0a Fix docker/docker compile issue
Use docker master branch for specific branches, so we can pick up the
fix in https://github.com/moby/moby/pull/40021 .
2020-03-30 16:27:05 -04:00
Mahmood Ali bbd48deed0 vendor: github.com/containerd/continuity pkgs 2020-03-30 15:30:50 -04:00
Mahmood Ali 8f57f78087 vendors: update fsouza/go-docker-client to v.1.6.3 2020-03-30 15:10:53 -04:00
Mahmood Ali 4d90bf3699 vendoring: Update docker packages to v19.03.8
Also, since github.com/docker/docker is the canonical package names and
is transparently forwarded to github.com/moby/moby, I removed the
moby/moby references in origin.
2020-03-30 15:02:56 -04:00
Mahmood Ali 65d2fb5e32
Merge pull request #7531 from greut/docker-v19.03.8
Docker v19.03.8
2020-03-30 14:45:10 -04:00