Commit Graph

3045 Commits

Author SHA1 Message Date
Mahmood Ali 4d90afb425 gofmt all the files
mostly to handle build directives in 1.17.
2021-10-01 10:14:28 -04:00
Michael Schurter 4ad0c258b9 client: add NOMAD_LICENSE to default env deny list
By default we should not expose the NOMAD_LICENSE environment variable
to tasks.

Also refactor where the DefaultEnvDenyList lives so we don't have to
maintain 2 copies of it. Since client/config is the most obvious
location, keep a reference there to its unfortunate home buried deep
in command/agent/host. Since the agent uses this list as well for the
/agent/host endpoint the list must be accessible from both command/agent
and client.
2021-09-21 13:51:17 -07:00
Florian Apolloner 7805b8edf4
Fixed usage of NOMAD_CLI_NO_COLOR env variable. (#11168) 2021-09-17 20:37:05 -04:00
James Rasell 0e926ef3fd
allow configuration of Docker hostnames in bridge mode (#11173)
Add a new hostname string parameter to the network block which
allows operators to specify the hostname of the network namespace.
Changing this causes a destructive update to the allocation and it
is omitted if empty from API responses. This parameter also supports
interpolation.

In order to have a hostname passed as a configuration param when
creating an allocation network, the CreateNetwork func of the
DriverNetworkManager interface needs to be updated. In order to
minimize the disruption of future changes, rather than add another
string func arg, the function now accepts a request struct along with
the allocID param. The struct has the hostname as a field.

The in-tree implementations of DriverNetworkManager.CreateNetwork
have been modified to account for the function signature change.
In updating for the change, the enhancement of adding hostnames to
network namespaces has also been added to the Docker driver, whilst
the default Linux manager does not current implement it.
2021-09-16 08:13:09 +02:00
James Rasell 04a15b5c16
Merge pull request #11105 from hashicorp/f-add-staticcheck-ci
ci: add staticcheck with ST1020 and update golangci-lint
2021-09-09 09:42:12 +02:00
Luiz Aoqui 4dd8b6b571
cli: include all possible scores in alloc status metric table (#11128) 2021-09-08 17:30:11 -04:00
James Rasell d4a333e9b5
lint: mark false positive or fix gocritic append lint errors. 2021-09-06 10:49:44 +02:00
James Rasell b6813f1221
chore: fix incorrect docstring formatting. 2021-08-30 11:08:12 +02:00
Luiz Aoqui 104d29e808
Don't timestamp active log file (#11070)
* don't timestamp active log file

* website: update log_file default value

* changelog: add entry for #11070

* website: add upgrade instructions for log_file in v1.14 and v1.2.0
2021-08-23 11:27:34 -04:00
Mahmood Ali c37339a8c8
Merge pull request #9160 from hashicorp/f-sysbatch
core: implement system batch scheduler
2021-08-16 09:30:24 -04:00
Michael Schurter a7aae6fa0c
Merge pull request #10848 from ggriffiths/listsnapshot_secrets
CSI Listsnapshot secrets support
2021-08-10 15:59:33 -07:00
Seth Hoenig 3371214431 core: implement system batch scheduler
This PR implements a new "System Batch" scheduler type. Jobs can
make use of this new scheduler by setting their type to 'sysbatch'.

Like the name implies, sysbatch can be thought of as a hybrid between
system and batch jobs - it is for running short lived jobs intended to
run on every compatible node in the cluster.

As with batch jobs, sysbatch jobs can also be periodic and/or parameterized
dispatch jobs. A sysbatch job is considered complete when it has been run
on all compatible nodes until reaching a terminal state (success or failed
on retries).

Feasibility and preemption are governed the same as with system jobs. In
this PR, the update stanza is not yet supported. The update stanza is sill
limited in functionality for the underlying system scheduler, and is
not useful yet for sysbatch jobs. Further work in #4740 will improve
support for the update stanza and deployments.

Closes #2527
2021-08-03 10:30:47 -04:00
James Rasell 78a489418d
cli: fix minor format error within `-ca-cert` help text. 2021-08-03 16:05:06 +02:00
Mahmood Ali 0bc12fba7c
Only initialize task.VolumeMounts when not-nil (#10990)
1.1.3 had a bug where task.VolumeMounts will be an empty slice instead of nil. Eventually, it gets canonicalized and is set to `nil`, but it seems to confuse dry-run planning.

The regression was introduced in https://github.com/hashicorp/nomad/pull/10855/files#diff-56b3c82fcbc857f8fb93a903f1610f6e6859b3610a4eddf92bad9ea27fdc85ecL1028-R1037 . Curiously, it's the only place where `len(apiTask.VolumeMounts)` check was dropped. I assume it was dropped accidentally.

Fixes #10981
2021-08-02 13:08:10 -04:00
Nomad Release bot b5dff8be42 Generate files for 1.1.3 release 2021-07-29 03:43:03 +00:00
Grant Griffiths fecbbaee22 CSI ListSnapshots secrets implementation
Signed-off-by: Grant Griffiths <ggriffiths@purestorage.com>
2021-07-28 11:30:29 -07:00
Mahmood Ali d97927ebcf
cli: Use glint to determine if os.Stdout is tty (#10926)
Use glint to determine if os.Stdout is a terminal.

glint Terminal renderer expects os.Stdout [not only to be a terminal, but also to have non-zero size](b492b545f6/renderer_term.go (L39-L46)). It's unclear how this condition arises, but this additional check causes Nomad to render deployments progress through glint when glint cannot support it.

By using golint to perform the check, we eliminate the risk of mis-judgement.
2021-07-23 11:27:47 -04:00
Luiz Aoqui 484037aff1
fix `nomad alloc signal` help message (#10917) 2021-07-21 11:02:44 -04:00
Kent 'picat' Gruber decd59dbd1
Merge pull request #10886 from hashicorp/cli-handle-successful-deployment
Handle successful/canceled/blocked deployments in CLI output
2021-07-16 12:27:22 -04:00
Kent 'picat' Gruber b85b56624b Handle `DeploymentStatusFailed` unless `hasAutoRevert` 2021-07-15 17:06:13 -04:00
Mahmood Ali 996ea1fa46
Merge pull request #10875 from hashicorp/b-namespace-flag-override
cli: `-namespace` should override job namespace
2021-07-14 17:28:36 -04:00
Kent 'picat' Gruber 15342d0f6a Handle successful/canceled/blocked deployments in CLI output
Otherwise the spinner would just end, which felt a bit awkward.

I wanted to see a  "✓" to know that everything was ok, and a "!" (maybe something else?) if something went wrong.
2021-07-09 19:27:55 -04:00
Seth Hoenig 7c3db812fd consul/connect: remove sidecar proxy before removing parent service
This PR will have Nomad de-register a sidecar proxy service before
attempting to de-register the parent service. Otherwise, Consul will
emit a warning and an error.

Fixes #10845
2021-07-08 13:30:19 -05:00
Seth Hoenig 2607853a26
Merge pull request #10872 from hashicorp/b-cc-regex-checkids
consul/connect: Avoid assumption of parent service when filtering connect proxies
2021-07-08 13:29:40 -05:00
Seth Hoenig 284cd214ec consul/connect: improve regex from CR suggestions 2021-07-08 13:05:05 -05:00
Tim Gross a3bc87a2eb cli: `-namespace` should override job namespace
When a jobspec doesn't include a namespace, we provide it with the default
namespace, but this ends up overriding the explicit `-namespace` flag. This
changeset uses the same logic as region parsing to create an order of
precedence: the query string parameter (the `-namespace` flag) overrides the
API request body which overrides the jobspec.
2021-07-08 13:17:27 -04:00
Seth Hoenig 868b246128 consul/connect: Avoid assumption of parent service when filtering connect proxies
This PR uses regex-based matching for sidecar proxy services and checks when syncing
with Consul. Previously we would check if the parent of the sidecar was still being
tracked in Nomad. This is a false invariant - one which we must not depend when we
make #10845 work.

Fixes #10843
2021-07-08 09:43:41 -05:00
Mahmood Ali 1f34f2197b
Merge pull request #10806 from hashicorp/munda/idempotent-job-dispatch
Enforce idempotency of dispatched jobs using token on dispatch request
2021-07-08 10:23:31 -04:00
Tim Gross 8f25a9d7cd
cni: respect default `cni_config_dir` and `cni_path` (#10870)
The default agent configuration values were not set, which meant they were not
being set in the client configuration and this results in fingerprints failing
unless the values were set explicitly.
2021-07-08 09:56:57 -04:00
Tim Gross e88e1e5001
testing: prevent panic when `job status` output changes (#10869)
The `command/TestJobStatusCommand_Run` test assumes that it gets back running
allocations and will panic the test runner rather than failing.
2021-07-08 09:25:44 -04:00
Alex Munda 02c1a4d912
Set/parse idempotency_token query param 2021-07-07 16:26:55 -05:00
Seth Hoenig a57b066402
Merge pull request #10865 from hashicorp/b-deregister-noops
consul: avoid extra sync operations when no action required
2021-07-07 13:42:46 -05:00
Isabel Suchanek 13db600665
cli: add -task flag to alloc signal, restart (#10859)
Alloc exec only works when task is passed as a flag and not an arg.
Alloc logs currently accepts either, but alloc signal and restart only
accept task as an arg. This adds -task as a flag to the other alloc
commands to make the cli UX consistent. If task is passed as a flag and
an arg, it ignores the arg.
2021-07-07 09:58:16 -07:00
Seth Hoenig 56a6a1b1df consul: avoid extra sync operations when no action required
This PR makes it so the Consul sync logic will ignore operations that
do not specify an action to take (i.e. [de-]register [services|checks]).

Ideally such noops would be discarded at the callsites (i.e. users
of [Create|Update|Remove]Workload], but we can also be defensive
at the commit point.

Also adds 2 trace logging statements which are helpful for diagnosing
sync operations with Consul - when they happen and why.

Fixes #10797
2021-07-07 11:24:56 -05:00
Tim Gross 69a7c9db7e
csi: account for nil volume_mount in API-to-structs conversion (#10855)
Fix a nil pointer in the API struct to `nomad/structs` conversion when a
`volume_mount` block is empty.
2021-07-07 08:06:39 -04:00
James Rasell 7a89dfe0cb
cli: fixed system commands so they correctly use passed flags. 2021-06-28 10:57:50 +02:00
Tim Gross 38e83f5ddc
csi: fix CLI panic when formatting volume status with -verbose flag (#10818)
When the `-verbose` flag is passed to the `nomad volume status` command, we
hit a code path where the rows of text to be formatted were not initialized
correctly, resulting in a panic in the CLI.
2021-06-25 16:17:37 -04:00
Dave May 1e51d00d98
Add remaining pprof profiles to nomad operator debug (#10748)
* Add remaining pprof profiles to debug dump
* Refactor pprof profile capture
* Add WaitForFilesUntil and WaitForResultUntil utility functions
* Add CHANGELOG entry
2021-06-21 14:22:49 -04:00
Russell Rollins 76446ba512
Adds error handling for client error in getRandomJobAlloc. (#10787) 2021-06-18 16:26:43 -04:00
Seth Hoenig 0d9208f1a0 consul: set task name only for group service checks
This PR fixes a bug introduced in a refactoring

https://github.com/hashicorp/nomad/pull/10764/files#diff-56b3c82fcbc857f8fb93a903f1610f6e6859b3610a4eddf92bad9ea27fdc85ec

where task level service checks would inherent the task name
field, when they shouldn't.

Fixes #10781
2021-06-18 12:16:27 -05:00
Seth Hoenig 532b898b07 consul/connect: in-place update service definition when connect upstreams are modified
This PR fixes a bug where modifying the upstreams of a Connect sidecar proxy
would not result Consul applying the changes, unless an additional change to
the job would trigger a task replacement (thus replacing the service definition).

The fix is to check if upstreams have been modified between Nomad's view of the
sidecar service definition, and the service definition for the sidecar that is
actually registered in Consul.

Fixes #8754
2021-06-16 16:48:26 -05:00
Seth Hoenig d75669da4a consul: make failures_before_critical and success_before_passing work with group services
This PR fixes some job submission plumbing to make sure the Consul Check parameters
- failure_before_critical
- success_before_passing

work with group-level services. They already work with task-level services.
2021-06-15 11:20:40 -05:00
Isabel Suchanek e3cde4f4b3
cli: check deployment exists before monitoring (#10757)
System and batch jobs don't create deployments, which means nomad tries
to monitor a non-existent deployment when it runs a job and outputs an
error message. This adds a check to make sure a deployment exists before
monitoring. Also fixes some formatting.

Co-authored-by: Tim Gross <tgross@hashicorp.com>
2021-06-14 16:42:38 -07:00
Luiz Aoqui 98e0e952a6
fix agent-info help message formatting (#10747) 2021-06-11 15:39:28 -04:00
James Rasell 939b23936a
Merge pull request #10744 from hashicorp/b-remove-duplicate-imports
chore: remove duplicate import statements
2021-06-11 16:42:34 +02:00
James Rasell 492e308846
tests: remove duplicate import statements. 2021-06-11 09:39:22 +02:00
Nomad Release bot 7cc7389afd Generate files for 1.1.1 release 2021-06-10 08:04:25 -04:00
Mahmood Ali aa77c2731b tests: use standard library testing.TB
Glint pulled in an updated version of mitchellh/go-testing-interface
which broke some existing tests because the update added a Parallel()
method to testing.T. This switches to the standard library testing.TB
which doesn't have a Parallel() method.
2021-06-09 16:18:45 -07:00
Isabel Suchanek dfaef2468c cli: add monitor flag to deployment status
Adding '-verbose' will print out the allocation information for the
deployment. This also changes the job run command so that it now blocks
until deployment is complete and adds timestamps to the output so that
it's more in line with the output of node drain.

This uses glint to print in place in running in a tty. Because glint
doesn't yet support cmd/powershell, Windows workflows use a different
library to print in place, which results in slightly different
formatting: 1) different margins, and 2) no spinner indicating
deployment in progress.
2021-06-09 16:18:45 -07:00
Seth Hoenig dbdc479970 consul: move consul acl tests into ent files
(cherry-pick ent back to oss)

This PR moves a lot of Consul ACL token validation tests into ent files,
so that we can verify correct behavior difference between OSS and ENT
Nomad versions.
2021-06-09 08:38:42 -05:00