Commit Graph

296 Commits

Author SHA1 Message Date
Seth Hoenig 2631659551 ci: swap ci parallelization for unconstrained gomaxprocs 2022-03-15 12:58:52 -05:00
Seth Hoenig ffe7f87912 connect: fix bug where sidecar_task.resources was ignored with hcl1
The HCL1 parser did not respect connect.sidecar_task.resources if the
connect.sidecar_service block was not set (an optimiztion that no longer
makes sense with connect gateways).

Fixes #10899
2022-01-25 10:17:54 -06:00
Charlie Voiselle 71643263a6
Parse `job > group > consul` block in HCL1 (#11423) 2021-11-03 13:49:32 -04:00
James Rasell 0e926ef3fd
allow configuration of Docker hostnames in bridge mode (#11173)
Add a new hostname string parameter to the network block which
allows operators to specify the hostname of the network namespace.
Changing this causes a destructive update to the allocation and it
is omitted if empty from API responses. This parameter also supports
interpolation.

In order to have a hostname passed as a configuration param when
creating an allocation network, the CreateNetwork func of the
DriverNetworkManager interface needs to be updated. In order to
minimize the disruption of future changes, rather than add another
string func arg, the function now accepts a request struct along with
the allocID param. The struct has the hostname as a field.

The in-tree implementations of DriverNetworkManager.CreateNetwork
have been modified to account for the function signature change.
In updating for the change, the enhancement of adding hostnames to
network namespaces has also been added to the Docker driver, whilst
the default Linux manager does not current implement it.
2021-09-16 08:13:09 +02:00
Mahmood Ali e0efe2f09d hcl: add new cores resources field 2021-07-26 11:45:10 -04:00
Mahmood Ali 63aa8157aa support gateway mesh 2021-07-22 17:10:59 -04:00
Mahmood Ali 715380d53c add a test for envoy_dns_discovery_type 2021-07-22 16:06:48 -04:00
Mahmood Ali d75a4db0a7 parse service check body 2021-07-22 16:06:48 -04:00
Mahmood Ali 17756a1e90 services OnUpdate 2021-07-22 16:06:48 -04:00
Mahmood Ali 22f786d375 hclv1: parse service upstreams 2021-07-22 16:06:48 -04:00
Mahmood Ali 7ac28d74e6 test parsing more service fields 2021-07-22 16:06:26 -04:00
Holt Wilkins c3b2a72ac4 Enable parsing of terminating gateways 2021-06-30 05:34:16 +00:00
Mahmood Ali 102763c979
Support disabling TCP checks for connect sidecar services 2021-05-07 12:10:26 -04:00
Mahmood Ali 04a96016e4 oversubscription: add memory_max to hclv1
Allow specifying the `memory_max` field in HCL under the resources block.

Though HCLv1 is deprecated, I've updated them to ease our testing.
2021-03-30 16:55:58 -04:00
Seth Hoenig 29d0e45961
Merge pull request #10075 from mr-karan/fix/parse_service
fix: parse_service: return err instead of panic
2021-02-24 10:14:55 -06:00
Karan Sharma 4591f68496 fix: parse_service: return err instead of panic
Fixes https://github.com/hashicorp/nomad/issues/10072
2021-02-23 22:15:11 +05:30
Drew Bailey b5585882e4
address pr comments 2021-02-08 13:43:05 -05:00
Drew Bailey 8507d54e3b
e2e test for on_update service checks
check_restart not compatible with on_update=ignore

reword caveat
2021-02-08 08:32:40 -05:00
Drew Bailey 82f971f289
OnUpdate configuration for services and checks
Allow for readiness type checks by configuring nomad to ignore warnings
or errors reported by a service check. This allows the deployment to
progress and while Consul handles introducing the sercive into a
resource pool once the check passes.
2021-02-08 08:32:40 -05:00
Kris Hicks 0a3a748053
Add gosimple linter (#9590) 2020-12-09 11:05:18 -08:00
Kris Hicks 93155ba3da
Add gocritic to golangci-lint config (#9556) 2020-12-08 12:47:04 -08:00
Chris Baker 719077a26d added new policy capabilities for recommendations API
state store: call-out to generic update of job recommendations from job update method
recommendations API work, and http endpoint errors for OSS
support for scaling polices in task block of job spec
add query filters for ScalingPolicy list endpoint
command: nomad scaling policy list: added -job and -type
2020-10-28 14:32:16 +00:00
Mahmood Ali 0c56438486 hclv2 tests: test complex config configuration 2020-10-26 16:20:41 -04:00
Mahmood Ali 1ae924973e hclv1: tweak HCLv1 tests
This ensures that gatway ReadOnly key is tested.  Also, update the hclv1
test-fixtures to be hclv1 compliant.
2020-10-21 14:05:46 -04:00
Mahmood Ali 618388d1c3 api: parse service gateway name
Adding gateway name eases HCLv2 parsing. This field is only used for parsing the
job and is ignored for any other pruposes
2020-10-21 14:05:46 -04:00
Mahmood Ali af8cab3d74
Isolate the jobspec package from the rest of Nomad (#8815)
This eases adoption of the jobspec package by other projects (e.g. terraform nomad provider, Lavant). Either by consuming directy as a library (hopefully without having go mod import rest of nomad) or by copying the package without modification.

Ideally, this package will be published as an independent module. We aren't ready for that considering we'll be switching to HCLv2 "soon", but eitherway, this seems like a reasonable intermediate step if we choose to.
2020-09-03 06:34:04 -05:00
Seth Hoenig c4fa644315 consul/connect: remove envoy dns option from gateway proxy config 2020-08-24 09:11:55 -05:00
Seth Hoenig 5b072029f2 consul/connect: add initial support for ingress gateways
This PR adds initial support for running Consul Connect Ingress Gateways (CIGs) in Nomad. These gateways are declared as part of a task group level service definition within the connect stanza.

```hcl
service {
  connect {
    gateway {
      proxy {
        // envoy proxy configuration
      }
      ingress {
        // ingress-gateway configuration entry
      }
    }
  }
}
```

A gateway can be run in `bridge` or `host` networking mode, with the caveat that host networking necessitates manually specifying the Envoy admin listener (which cannot be disabled) via the service port value.

Currently Envoy is the only supported gateway implementation in Consul, and Nomad only supports running Envoy as a gateway using the docker driver.

Aims to address #8294 and tangentially #8647
2020-08-21 16:21:54 -05:00
Seth Hoenig 546a724d1c consul: grrrrr hclfmt test resource file 2020-08-10 14:08:09 -05:00
Seth Hoenig fd4804bf26 consul: able to set pass/fail thresholds on consul service checks
This change adds the ability to set the fields `success_before_passing` and
`failures_before_critical` on Consul service check definitions. This is a
feature added to Consul v1.7.0 and later.
  https://www.consul.io/docs/agent/checks#success-failures-before-passing-critical

Nomad doesn't do much besides pass the fields through to Consul.

Fixes #6913
2020-08-10 14:08:09 -05:00
Drew Bailey b296558b8e
oss compoments for multi-vault namespaces
adds in oss components to support enterprise multi-vault namespace feature

upgrade specific doc on vault multi-namespaces

vault docs

update test to reflect new error
2020-07-24 10:14:59 -04:00
Chris Baker a77e012220 better testing of scaling parsing, fixed some broken tests by api
changes
2020-07-04 19:32:37 +00:00
Chris Baker 9100b6b7c0 changes to make sure that Max is present and valid, to improve error messages
* made api.Scaling.Max a pointer, so we can detect (and complain) when it is neglected
* added checks to HCL parsing that it is present
* when Scaling.Max is absent/invalid, don't return extraneous error messages during validation
* tweak to multiregion handling to ensure that the count is valid on the interpolated regional jobs

resolves #8355
2020-07-04 19:05:50 +00:00
Seth Hoenig 011c6b027f connect/native: doc and comment tweaks from PR 2020-06-24 10:13:22 -05:00
Seth Hoenig 6c5ab7f45e consul/connect: split connect native flag and task in service 2020-06-23 10:22:22 -05:00
Seth Hoenig 4d71f22a11 consul/connect: add support for running connect native tasks
This PR adds the capability of running Connect Native Tasks on Nomad,
particularly when TLS and ACLs are enabled on Consul.

The `connect` stanza now includes a `native` parameter, which can be
set to the name of task that backs the Connect Native Consul service.

There is a new Client configuration parameter for the `consul` stanza
called `share_ssl`. Like `allow_unauthenticated` the default value is
true, but recommended to be disabled in production environments. When
enabled, the Nomad Client's Consul TLS information is shared with
Connect Native tasks through the normal Consul environment variables.
This does NOT include auth or token information.

If Consul ACLs are enabled, Service Identity Tokens are automatically
and injected into the Connect Native task through the CONSUL_HTTP_TOKEN
environment variable.

Any of the automatically set environment variables can be overridden by
the Connect Native task using the `env` stanza.

Fixes #6083
2020-06-22 14:07:44 -05:00
Nick Ethier f0559a8162
multi-interface network support 2020-06-19 09:42:10 -04:00
Nick Ethier 0bc0403cc3 Task DNS Options (#7661)
Co-Authored-By: Tim Gross <tgross@hashicorp.com>
Co-Authored-By: Seth Hoenig <shoenig@hashicorp.com>
2020-06-18 11:01:31 -07:00
Tim Gross 7b12445f29 multiregion: change AutoRevert to OnFailure 2020-06-17 11:05:45 -04:00
Drew Bailey 9263fcb0d3 Multiregion deploy status and job status CLI 2020-06-17 11:03:34 -04:00
Tim Gross 6851024925 Multiregion structs
Initial struct definitions, jobspec parsing, validation, and conversion
between Nomad structs and API structs for multi-region deployments.
2020-06-17 11:00:14 -04:00
Seth Hoenig 889e7ddd0c build: use hashicorp hclfmt
We have been using fatih/hclfmt which is long abandoned. Instead, switch
to HashiCorp's own hclfmt implementation. There are some trivial changes in
behavior around whitespace.
2020-05-24 18:31:57 -05:00
Lang Martin d3c4700cd3
server: stop after client disconnect (#7939)
* jobspec, api: add stop_after_client_disconnect

* nomad/state/state_store: error message typo

* structs: alloc methods to support stop_after_client_disconnect

1. a global AllocStates to track status changes with timestamps. We
   need this to track the time at which the alloc became lost
   originally.

2. ShouldClientStop() and WaitClientStop() to actually do the math

* scheduler/reconcile_util: delayByStopAfterClientDisconnect

* scheduler/reconcile: use delayByStopAfterClientDisconnect

* scheduler/util: updateNonTerminalAllocsToLost comments

This was setup to only update allocs to lost if the DesiredStatus had
already been set by the scheduler. It seems like the intention was to
update the status from any non-terminal state, and not all lost allocs
have been marked stop or evict by now

* scheduler/testing: AssertEvalStatus just use require

* scheduler/generic_sched: don't create a blocked eval if delayed

* scheduler/generic_sched_test: several scheduling cases
2020-05-13 16:39:04 -04:00
Tim Gross bad9a82df8
ci: add a linting check for HCL files (#7791)
Running `make dev` runs `hclfmt`, but this isn't checked as part of
CI. That makes it possible to merge un-formatted HCL and Nomad
jobspecs that later will make for dirty git staging areas when
developers pull master.

This changeset adds HCL linting to the `make check` target.
2020-04-23 14:32:44 -04:00
Chris Baker 8ea4a7e84b return parsing error if scaling policy includes more than one policy block
also, check that parsing a minimal scaling block doesn't throw any errors
2020-04-23 12:37:45 +00:00
Seth Hoenig 40e0f8a346
Merge pull request #7690 from hashicorp/b-inspect-proxy-output
two fixes for inspect on connect proxy
2020-04-20 10:17:54 -06:00
Anthony Scalisi 9664c6b270
fix spelling errors (#6985) 2020-04-20 09:28:19 -04:00
Seth Hoenig 7e3b16fa90 jobspec: correctly parse proxy fields from jobspec
Before, the proxy stanza did not parse non-object fields
`local_service_port` and `local_service_address` from the
connect `proxy` stanza. This change fixes that.
2020-04-13 15:59:45 -06:00
Seth Hoenig 0407eaaf88 connect: extract common task keys 2020-04-10 09:49:19 -06:00
Seth Hoenig db865e05d8 connect: enable configuring sidecar_task.name
Before, the submitted jobspec for sidecar_task would pass
through 2 key validation steps - once for the subset specific
to connect sidecar task definitions, and once again for the set
of normal task definition where the task would actually get
unmarshalled.

The valid keys for the normal task definition did not include
"name", which is supposed to be configurable for the sidecar
task. To fix this, just eliminate the double validation step,
and instead pass-in the correct set of keys to validate against
to the one generic task parser.

Fixes #7680
2020-04-09 21:01:16 -06:00