Commit graph

35 commits

Author SHA1 Message Date
Luiz Aoqui 15f9d54dea
api: prevent excessice CPU load on job parse
Add new namespace ACL requirement for the /v1/jobs/parse endpoint and
return early if HCLv2 parsing fails.

The endpoint now requires the new `parse-job` ACL capability or
`submit-job`.
2022-02-09 19:51:47 -05:00
Derek Strickland 0a8e03f0f7
Expose Consul template configuration parameters (#11606)
This PR exposes the following existing`consul-template` configuration options to Nomad jobspec authors in the `{job.group.task.template}` stanza.

- `wait`

It also exposes the following`consul-template` configuration to Nomad operators in the `{client.template}` stanza.

- `max_stale`
- `block_query_wait`
- `consul_retry`
- `vault_retry` 
- `wait` 

Finally, it adds the following new Nomad-specific configuration to the `{client.template}` stanza that allows Operators to set bounds on what `jobspec` authors configure.

- `wait_bounds`

Co-authored-by: Tim Gross <tgross@hashicorp.com>
Co-authored-by: Michael Schurter <mschurter@hashicorp.com>
2022-01-10 10:19:07 -05:00
James Rasell 257d63eec9
jobspec2: ensure consistent error handling between var-file & var. 2021-09-09 11:18:11 +02:00
James Rasell b6813f1221
chore: fix incorrect docstring formatting. 2021-08-30 11:08:12 +02:00
James Rasell 3b16e4d0b3
jobspec2: remove duplicate imports statements. 2021-06-11 09:38:47 +02:00
Drew Bailey a805cd8f52
check and return error from parsing var-files (#10569)
* check and return error from parsing var-files

* changelog entry for 1.1.0 and 1.0.5
2021-05-12 09:08:59 -04:00
Mahmood Ali 2d6c75a17f
hcl2: handle unquoted undefined variables (#10419)
This fixes a regression in #10326, to handle unquoted unknown variables.

The HCL job may contain unquoted undefined variable references without ${...} wrapping, e.g. value = meta.node_class. In 1.0.4, this got parsed as value = "${meta.node_class}".

This code performs a scan to find the relevant ${ and }, and only tries to find the closest ones with whitespace as the only separator.
2021-04-21 13:24:22 -04:00
Mahmood Ali a239224555
Merge pull request #10381 from hashicorp/f-cv-fix-missing-var-msg
Fix link in error for missing variable
2021-04-14 13:02:22 -04:00
Charlie Voiselle 2386c1348e
Fix link in error for missing variable 2021-04-14 12:55:00 -04:00
Nick Spain 4024c7562c Test parsing of body field in jobspec2 2021-04-13 09:15:35 -04:00
Mahmood Ali 04bfeacc5b hcl2: Use the actual string snippet as it appears from undefined
With the updated undefined variable code, we attempt to pick the text of
`${....}` verbatim from the hcl body. Previously, we'd attempt to
regenerate the string from the AST and pray it matches input; the
generation is lossy, as the same AST can represent multiple variations
(e.g. `${v.0}` and `${v[0]}` have the same HCLv2 AST). In this change,
we attempt to go back to the hcl2 source and find the string snippet
corresponding to the variable reference.
2021-04-07 16:45:37 -04:00
Mahmood Ali a190c3cbbe hcl2: add failing tests 2021-04-07 16:15:51 -04:00
Tim Gross daf1c8a0f6 hcl2: avoid panic on unset variable
Variables that are unset return the correct diagnostic but throw a panic when
we later parse the job body. Return early if there are any variable parsing
errors instead of continuing in a potentially invalid state.
2021-02-18 10:48:56 -05:00
Mahmood Ali c819bf3cad tests: add tests for invalid syntax cases 2021-02-01 12:51:51 -05:00
Mahmood Ali bb80ea741b hcl2: special case meta and env blocks
Allow expressing `meta` and `env` blocks as map attributes as well.
`env` and `meta` should support arbitrary key and values, yet hcl2
restricts the keys to valid identifiers. For example, block attribute
identifiers may not contain dots, `.`, which frequently used in meta
fields, and sometimes in environment variable fields.

This change attempts to parse `env`/`meta` both as an attribute and as a
block.

This additionally allows better expressivity for env/meta blocks, using
functions. For example, one can reuse a set of environment variables for
multiple tasks, using a local common_envs value:

```hcl
env = merge(local.common_envs, {"more_env_key", "..."})
```
2021-02-01 10:45:03 -05:00
Tim Gross 23cc8b4d77
interpolate maps in dynamic blocks (#9921)
Update hcl2 for patch to fix map interpolation in dynamic blocks
2021-01-29 10:16:07 -05:00
Kris Hicks 554f1e23de
jobspec2: decode returns error in more cases (#9613)
Previously, if decoding the job, tasks, or vault portion of the config
failed, we would not return an error; it was silently ignored.

This also includes a little refactor to reduce some duplication.
2020-12-16 07:49:09 -08:00
Kris Hicks 0cf9cae656
Apply some suggested fixes from staticcheck (#9598) 2020-12-10 07:29:18 -08:00
Mahmood Ali 1798523a78 appease deadcode linter 2020-11-12 11:44:49 -05:00
Mahmood Ali 73af6fd220 Restrict HCL special casing of map[string]interface{} fields
The HCL2 parser needs to apply special parsing tweaks so it can parse
the task config the same way as HCL1. Particularly, it needs to
reinterprets `map[string]interface{}` fields and blocks that appear when
attributes are expected.

This commit restricts the special casing to the Job fields, and ignore
`variables` and `locals` block.
2020-11-12 11:35:39 -05:00
Mahmood Ali 61650167bf use a constructor to initialize job config 2020-11-09 16:27:22 -05:00
Mahmood Ali 2e5ce5aa96 clarify test 2020-11-09 16:24:22 -05:00
Mahmood Ali a155423ba3 clarify variable references 2020-11-09 16:23:09 -05:00
Mahmood Ali 725e715c2e Update variable interpolation tests 2020-11-09 15:03:17 -05:00
Mahmood Ali 8f5bc0d8cd Parse variable files 2020-11-09 15:02:21 -05:00
Mahmood Ali bbe3d315d7 Restructure Variables and Locals
This reimplements the handling of Variables and Locals in HCL2 config.

This change supports declaring variables and locals, with defaults.
2020-11-09 14:58:57 -05:00
Mahmood Ali 9927a6b8cd copy variables files from Packer 2020-11-09 14:35:02 -05:00
Chris Baker 719077a26d added new policy capabilities for recommendations API
state store: call-out to generic update of job recommendations from job update method
recommendations API work, and http endpoint errors for OSS
support for scaling polices in task block of job spec
add query filters for ScalingPolicy list endpoint
command: nomad scaling policy list: added -job and -type
2020-10-28 14:32:16 +00:00
Mahmood Ali b8b993eea0 hclv2: handle block labels 2020-10-26 16:24:43 -04:00
Mahmood Ali 1d948bb4e8 hclv2: special case attributes with map values
HCLv1 special case map assignment:
```hcl
map_attr = { "K" = "V" }
```

is parsed as

```json
{"map_attr": [{"K": "V"}]}
```
2020-10-26 16:20:41 -04:00
Mahmood Ali 639fc3a88d floats always parsed as float64 2020-10-26 16:20:41 -04:00
Mahmood Ali e3d0d3a94e Refactor handling of blocks as attr 2020-10-26 16:20:41 -04:00
Mahmood Ali 0c56438486 hclv2 tests: test complex config configuration 2020-10-26 16:20:41 -04:00
Mahmood Ali d3a17b5c82 address review feedback 2020-10-22 11:49:37 -04:00
Mahmood Ali b3c118ae22 Add hclv2 parser 2020-10-21 15:46:56 -04:00