dbffdca92e
When a Nomad agent starts and loads jobs that already existed in the cluster, the default template uid and gid was being set to 0, since this is the zero value for int. This caused these jobs to fail in environments where it was not possible to use 0, such as in Windows clients. In order to differentiate between an explicit 0 and a template where these properties were not set we need to use a pointer.
582 lines
20 KiB
Plaintext
582 lines
20 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: template Stanza - Job Specification
|
|
description: |-
|
|
The "template" block instantiates an instance of a template renderer. This
|
|
creates a convenient way to ship configuration files that are populated from
|
|
environment variables, Consul data, Vault secrets, or just general
|
|
configurations within a Nomad task.
|
|
---
|
|
|
|
# `template` Stanza
|
|
|
|
<Placement groups={['job', 'group', 'task', 'template']} />
|
|
|
|
The `template` block instantiates an instance of a template renderer. This
|
|
creates a convenient way to ship configuration files that are populated from
|
|
environment variables, Consul data, Vault secrets, or just general
|
|
configurations within a Nomad task.
|
|
|
|
```hcl
|
|
job "docs" {
|
|
group "example" {
|
|
task "server" {
|
|
template {
|
|
source = "local/redis.conf.tpl"
|
|
destination = "local/redis.conf"
|
|
change_mode = "signal"
|
|
change_signal = "SIGINT"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Nomad utilizes [Go template][gt] and a tool called [Consul Template][ct], which
|
|
adds a set of new functions that can be used to retrieve data from Consul and
|
|
Vault. Since Nomad v0.5.3, the template can reference [Nomad's runtime
|
|
environment variables][env], and since Nomad v0.5.6, the template can reference
|
|
[Node attributes and metadata][nodevars]. Since Nomad v0.6.0, templates can be
|
|
read as environment variables.
|
|
|
|
For a full list of the API template functions, please refer to the [Consul
|
|
Template documentation][ct_api]. For a an introduction to Go templates, please
|
|
refer to the [Learn Go Template Syntax][gt_learn] Learn guide.
|
|
|
|
## `template` Parameters
|
|
|
|
- `change_mode` `(string: "restart")` - Specifies the behavior Nomad should take
|
|
if the rendered template changes. Nomad will always write the new contents of
|
|
the template to the specified destination. The possible values below describe
|
|
Nomad's action after writing the template to disk.
|
|
|
|
- `"noop"` - take no action (continue running the task)
|
|
- `"restart"` - restart the task
|
|
- `"signal"` - send a configurable signal to the task
|
|
|
|
- `change_signal` `(string: "")` - Specifies the signal to send to the task as a
|
|
string like `"SIGUSR1"` or `"SIGINT"`. This option is required if the
|
|
`change_mode` is `signal`.
|
|
|
|
- `data` `(string: "")` - Specifies the raw template to execute. One of `source`
|
|
or `data` must be specified, but not both. This is useful for smaller
|
|
templates, but we recommend using `source` for larger templates.
|
|
|
|
- `destination` `(string: <required>)` - Specifies the location where the
|
|
resulting template should be rendered, relative to the [task working
|
|
directory]. Only drivers without filesystem isolation (ex. `raw_exec`) or
|
|
that build a chroot in the task working directory (ex. `exec`) can render
|
|
templates outside of the `NOMAD_ALLOC_DIR`, `NOMAD_TASK_DIR`, or
|
|
`NOMAD_SECRETS_DIR`. For more details on how `destination` interacts with
|
|
task drivers, see the [Filesystem internals] documentation.
|
|
|
|
- `env` `(bool: false)` - Specifies the template should be read back in as
|
|
environment variables for the task ([see below](#environment-variables)). To
|
|
update the environment on changes, you must set `change_mode` to
|
|
`restart`. Setting `env` when the `change_mode` is `signal` will return a
|
|
validation error. Setting `env` when the `change_mode` is `noop` is
|
|
permitted but will not update the environment variables in the task.
|
|
|
|
- `left_delimiter` `(string: "{{")` - Specifies the left delimiter to use in the
|
|
template. The default is "{{" for some templates, it may be easier to use a
|
|
different delimiter that does not conflict with the output file itself.
|
|
|
|
- `perms` `(string: "644")` - Specifies the rendered template's permissions.
|
|
File permissions are given as octal of the Unix file permissions `rwxrwxrwx`.
|
|
|
|
- `uid` `(int: nil)` - Specifies the rendered template owner's user ID. If
|
|
negative or not specified (`nil`) the ID of the Nomad agent user wil be used.
|
|
|
|
~> **Caveat:** Works only on Unix-based systems. Be careful when using
|
|
containerized drivers, such as `docker` or `podman`, as groups and users
|
|
inside the container may have different IDs than on the host system. This
|
|
feature will also **not** work with Docker Desktop.
|
|
|
|
- `gid` `(int: nil)` - Specifies the rendered template owner's group ID. If
|
|
negative or not specified (`nil`) the ID of the Nomad agent group will be
|
|
used.
|
|
|
|
~> **Caveat:** Works only on Unix-based systems. Be careful when using
|
|
containerized drivers, such as `docker` or `podman`, as groups and users
|
|
inside the container may have different IDs than on the host system. This
|
|
feature will also **not** work with Docker Desktop.
|
|
|
|
- `right_delimiter` `(string: "}}")` - Specifies the right delimiter to use in the
|
|
template. The default is "}}" for some templates, it may be easier to use a
|
|
different delimiter that does not conflict with the output file itself.
|
|
|
|
- `source` `(string: "")` - Specifies the path to the template to be rendered.
|
|
One of `source` or `data` must be specified, but not both. This source can
|
|
optionally be fetched using an [`artifact`][artifact] resource. This template
|
|
must exist on the machine prior to starting the task; it is not possible to
|
|
reference a template inside a Docker container, for example.
|
|
|
|
- `splay` `(string: "5s")` - Specifies a random amount of time to wait between
|
|
0 ms and the given splay value before invoking the change mode. This is
|
|
specified using a label suffix like "30s" or "1h", and is often used to
|
|
prevent a thundering herd problem where all task instances restart at the same
|
|
time.
|
|
|
|
- `wait` `(Code: nil)` - Defines the minimum and maximum amount of time to wait
|
|
for the Consul cluster to reach a consistent state before rendering a template.
|
|
This is useful to enable in systems where network connectivity to Consul is degraded,
|
|
because it will reduce the number of times a template is rendered. This setting
|
|
can be overridden by the [`client.template.wait_bounds`]. If the template
|
|
configuration has a `min` lower than `client.template.wait_bounds.min` or a `max`
|
|
greater than `client.template.wait_bounds.max`, the client's bounds will be enforced,
|
|
and the template `wait` will be adjusted before being sent to `consul-template`.
|
|
|
|
```hcl
|
|
wait {
|
|
min = "5s"
|
|
max = "10s"
|
|
}
|
|
```
|
|
|
|
- `vault_grace` `(string: "15s")` - [Deprecated](https://github.com/hashicorp/consul-template/issues/1268)
|
|
|
|
## `template` Examples
|
|
|
|
The following examples only show the `template` stanzas. Remember that the
|
|
`template` stanza is only valid in the placements listed above.
|
|
|
|
### Inline Template
|
|
|
|
This example uses an inline template to render a file to disk. This file watches
|
|
various keys in Consul for changes:
|
|
|
|
```hcl
|
|
template {
|
|
data = "---\nkey: {{ key \"service/my-key\" }}"
|
|
destination = "local/file.yml"
|
|
}
|
|
```
|
|
|
|
It is also possible to use heredocs for multi-line templates, like:
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOH
|
|
---
|
|
bind_port: {{ env "NOMAD_PORT_db" }}
|
|
scratch_dir: {{ env "NOMAD_TASK_DIR" }}
|
|
node_id: {{ env "node.unique.id" }}
|
|
service_key: {{ key "service/my-key" }}
|
|
EOH
|
|
|
|
destination = "local/file.yml"
|
|
}
|
|
```
|
|
|
|
### Remote Template
|
|
|
|
This example uses an [`artifact`][artifact] stanza to download an input template
|
|
before passing it to the template engine:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.yml.tpl"
|
|
destination = "local/file.yml.tpl"
|
|
}
|
|
|
|
template {
|
|
source = "local/file.yml.tpl"
|
|
destination = "local/file.yml"
|
|
}
|
|
```
|
|
|
|
### Node Variables
|
|
|
|
As of Nomad v0.5.6 it is possible to access the Node's attributes and metadata.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOH
|
|
---
|
|
node_dc: {{ env "node.datacenter" }}
|
|
node_cores: {{ env "attr.cpu.numcores" }}
|
|
meta_key: {{ env "meta.node_meta_key" }}
|
|
EOH
|
|
|
|
destination = "local/file.yml"
|
|
}
|
|
```
|
|
|
|
### Environment Variables
|
|
|
|
Since v0.6.0 templates may be used to create environment variables for tasks.
|
|
Env templates work exactly like other templates except once the templates are
|
|
written, they are parsed as `KEY=value` pairs. Those key value pairs are
|
|
included in the task's environment.
|
|
|
|
For example the following template stanza:
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOH
|
|
# Lines starting with a # are ignored
|
|
|
|
# Empty lines are also ignored
|
|
LOG_LEVEL="{{key "service/geo-api/log-verbosity"}}"
|
|
API_KEY="{{with secret "secret/geo-api-key"}}{{.Data.value}}{{end}}"
|
|
EOH
|
|
|
|
destination = "secrets/file.env"
|
|
env = true
|
|
}
|
|
```
|
|
|
|
The task's environment would then have environment variables like the
|
|
following:
|
|
|
|
```
|
|
LOG_LEVEL=DEBUG
|
|
API_KEY=12345678-1234-1234-1234-1234-123456789abc
|
|
```
|
|
|
|
This allows [12factor app](https://12factor.net/config) style environment
|
|
variable based configuration while keeping all of the familiar features and
|
|
semantics of Nomad templates.
|
|
|
|
Secrets or certificates may contain a wide variety of characters such as
|
|
newlines, quotes, and backslashes which may be difficult to quote or escape
|
|
properly.
|
|
|
|
Whenever a templated variable may include special characters, use the `toJSON`
|
|
function to ensure special characters are properly parsed by Nomad:
|
|
|
|
```
|
|
CERT_PEM={{ file "path/to/cert.pem" | toJSON }}
|
|
```
|
|
|
|
The parser will read the JSON string, so the `$CERT_PEM` environment variable
|
|
will be identical to the contents of the file.
|
|
|
|
Likewise when evaluating a password that may contain quotes or `#`, use the
|
|
`toJSON` function to ensure Nomad passes the password to task unchanged:
|
|
|
|
```
|
|
# Passwords may contain any character including special characters like:
|
|
# \"'#
|
|
# Use toJSON to ensure Nomad passes them to the environment unchanged.
|
|
{{ with secret "secrets/data/application/backend" }}
|
|
DB_PASSWD={{ .Data.data.DB_PASSWD | toJSON }}
|
|
{{ end }}
|
|
```
|
|
|
|
For more details see [go-envparser's README][go-envparse].
|
|
|
|
### Template Destinations
|
|
|
|
Templates are rendered into the task working directory. Drivers without
|
|
filesystem isolation (such as `raw_exec`) or drivers that build a chroot in
|
|
the task working directory (such as `exec`) can have templates rendered to
|
|
arbitrary paths in the task. But task drivers such as `docker` can only access
|
|
templates rendered into the `NOMAD_ALLOC_DIR`, `NOMAD_TASK_DIR`, or
|
|
`NOMAD_SECRETS_DIR`. To workaround this restriction, you can create a mount
|
|
from the template `destination` to another location in the task.
|
|
|
|
```hcl
|
|
task "task" {
|
|
driver = "docker"
|
|
|
|
config {
|
|
image = "redis:6.0"
|
|
mount {
|
|
type = "bind"
|
|
source = "local"
|
|
target = "/etc/redis.d"
|
|
}
|
|
}
|
|
|
|
template {
|
|
destination = "local/redis.conf"
|
|
}
|
|
}
|
|
```
|
|
|
|
## Nomad Integration
|
|
|
|
### Nomad Services
|
|
|
|
~> Nomad Services are new in Nomad 1.3.
|
|
|
|
Nomad service registrations can be queried using the `nomadService` and
|
|
`nomadServices` functions. The requests are tied to the same namespace as the
|
|
job which contains the template stanza.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOF
|
|
# Configuration for a single NGINX upstream service.
|
|
upstream my_app {
|
|
{{- range nomadService "my-app" }}
|
|
server {{ .Address }}:{{ .Port }};{{- end }}
|
|
}
|
|
|
|
# Configuration for all services registered in Nomad as an NGINX upstream
|
|
# service.
|
|
{{ range nomadServices }}
|
|
# Configuration for service {{ .Name }}.
|
|
upstream {{ .Name | toLower }} {
|
|
{{- range nomadService .Name }}
|
|
server {{ .Address}}:{{ .Port }};{{- end }}
|
|
}
|
|
{{ end -}}
|
|
EOF
|
|
|
|
destination = "local/nginx.conf"
|
|
}
|
|
```
|
|
|
|
### Simple Load Balancing with Nomad Services
|
|
|
|
~> Simple load balancing with Nomad Services is new in Nomad 1.3.2.
|
|
|
|
The `nomadService` function now supports simple load balancing by selecting
|
|
instances of a service via [rendezvous hashing][rhash].
|
|
To enable simple load balancing, the `nomadService` function requires 3 arguments.
|
|
|
|
- The number of services to select
|
|
- The hashing key (should be unique, but consistent per requester)
|
|
- The service name
|
|
|
|
By using `NOMAD_ALLOC_ID` as the hashing key, the selected instances will remain
|
|
mostly stable for the allocation. Each time the template is run, `nomadService`
|
|
will return the same set of instances for each allocation - unless N instances of
|
|
the service are added or removed, in which case there is a 1/N chance of a selected
|
|
instance being replaced. This helps maintain a more consistent output when rendering
|
|
configuration files, triggering fewer restarts and signaling of Nomad tasks.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOH
|
|
# Configuration for 1 redis instances, as assigned via rendezvous hashing.
|
|
{{$allocID := env "NOMAD_ALLOC_ID" -}}
|
|
{{range nomadService 1 $allocID "redis"}}
|
|
server {{ .Address }}:{{ .Port }};{{- end }}
|
|
{{- end}}
|
|
EOH
|
|
}
|
|
```
|
|
|
|
## Consul Integration
|
|
|
|
### Consul KV
|
|
|
|
Consul KV values can be accessed using the [`key`][ct_api_key] function to
|
|
retrieve a single value from a key path. The [`ls`][ct_api_ls] function can be
|
|
used to retrieve all keys in a path. For deeply nested paths, use the
|
|
[`tree`][ct_api_tree] function.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOF
|
|
# Read single key from Consul KV.
|
|
APP_NAME = "{{key "app/name"}}"
|
|
|
|
# Read all keys in the path `app/environment` from Consul KV.
|
|
{{range ls "app/environment"}}
|
|
{{.Key}}={{.Value}}
|
|
{{end}}
|
|
EOF
|
|
|
|
destination = "local/env"
|
|
env = true
|
|
}
|
|
```
|
|
|
|
### Consul Services
|
|
|
|
The Consul service catalog can be queried using the [`service`][ct_api_service]
|
|
and [`services`][ct_api_services] functions. For Connect-capable services, use
|
|
the [`connect`][ct_api_connect] function.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOF
|
|
# Configuration for a single upstream service.
|
|
upstream my_app {
|
|
{{- range service "my-app" }}
|
|
server {{ .Address }}:{{ .Port }};{{- end }}
|
|
}
|
|
|
|
# Configuration for all services in the catalog.
|
|
{{ range services }}
|
|
# Configuration for service {{ .Name }}.
|
|
upstream {{ .Name | toLower }} {
|
|
{{- range service .Name }}
|
|
server {{ .Address}}:{{ .Port }};{{- end }}
|
|
}
|
|
{{ end -}}
|
|
EOF
|
|
|
|
destination = "local/nginx.conf"
|
|
}
|
|
```
|
|
|
|
## Vault Integration
|
|
|
|
### PKI Certificate
|
|
|
|
Vault is a popular open source tool for managing secrets. In addition to acting
|
|
as an encrypted KV store, Vault can also generate dynamic secrets, like PKI/TLS
|
|
certificates.
|
|
|
|
When generating PKI certificates with Vault, the certificate, private key, and
|
|
any intermediate certs are all returned as part of the same API call. Most
|
|
software requires these files be placed in separate files on the system.
|
|
|
|
~> **Note**: `generate_lease` must be set to `true` (non-default) on the Vault PKI
|
|
role.<br /><br /> Failure to do so will cause the template to frequently render a new
|
|
certificate, approximately every minute. This creates a significant number of
|
|
certificates to be expired in Vault and could ultimately lead to Vault performance
|
|
impacts and failures.
|
|
|
|
#### As individual files
|
|
|
|
For templates, all dependencies are mapped into a single list. This means that
|
|
multiple templates watching the same path return the same data.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOH
|
|
{{ with secret "pki/issue/foo" "common_name=foo.service.consul" "ip_sans=127.0.0.1" }}
|
|
{{- .Data.certificate -}}
|
|
{{ end }}
|
|
EOH
|
|
destination = "${NOMAD_SECRETS_DIR}/certificate.crt"
|
|
change_mode = "restart"
|
|
}
|
|
|
|
template {
|
|
data = <<EOH
|
|
{{ with secret "pki/issue/foo" "common_name=foo.service.consul" "ip_sans=127.0.0.1" }}
|
|
{{- .Data.issuing_ca -}}
|
|
{{ end }}
|
|
EOH
|
|
destination = "${NOMAD_SECRETS_DIR}/ca.crt"
|
|
change_mode = "restart"
|
|
}
|
|
|
|
template {
|
|
data = <<EOH
|
|
{{ with secret "pki/issue/foo" "common_name=foo.service.consul" "ip_sans=127.0.0.1" }}
|
|
{{- .Data.private_key -}}
|
|
{{ end }}
|
|
EOH
|
|
destination = "${NOMAD_SECRETS_DIR}/private_key.key"
|
|
change_mode = "restart"
|
|
}
|
|
```
|
|
|
|
These are three different input templates, but when run under the Nomad job,
|
|
they are compressed into a single call, sharing the resulting data.
|
|
|
|
#### As a PEM formatted file
|
|
|
|
This example acquires a PKI certificate from Vault in PEM format, concatenates
|
|
the elements into a bundle, and stores it into your application's secret
|
|
directory.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOH
|
|
{{ with secret "pki/issue/foo" "common_name=foo.service.consul" "ip_sans=127.0.0.1" "format=pem" }}
|
|
{{ .Data.certificate }}
|
|
{{ .Data.issuing_ca }}
|
|
{{ .Data.private_key }}{{ end }}
|
|
EOH
|
|
destination = "${NOMAD_SECRETS_DIR}/bundle.pem"
|
|
change_mode = "restart"
|
|
}
|
|
```
|
|
|
|
### Vault KV API v1
|
|
|
|
Under Vault KV API v1, paths start with `secret/`, and the response returns the
|
|
raw key/value data. This secret was set using
|
|
`vault kv put secret/aws/s3 aws_access_key_id=somekeyid`.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOF
|
|
AWS_ACCESS_KEY_ID = "{{with secret "secret/aws/s3"}}{{.Data.aws_access_key_id}}{{end}}"
|
|
EOF
|
|
}
|
|
```
|
|
|
|
Note that if the name of a secret includes the `-` character, you must access
|
|
it by index. This secret was set using `vault kv put secret/app db-password=somepassword`.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOF
|
|
DB_PASSWORD = "{{with secret "secret/app"}}{{index .Data "db-password"}}{{end}}"
|
|
EOF
|
|
}
|
|
```
|
|
|
|
### Vault KV API v2
|
|
|
|
Under Vault KV API v2, paths start with `secret/data/`, and the response returns
|
|
metadata in addition to key/value data. This secret was set using
|
|
`vault kv put secret/aws/s3 aws_access_key_id=somekeyid`.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOF
|
|
AWS_ACCESS_KEY_ID = "{{with secret "secret/data/aws/s3"}}{{.Data.data.aws_access_key_id}}{{end}}"
|
|
EOF
|
|
}
|
|
```
|
|
|
|
Notice the addition of `data` in both the path and the field accessor string.
|
|
Additionally, when using the Vault v2 API, the Vault policies applied to your
|
|
Nomad jobs will need to grant permissions to `read` under `secret/data/...`
|
|
rather than `secret/...`.
|
|
|
|
Similar to KV API v1, if the name of a secret includes the `-` character, you
|
|
must access it by index. This secret was set using `vault kv put secret/app db-password=somepassword`.
|
|
|
|
```hcl
|
|
template {
|
|
data = <<EOF
|
|
DB_PASSWORD = "{{with secret "secret/data/app"}}{{index .Data.data "db-password"}}{{end}}"
|
|
EOF
|
|
}
|
|
```
|
|
|
|
## Client Configuration
|
|
|
|
The `template` block has the following [client configuration
|
|
options](/docs/configuration/client#options):
|
|
|
|
- `function_denylist` `([]string: ["plugin"])` - Specifies a list of template
|
|
rendering functions that should be disallowed in job specs. By default the
|
|
`plugin` function is disallowed as it allows running arbitrary commands on
|
|
the host as root (unless Nomad is configured to run as a non-root user).
|
|
|
|
- `disable_file_sandbox` `(bool: false)` - Allows templates access to arbitrary
|
|
files on the client host via the `file` function. By default templates can
|
|
access files only within the [task working directory].
|
|
|
|
[ct]: https://github.com/hashicorp/consul-template 'Consul Template by HashiCorp'
|
|
[ct_api]: https://github.com/hashicorp/consul-template/blob/master/docs/templating-language.md 'Consul Template API by HashiCorp'
|
|
[ct_api_connect]: https://github.com/hashicorp/consul-template/blob/master/docs/templating-language.md#connect 'Consul Template API by HashiCorp - connect'
|
|
[ct_api_key]: https://github.com/hashicorp/consul-template/blob/master/docs/templating-language.md#key 'Consul Template API by HashiCorp - key'
|
|
[ct_api_ls]: https://github.com/hashicorp/consul-template/blob/master/docs/templating-language.md#ls 'Consul Template API by HashiCorp - ls'
|
|
[ct_api_service]: https://github.com/hashicorp/consul-template/blob/master/docs/templating-language.md#service 'Consul Template API by HashiCorp - service'
|
|
[ct_api_services]: https://github.com/hashicorp/consul-template/blob/master/docs/templating-language.md#services 'Consul Template API by HashiCorp - services'
|
|
[ct_api_tree]: https://github.com/hashicorp/consul-template/blob/master/docs/templating-language.md#tree 'Consul Template API by HashiCorp - tree'
|
|
[gt]: https://pkg.go.dev/text/template 'Go template package'
|
|
[gt_learn]: https://learn.hashicorp.com/tutorials/nomad/go-template-syntax
|
|
[artifact]: /docs/job-specification/artifact 'Nomad artifact Job Specification'
|
|
[env]: /docs/runtime/environment 'Nomad Runtime Environment'
|
|
[nodevars]: /docs/runtime/interpolation#interpreted_node_vars 'Nomad Node Variables'
|
|
[go-envparse]: https://github.com/hashicorp/go-envparse#readme 'The go-envparse Readme'
|
|
[task working directory]: /docs/runtime/environment#task-directories 'Task Directories'
|
|
[filesystem internals]: /docs/concepts/filesystem#templates-artifacts-and-dispatch-payloads
|
|
[`client.template.wait_bounds`]: /docs/configuration/client#wait_bounds
|
|
[rhash]: https://en.wikipedia.org/wiki/Rendezvous_hashing
|