open-nomad/website/content/docs/job-specification/parameterized.mdx

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

161 lines
4.8 KiB
Plaintext
Raw Normal View History

2017-01-26 03:14:58 +00:00
---
2020-02-06 23:45:31 +00:00
layout: docs
page_title: parameterized Block - Job Specification
2017-01-26 03:14:58 +00:00
description: |-
2020-02-06 23:45:31 +00:00
A parameterized job is used to encapsulate a set of work that can be carried
out on various inputs much like a function definition. When the
`parameterized` block is added to a job, the job acts as a function to the
2020-02-06 23:45:31 +00:00
cluster as a whole.
2017-01-26 03:14:58 +00:00
---
# `parameterized` Block
2017-01-26 03:14:58 +00:00
2020-02-06 23:45:31 +00:00
<Placement groups={['job', 'parameterized']} />
2017-01-26 03:14:58 +00:00
A parameterized job is used to encapsulate a set of work that can be carried out
on various inputs much like a function definition. When the `parameterized`
block is added to a job, the job acts as a function to the cluster as a whole.
2017-01-26 03:14:58 +00:00
The `parameterized` block allows job operators to configure a job that carries
2017-01-26 03:14:58 +00:00
out a particular action, define its resource requirements and configure how
inputs and configuration are retrieved by the tasks within the job.
2017-01-26 03:14:58 +00:00
2020-02-06 23:45:31 +00:00
To invoke a parameterized job, [`nomad job dispatch`][dispatch command] or the equivalent HTTP APIs are
2017-01-26 03:14:58 +00:00
used. When dispatching against a parameterized job, an opaque payload and
metadata may be injected into the job. These inputs to the parameterized job act
like arguments to a function. The job consumes them to change its behavior,
2017-02-06 18:59:07 +00:00
without exposing the implementation details to the caller.
2017-01-26 03:14:58 +00:00
2017-01-26 04:49:06 +00:00
To that end, tasks within the job can add a
[`dispatch_payload`][dispatch_payload] block that
2017-01-26 03:14:58 +00:00
defines where on the filesystem this payload gets written to. An example payload
would be a task's JSON configuration.
Further, certain metadata may be marked as required when dispatching a job so it
can be used to inject configuration directly into a task's arguments using
[interpolation]. An example of this would be to require a run ID key that
could be used to lookup the work the job is suppose to do from a management
service or database.
Each time a job is dispatched, a unique job ID is generated. This
allows a caller to track the status of the job, much like a future or
promise in some programming languages. The dispatched job cannot be
updated after dispatching; to update the job definition you need to
update the parent job.
2017-01-26 03:14:58 +00:00
```hcl
job "docs" {
parameterized {
payload = "required"
2017-01-26 05:13:18 +00:00
meta_required = ["dispatcher_email"]
meta_optional = ["pager_email"]
2017-01-26 03:14:58 +00:00
}
}
```
2022-09-28 12:18:56 +00:00
See the [multiregion] documentation for additional considerations when
dispatching parameterized jobs.
2017-01-26 03:14:58 +00:00
## `parameterized` Requirements
- The job's [scheduler type][batch-type] must be `batch` or `sysbatch`.
2017-01-26 03:14:58 +00:00
## `parameterized` Parameters
2017-01-26 06:15:00 +00:00
- `meta_optional` `(array<string>: nil)` - Specifies the set of metadata keys that
2020-02-06 23:45:31 +00:00
may be provided when dispatching against the job.
2017-01-26 05:13:18 +00:00
2017-01-26 06:15:00 +00:00
- `meta_required` `(array<string>: nil)` - Specifies the set of metadata keys that
2017-01-26 05:13:18 +00:00
must be provided when dispatching against the job.
2017-01-26 03:14:58 +00:00
- `payload` `(string: "optional")` - Specifies the requirement of providing a
2017-01-26 19:31:47 +00:00
payload when dispatching against the parameterized job. The **maximum size of a
`payload` is 16 KiB**. The options for this
2017-01-26 03:14:58 +00:00
field are:
- `"optional"` - A payload is optional when dispatching against the job.
- `"required"` - A payload must be provided when dispatching against the job.
- `"forbidden"` - A payload is forbidden when dispatching against the job.
## `parameterized` Examples
The following examples show non-runnable example parameterized jobs:
### Required Inputs
This example shows a parameterized job that requires both a payload and
metadata:
```hcl
job "video-encode" {
2017-02-06 18:59:07 +00:00
# ...
2017-01-26 03:14:58 +00:00
type = "batch"
parameterized {
payload = "required"
2017-01-26 05:13:18 +00:00
meta_required = ["dispatcher_email"]
2017-01-26 03:14:58 +00:00
}
2017-02-06 18:59:07 +00:00
2017-01-26 03:14:58 +00:00
group "encode" {
2017-02-06 18:59:07 +00:00
# ...
2017-01-26 03:14:58 +00:00
task "ffmpeg" {
driver = "exec"
config {
command = "ffmpeg-wrapper"
# When dispatched, the payload is written to a file that is then read by
# the created task upon startup
args = ["-config=${NOMAD_TASK_DIR}/config.json"]
}
2017-02-06 18:59:07 +00:00
2017-01-26 04:49:06 +00:00
dispatch_payload {
2017-01-26 03:14:58 +00:00
file = "config.json"
}
}
}
}
```
### Metadata Interpolation
```hcl
job "email-blast" {
2017-02-06 18:59:07 +00:00
# ...
2017-01-26 03:14:58 +00:00
type = "batch"
parameterized {
payload = "forbidden"
2017-01-26 05:13:18 +00:00
meta_required = ["CAMPAIGN_ID"]
2017-01-26 03:14:58 +00:00
}
2017-02-06 18:59:07 +00:00
2017-01-26 03:14:58 +00:00
group "emails" {
2017-02-06 18:59:07 +00:00
# ...
2017-01-26 03:14:58 +00:00
task "emailer" {
driver = "exec"
config {
command = "emailer"
2018-03-11 17:44:58 +00:00
# The campaign ID is interpolated and injected into the task's
2017-01-26 03:14:58 +00:00
# arguments
args = ["-campaign=${NOMAD_META_CAMPAIGN_ID}"]
}
}
}
}
```
[batch-type]: /nomad/docs/job-specification/job#type 'Batch scheduler type'
[dispatch command]: /nomad/docs/commands/job/dispatch 'Nomad Job Dispatch Command'
[resources]: /nomad/docs/job-specification/resources 'Nomad resources Job Specification'
[interpolation]: /nomad/docs/runtime/interpolation 'Nomad Runtime Interpolation'
[dispatch_payload]: /nomad/docs/job-specification/dispatch_payload 'Nomad dispatch_payload Job Specification'
[multiregion]: /nomad/docs/job-specification/multiregion#parameterized-dispatch