From 05628dbf24a4745b54c96e5b5bca61b2559bbe25 Mon Sep 17 00:00:00 2001 From: Alex Dadgar Date: Wed, 25 Jan 2017 19:14:58 -0800 Subject: [PATCH] Parameterized page --- .../docs/commands/job-dispatch.html.md.erb | 8 +- .../job-specification/parameterized.html.md | 159 ++++++++++++++++++ .../docs/job-specification/periodic.html.md | 5 + 3 files changed, 171 insertions(+), 1 deletion(-) create mode 100644 website/source/docs/job-specification/parameterized.html.md diff --git a/website/source/docs/commands/job-dispatch.html.md.erb b/website/source/docs/commands/job-dispatch.html.md.erb index 551aa2d45..cd5b24ffb 100644 --- a/website/source/docs/commands/job-dispatch.html.md.erb +++ b/website/source/docs/commands/job-dispatch.html.md.erb @@ -14,11 +14,15 @@ description: > page.](https://releases.hashicorp.com/nomad/0.5.3-rc1/) The `job dispatch` command is used to create new instances of a [parameterized -job](TODO). The parameterized job captures a job's configuration and runtime +job]. The parameterized job captures a job's configuration and runtime requirements in a generic way and `dispatch` is used to provide the input for the job to run against. One can think of the parameterized job as a function definition and dispatch is used to invoke the function. +Each time a job is dispatched, a unique job ID is generated. This allows a +caller to track the status of the job, much like a future or promise in some +programming languages. + ## Usage ``` @@ -104,3 +108,5 @@ $ nomad job dispatch -detach video-encode video-config.json Dispatched Job ID = example/dispatch-1485380684-c37b3dba Evaluation ID = d9034c4e ``` + +[parameterized job]: /docs/job-specification/parameterized.html "Nomad parameterized Job Specification" diff --git a/website/source/docs/job-specification/parameterized.html.md b/website/source/docs/job-specification/parameterized.html.md new file mode 100644 index 000000000..72ea3e297 --- /dev/null +++ b/website/source/docs/job-specification/parameterized.html.md @@ -0,0 +1,159 @@ +--- +layout: "docs" +page_title: "parameterized Stanza - Job Specification" +sidebar_current: "docs-job-specification-parameterized" +description: |- + A parameterized job is used to encapsulate a set of work that can be carried + out on various inputs much like a function definition. When the + `parameterized` stanza is added to a job, the job acts as a function to the + cluster as a whole. +--- + +# `parameterized` Stanza + + + + + + +
Placement + job -> **parameterized** +
+ +A parameterized job is used to encapsulate a set of work that can be carried out +on various inputs much like a function definition. When the `parameterized` +stanza is added to a job, the job acts as a function to the cluster as a whole. + +The `parameterized` stanza allows job operators to configure a job that carries +out a particular action, define its resource requirements and configure how +inputs and configuration are retreived by the tasks within the job. + +To invoke a parameterized job, [`nomad job +dispatch`][dispatch command] or the equivalent HTTP APIs are +used. When dispatching against a parameterized job, an opaque payload and +metadata may be injected into the job. These inputs to the parameterized job act +like arguments to a function. The job consumes them to change it's behavior, +without exposing the implementation details to the caller. + +To that end, tasks within the job can add a [`dispatch_input`][dispatch_input] stanza that +defines where on the filesystem this payload gets written to. An example payload +would be a task's JSON configuration. + +Further, certain metadata may be marked as required when dispatching a job so it +can be used to inject configuration directly into a task's arguments using +[interpolation]. An example of this would be to require a run ID key that +could be used to lookup the work the job is suppose to do from a management +service or database. + +Each time a job is dispatched, a unique job ID is generated. This allows a +caller to track the status of the job, much like a future or promise in some +programming languages. + +```hcl +job "docs" { + parameterized { + payload = "required" + required_meta = ["dispatcher_email"] + optional_meta = ["pager_email"] + } +} +``` + +## `parameterized` Requirements + + - The job's [scheduler type][batch-type] must be `batch`. + +## `parameterized` Parameters + +- `payload` `(string: "optional")` - Specifies the requirement of providing a + payload when dispatching against the parameterized job. The options for this + field are: + + - `"optional"` - A payload is optional when dispatching against the job. + + - `"required"` - A payload must be provided when dispatching against the job. + + - `"forbidden"` - A payload is forbidden when dispatching against the job. + +- `required_meta` `([]string: nil)` - Specifies the set of metadata keys that + must be provided when dispatching against the job. + +- `optional_meta` `([]string: nil)` - Specifies the set of metadata keys that + may be provided when dispatching against the job. + +## `parameterized` Examples + +The following examples show non-runnable example parameterized jobs: + +### Required Inputs + +This example shows a parameterized job that requires both a payload and +metadata: + +```hcl +job "video-encode" { + ... + type = "batch" + + parameterized { + payload = "required" + required_meta = ["dispatcher_email"] + } + + group "encode" { + ... + + task "ffmpeg" { + driver = "exec" + + config { + command = "ffmpeg-wrapper" + + # When dispatched, the payload is written to a file that is then read by + # the created task upon startup + args = ["-config=${NOMAD_TASK_DIR}/config.json"] + } + + dispatch_input { + file = "config.json" + } + } + } +} +``` + +### Metadata Interpolation + +```hcl +job "email-blast" { + ... + type = "batch" + + parameterized { + payload = "forbidden" + required_meta = ["CAMPAIGN_ID"] + } + + group "emails" { + ... + + task "emailer" { + driver = "exec" + + config { + command = "emailer" + + # The campagain ID is interpolated and injected into the task's + # arguments + args = ["-campaign=${NOMAD_META_CAMPAIGN_ID}"] + } + } + } +} +``` + +[batch-type]: /docs/job-specification/job.html#type "Batch scheduler type" +[dispatch command]: /docs/commands/job-dispatch.html "Nomad Job Dispatch Command" +[resources]: /docs/job-specification/resources.html "Nomad resources Job Specification" +[interpolation]: /docs/runtime/interpolation.html "Nomad Runtime Interpolation" +[dispatch_input]: /docs/job-specification/dispatch-input.html "Nomad dispatch_input Job Specification" diff --git a/website/source/docs/job-specification/periodic.html.md b/website/source/docs/job-specification/periodic.html.md index 8b010c149..4f480448f 100644 --- a/website/source/docs/job-specification/periodic.html.md +++ b/website/source/docs/job-specification/periodic.html.md @@ -35,6 +35,10 @@ job "docs" { The periodic expression is always evaluated in the **UTC timezone** to ensure consistent evaluation when Nomad spans multiple time zones. +## `periodic` Requirements + + - The job's [scheduler type][batch-type] must be `batch`. + ## `periodic` Parameters - `cron` `(string: )` - Specifies a cron expression configuring the @@ -60,4 +64,5 @@ periodic { } ``` +[batch-type]: /docs/job-specification/job.html#type "Batch scheduler type" [cron]: https://github.com/gorhill/cronexpr#implementation "List of cron expressions"