47ce5ff471
Adds an example of using HTTP Basic Auth, git options, and using HCL2 syntax to encode an SSH key from file.
243 lines
7.3 KiB
Plaintext
243 lines
7.3 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: artifact Stanza - Job Specification
|
|
sidebar_title: artifact
|
|
description: |-
|
|
The "artifact" stanza instructs Nomad to fetch and unpack a remote resource,
|
|
such as a file, tarball, or binary, and permits downloading artifacts from a
|
|
variety of locations using a URL as the input source.
|
|
---
|
|
|
|
# `artifact` Stanza
|
|
|
|
<Placement groups={['job', 'group', 'task', 'artifact']} />
|
|
|
|
The `artifact` stanza instructs Nomad to fetch and unpack a remote resource,
|
|
such as a file, tarball, or binary. Nomad downloads artifacts using the popular
|
|
[`go-getter`][go-getter] library, which permits downloading artifacts from a
|
|
variety of locations using a URL as the input source.
|
|
|
|
```hcl
|
|
job "docs" {
|
|
group "example" {
|
|
task "server" {
|
|
artifact {
|
|
source = "https://example.com/file.tar.gz"
|
|
destination = "local/some-directory"
|
|
options {
|
|
checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Nomad supports downloading `http`, `https`, `git`, `hg` and `S3` artifacts. If
|
|
these artifacts are archived (`zip`, `tgz`, `bz2`, `xz`), they are
|
|
automatically unarchived before the starting the task.
|
|
|
|
## `artifact` Parameters
|
|
|
|
- `destination` `(string: "local/")` - Specifies the directory path to
|
|
download the artifact, relative to the root of the [task's working
|
|
directory]. If omitted, the default value is to place the artifact in
|
|
`local/`. The destination is treated as a directory unless `mode` is set to
|
|
`file`. Source files will be downloaded into that directory path. For more
|
|
details on how the `destination` interacts with task drivers, see the
|
|
[Filesystem internals] documentation. [Interpolation is not yet
|
|
supported.][gh-6929]
|
|
|
|
- `mode` `(string: "any")` - One of `any`, `file`, or `dir`. If set to `file`
|
|
the `destination` must be a file, not a directory. By default the
|
|
`destination` will be `local/<filename>`.
|
|
|
|
- `options` `(map<string|string>: nil)` - Specifies configuration parameters to
|
|
fetch the artifact. The key-value pairs map directly to parameters appended to
|
|
the supplied `source` URL. Please see the [`go-getter`
|
|
documentation][go-getter] for a complete list of options and examples.
|
|
|
|
- `headers` `(map<string|string>: nil)` - Specifies HTTP headers to set when
|
|
fetching the artifact using `http` or `https` protocol. Please see the
|
|
[`go-getter` headers documentation][go-getter-headers] for more information.
|
|
|
|
- `source` `(string: <required>)` - Specifies the URL of the artifact to download.
|
|
See [`go-getter`][go-getter] for details.
|
|
|
|
## `artifact` Examples
|
|
|
|
The following examples only show the `artifact` stanzas. Remember that the
|
|
`artifact` stanza is only valid in the placements listed above.
|
|
|
|
### Download File
|
|
|
|
This example downloads the artifact from the provided URL and places it in
|
|
`local/file.txt`. The `local/` path is relative to the [task's working
|
|
directory].
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.txt"
|
|
}
|
|
```
|
|
|
|
To set HTTP headers in the request for the source the optional `headers` field
|
|
can be configured.
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.txt"
|
|
|
|
headers {
|
|
User-Agent = "nomad-[${NOMAD_JOB_ID}]-[${NOMAD_GROUP_NAME}]-[${NOMAD_TASK_NAME}]"
|
|
X-Nomad-Alloc = "${NOMAD_ALLOC_ID}"
|
|
}
|
|
}
|
|
```
|
|
|
|
To use HTTP basic authentication, preprend the username and password to the
|
|
hostname in the URL. All special characters, including the username and
|
|
password, must be URL encoded. For example, for a username `exampleUser` and
|
|
the password `pass/word!`:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://exampleUser:pass%2Fword%21@example.com/file.txt"
|
|
}
|
|
```
|
|
|
|
### Download using git
|
|
|
|
This example downloads the artifact from the provided GitHub URL and places it at
|
|
`local/repo`, as specified by the optional `destination` parameter.
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "git::https://github.com/example/nomad-examples"
|
|
destination = "local/repo"
|
|
}
|
|
```
|
|
|
|
To download from private repo, sshkey needs to be set. The key must be
|
|
base64-encoded string. On Linux, you can run `base64 -w0 <file>` to encode the
|
|
file. Or use [HCL2](https://www.nomadproject.io/docs/job-specification/hcl2)
|
|
expressions to read and encode the key from a file on your machine:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "git@github.com:example/nomad-examples"
|
|
destination = "local/repo"
|
|
options {
|
|
sshkey = "${base64encode(file(pathexpand("~/.ssh/id_rsa")))}"
|
|
}
|
|
}
|
|
```
|
|
|
|
To clone specific refs or at a specific depth, use the `ref` and `depth`
|
|
options:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "git::https://github.com/example/nomad-examples"
|
|
destination = "local/repo"
|
|
options {
|
|
ref = "main"
|
|
depth = 1
|
|
}
|
|
}
|
|
```
|
|
|
|
### Download and Unarchive
|
|
|
|
This example downloads and unarchives the result in `local/file`. Because the
|
|
source URL is an archive extension, Nomad will automatically decompress it:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.tar.gz"
|
|
}
|
|
```
|
|
|
|
To disable automatic unarchiving, set the `archive` option to false:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.tar.gz"
|
|
options {
|
|
archive = false
|
|
}
|
|
}
|
|
```
|
|
|
|
### Download and Verify Checksums
|
|
|
|
This example downloads an artifact and verifies the resulting artifact's
|
|
checksum before proceeding. If the checksum is invalid, an error will be
|
|
returned.
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://example.com/file.zip"
|
|
|
|
options {
|
|
checksum = "md5:df6a4178aec9fbdc1d6d7e3634d1bc33"
|
|
}
|
|
}
|
|
```
|
|
|
|
### Download from an S3-compatible Bucket
|
|
|
|
These examples download artifacts from Amazon S3. There are several different
|
|
types of [S3 bucket addressing][s3-bucket-addr] and [S3 region-specific
|
|
endpoints][s3-region-endpoints]. As of Nomad 0.6 non-Amazon S3-compatible
|
|
endpoints like [Minio] are supported, but you must explicitly set the "s3::"
|
|
prefix.
|
|
|
|
This example uses path-based notation on a publicly-accessible bucket:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://my-bucket-example.s3-us-west-2.amazonaws.com/my_app.tar.gz"
|
|
}
|
|
```
|
|
|
|
If a bucket requires authentication, you can avoid the use of credentials by
|
|
using [EC2 IAM instance profiles][iam-instance-profiles]. If this is not possible,
|
|
credentials may be supplied via the `options` parameter:
|
|
|
|
```hcl
|
|
artifact {
|
|
options {
|
|
aws_access_key_id = "<id>"
|
|
aws_access_key_secret = "<secret>"
|
|
aws_access_token = "<token>"
|
|
}
|
|
}
|
|
```
|
|
|
|
To force the S3-specific syntax, use the `s3::` prefix:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "s3::https://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz"
|
|
}
|
|
```
|
|
|
|
Alternatively you can use virtual hosted style:
|
|
|
|
```hcl
|
|
artifact {
|
|
source = "https://my-bucket-example.s3-eu-west-1.amazonaws.com/my_app.tar.gz"
|
|
}
|
|
```
|
|
|
|
[gh-6929]: https://github.com/hashicorp/nomad/issues/6929
|
|
[go-getter]: https://github.com/hashicorp/go-getter 'HashiCorp go-getter Library'
|
|
[go-getter-headers]: https://github.com/hashicorp/go-getter#headers 'HashiCorp go-getter Headers'
|
|
[minio]: https://www.minio.io/
|
|
[s3-bucket-addr]: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html#access-bucket-intro 'Amazon S3 Bucket Addressing'
|
|
[s3-region-endpoints]: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region 'Amazon S3 Region Endpoints'
|
|
[iam-instance-profiles]: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html 'EC2 IAM instance profiles'
|
|
[task's working directory]: /docs/runtime/environment#task-directories 'Task Directories'
|
|
[Filesystem internals]: /docs/internals/filesystem#templates-artifacts-and-dispatch-payloads
|