19a2c065f4
Service jobs should have unique allocation Names, derived from the Job.ID. System jobs do not have unique allocation Names because the index is intended to indicated the instance out of a desired count size. Because system jobs do not have an explicit count but the results are based on the targeted nodes, the index is less informative and this was intentionally omitted from the original design. Update docs to make it clear that NOMAD_ALLOC_INDEX is always zero for system/sysbatch jobs Validate that `volume.per_alloc` is incompatible with system/sysbatch jobs. System and sysbatch jobs always have a `NOMAD_ALLOC_INDEX` of 0. So interpolation via `per_alloc` will not work as soon as there's more than one allocation placed. Validate against this on job submission.
210 lines
6.8 KiB
Plaintext
210 lines
6.8 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: volume Block - Job Specification
|
|
description: >-
|
|
The "volume" block allows the group to specify that it requires a given
|
|
volume from the cluster. Nomad will automatically handle ensuring that the
|
|
volume is available and mounted into the task.
|
|
---
|
|
|
|
# `volume` Block
|
|
|
|
<Placement groups={['job', 'group', 'volume']} />
|
|
|
|
The `volume` block allows the group to specify that it requires a
|
|
given volume from the cluster.
|
|
|
|
The key of the block is the name of the volume as it will be exposed
|
|
to task configuration.
|
|
|
|
```hcl
|
|
job "docs" {
|
|
group "example" {
|
|
volume "certs" {
|
|
type = "host"
|
|
source = "ca-certificates"
|
|
read_only = true
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
```hcl
|
|
job "docs" {
|
|
group "example" {
|
|
volume "data" {
|
|
type = "csi"
|
|
source = "csi-volume"
|
|
read_only = true
|
|
attachment_mode = "file-system"
|
|
access_mode = "single-node-writer"
|
|
per_alloc = true
|
|
|
|
mount_options {
|
|
fs_type = "ext4"
|
|
mount_flags = ["noatime"]
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
|
|
The Nomad server will ensure that the allocations are only scheduled
|
|
on hosts that have a set of volumes that meet the criteria specified
|
|
in the `volume` blocks. These may be [host volumes][host_volume]
|
|
configured on the client, or [CSI volumes][csi_volume] dynamically
|
|
mounted by [CSI plugins][csi_plugin].
|
|
|
|
The Nomad client will make the volumes available to tasks according to
|
|
the [volume_mount][volume_mount] block in the `task` configuration.
|
|
|
|
## `volume` Parameters
|
|
|
|
- `type` `(string: "")` - Specifies the type of a given volume. The
|
|
valid volume types are `"host"` and `"csi"`.
|
|
|
|
- `source` `(string: <required>)` - The name of the volume to
|
|
request. When using `host_volume`'s this should match the published
|
|
name of the host volume. When using `csi` volumes, this should match
|
|
the ID of the registered volume.
|
|
|
|
- `read_only` `(bool: false)` - Specifies that the group only requires
|
|
read only access to a volume and is used as the default value for
|
|
the `volume_mount -> read_only` configuration. This value is also
|
|
used for validating `host_volume` ACLs and for scheduling when a
|
|
matching `host_volume` requires `read_only` usage.
|
|
|
|
- `per_alloc` `(bool: false)` - Specifies that the `source` of the volume
|
|
should have the suffix `[n]`, where `n` is the allocation index. This allows
|
|
mounting a unique volume per allocation, so long as the volume's source is
|
|
named appropriately. For example, with the source `myvolume` and `per_alloc
|
|
= true`, the allocation named `myjob.mygroup.mytask[0]` will require a
|
|
volume ID `myvolume[0]`.
|
|
|
|
The `per_alloc` field cannot be true for system jobs, sysbatch jobs, or jobs
|
|
that use canaries.
|
|
|
|
The following fields are only valid for volumes with `type = "csi"`:
|
|
|
|
- `access_mode` `(string: <required>)` - Defines whether a volume should be
|
|
available concurrently. The `access_mode` and `attachment_mode` together
|
|
must exactly match one of the volume's `capability` blocks. Can be one of
|
|
`"single-node-reader-only"`, `"single-node-writer"`,
|
|
`"multi-node-reader-only"`, `"multi-node-single-writer"`, or
|
|
`"multi-node-multi-writer"`. Most CSI plugins support only single-node
|
|
modes. Consult the documentation of the storage provider and CSI plugin.
|
|
|
|
- `attachment_mode` `(string: <required>)` - The storage API that will be used
|
|
by the volume. The `access_mode` and `attachment_mode` together must exactly
|
|
match one of the volume's `capability` blocks. Most storage providers will
|
|
support `"file-system"`, to mount volumes using the CSI filesystem API. Some
|
|
storage providers will support `"block-device"`, which will mount the volume
|
|
with the CSI block device API within the container.
|
|
|
|
- `mount_options` - Options for mounting CSI volumes that have the
|
|
`file-system` [attachment mode]. These options override the `mount_options`
|
|
field from [volume registration]. Consult the documentation for your storage
|
|
provider and CSI plugin as to whether these options are required or
|
|
necessary.
|
|
|
|
- `fs_type`: file system type (ex. `"ext4"`)
|
|
- `mount_flags`: the flags passed to `mount` (ex. `["ro", "noatime"]`)
|
|
|
|
## Volume Interpolation
|
|
|
|
Because volumes represent state, many workloads with multiple allocations will
|
|
want to mount specific volumes to specific tasks. The `volume` block is used
|
|
to schedule workloads, so `${NOMAD_ALLOC_INDEX}` can't be used directly in the
|
|
`volume.source` field. The following job specification demonstrates how to use
|
|
multiple volumes with multiple allocations, using the `per_alloc` field. This
|
|
job specification also shows using HCL2 -variables interpolation to expose
|
|
information to the task's environment.
|
|
|
|
```hcl
|
|
variables {
|
|
path = "test"
|
|
}
|
|
|
|
job "example" {
|
|
datacenters = ["dc1"]
|
|
|
|
group "cache" {
|
|
|
|
count = 2
|
|
|
|
volume "cache-volume" {
|
|
type = "csi"
|
|
source = "test-volume"
|
|
attachment_mode = "file-system"
|
|
access_mode = "single-node-writer"
|
|
per_alloc = true
|
|
}
|
|
|
|
network {
|
|
port "db" {
|
|
to = 6379
|
|
}
|
|
}
|
|
|
|
task "redis" {
|
|
driver = "docker"
|
|
config {
|
|
image = "redis:7"
|
|
ports = ["db"]
|
|
}
|
|
resources {
|
|
cpu = 500
|
|
memory = 256
|
|
}
|
|
|
|
env {
|
|
# this will be available as the MOUNT_PATH environment
|
|
# variable in the task
|
|
MOUNT_PATH = "${NOMAD_ALLOC_DIR}/${var.path}"
|
|
}
|
|
|
|
volume_mount {
|
|
volume = "cache-volume"
|
|
destination = "${NOMAD_ALLOC_DIR}/${var.path}"
|
|
}
|
|
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
The job that results from this job specification has two task groups, each one
|
|
named for each of the two volumes. Each allocation has its own volume.
|
|
|
|
```shell-session
|
|
$ nomad job status example
|
|
ID = example
|
|
...
|
|
|
|
Allocations
|
|
ID Node ID Task Group Version Desired Status Created Modified
|
|
81d32909 352c6926 cache-1 0 run running 4s ago 3s ago
|
|
ce6fbfc8 352c6926 cache-0 0 run running 4s ago 3s ago
|
|
|
|
$ nomad volume status 'test-volume[0]'
|
|
...
|
|
Allocations
|
|
ID Node ID Task Group Version Desired Status Created Modified
|
|
ce6fbfc8 352c6926 cache-0 0 run running 29s ago 18s ago
|
|
|
|
$ nomad volume status 'test-volume[1]'
|
|
...
|
|
Allocations
|
|
ID Node ID Task Group Version Desired Status Created Modified
|
|
81d32909 352c6926 cache-0 0 run running 29s ago 18s ago
|
|
```
|
|
|
|
[volume_mount]: /nomad/docs/job-specification/volume_mount 'Nomad volume_mount Job Specification'
|
|
[host_volume]: /nomad/docs/configuration/client#host_volume-block
|
|
[csi_volume]: /nomad/docs/commands/volume/register
|
|
[csi_plugin]: /nomad/docs/job-specification/csi_plugin
|
|
[csi_volume]: /nomad/docs/commands/volume/register
|
|
[attachment mode]: /nomad/docs/commands/volume/register#attachment_mode
|
|
[volume registration]: /nomad/docs/commands/volume/register#mount_options
|