135 lines
4.7 KiB
Plaintext
135 lines
4.7 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: csi_plugin Stanza - Job Specification
|
|
description: >-
|
|
The "csi_plugin" stanza allows the task to specify it provides a
|
|
Container Storage Interface plugin to the cluster.
|
|
---
|
|
|
|
# `csi_plugin` Stanza
|
|
|
|
<Placement groups={['job', 'group', 'task', 'volume']} />
|
|
|
|
The "csi_plugin" stanza allows the task to specify it provides a
|
|
Container Storage Interface plugin to the cluster. Nomad will
|
|
automatically register the plugin so that it can be used by other jobs
|
|
to claim [volumes][csi_volumes].
|
|
|
|
```hcl
|
|
csi_plugin {
|
|
id = "csi-hostpath"
|
|
type = "monolith"
|
|
mount_dir = "/csi"
|
|
health_timeout = "30s"
|
|
}
|
|
```
|
|
|
|
## `csi_plugin` Parameters
|
|
|
|
- `id` `(string: <required>)` - This is the ID for the plugin. Some
|
|
plugins will require both controller and node plugin types (see
|
|
below); you need to use the same ID for both so that Nomad knows they
|
|
belong to the same plugin.
|
|
|
|
- `type` `(string: <required>)` - One of `node`, `controller`, or
|
|
`monolith`. Each plugin supports one or more types. Each Nomad
|
|
client node where you want to mount a volume will need a `node`
|
|
plugin instance. Some plugins will also require one or more
|
|
`controller` plugin instances to communicate with the storage
|
|
provider's APIs. Some plugins can serve as both `controller` and
|
|
`node` at the same time, and these are called `monolith`
|
|
plugins. Refer to your CSI plugin's documentation.
|
|
|
|
- `mount_dir` `(string: <required>)` - The directory path inside the
|
|
container where the plugin will expect a Unix domain socket for
|
|
bidirectional communication with Nomad.
|
|
|
|
- `health_timeout` `(duration: <optional>)` - The duration that
|
|
the plugin supervisor will wait before restarting an unhealthy
|
|
CSI plugin. Must be a duration value such as `30s` or `2m`.
|
|
Defaults to `30s` if not set.
|
|
|
|
~> **Note:** Plugins running as `node` or `monolith` require root
|
|
privileges (or `CAP_SYS_ADMIN` on Linux) to mount volumes on the
|
|
host. With the Docker task driver, you can use the `privileged = true`
|
|
configuration, but no other default task drivers currently have this
|
|
option.
|
|
|
|
## Recommendations for Deploying CSI Plugins
|
|
|
|
CSI plugins run as Nomad tasks, but after mounting the volume are not in the
|
|
data path for the volume. Tasks that mount volumes write and read directly to
|
|
the volume via a bind-mount and there is no communication between the job and
|
|
the CSI plugin. But when an allocation that mounts a volume stops, Nomad will
|
|
need to communicate with the plugin on that allocation's node to unmount the
|
|
volume. This has implications on how to deploy CSI plugins:
|
|
|
|
* If you are stopping jobs on a node, you must stop tasks that claim
|
|
volumes before stopping the `node` or `monolith` plugin for those
|
|
volumes. If you use the `node drain` feature, plugin tasks will
|
|
automatically be drained last.
|
|
|
|
* Only the most recently-placed allocation for a given plugin ID and
|
|
type (controller or node) will be used by any given client node. Run
|
|
`node` plugins as system jobs and distribute `controller` plugins
|
|
across client nodes using a constraint as shown below.
|
|
|
|
* Some plugins will create volumes only in the same location as the
|
|
plugin. For example, the AWS EBS plugin will create and mount
|
|
volumes only within the same Availability Zone. You should configure
|
|
your plugin task as recommended by the plugin's documentation to use
|
|
the [`topology_request`] field in your volume specification.
|
|
|
|
## `csi_plugin` Examples
|
|
|
|
```hcl
|
|
job "plugin-efs" {
|
|
datacenters = ["dc1"]
|
|
|
|
# you can run node plugins as service jobs as well, but running
|
|
# as a system job ensures all nodes in the DC have a copy.
|
|
type = "system"
|
|
|
|
# only one plugin of a given type and ID should be deployed on
|
|
# any given client node
|
|
constraint {
|
|
operator = "distinct_hosts"
|
|
value = true
|
|
}
|
|
|
|
group "nodes" {
|
|
task "plugin" {
|
|
driver = "docker"
|
|
|
|
config {
|
|
image = "amazon/aws-efs-csi-driver:v.1.3.2"
|
|
|
|
args = [
|
|
"--endpoint=unix://csi/csi.sock",
|
|
"--logtostderr",
|
|
"--v=5",
|
|
]
|
|
|
|
# all CSI node plugins will need to run as privileged tasks
|
|
# so they can mount volumes to the host. controller plugins
|
|
# do not need to be privileged.
|
|
privileged = true
|
|
}
|
|
|
|
csi_plugin {
|
|
id = "aws-efs0"
|
|
type = "node"
|
|
mount_dir = "/csi" # this path /csi matches the --endpoint
|
|
# argument for the container
|
|
health_timeout = "30s"
|
|
}
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
[csi]: https://github.com/container-storage-interface/spec
|
|
[csi_volumes]: /docs/job-specification/volume
|
|
[system]: /docs/schedulers#system
|
|
[`topology_request`]: /docs/commands/volume/create#topology_request
|