open-nomad/website/content/api-docs/volumes.mdx

---
layout: api
page_title: Volumes - HTTP API
description: The `/volume` endpoints are used to query for and interact with volumes.
---

# Volumes HTTP API

The `/volume` and `/volumes` endpoints are used to query for and interact with
volumes.

## List Volumes

This endpoint lists all volumes.

| Method | Path          | Produces           |
| ------ | ------------- | ------------------ |
| `GET`  | `/v1/volumes` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                |
| ---------------- | --------------------------- |
| `YES`            | `namespace:csi-list-volume` |

### Parameters

- `type` `(string: "")` - Specifies the type of volume to
  query. Currently only supports `csi`. This is specified as a query
  string parameter. Returns an empty list if omitted.

- `node_id` `(string: "")` - Specifies a string to filter volumes
  based on an Node ID prefix. Because the value is decoded to bytes,
  the prefix must have an even number of hexadecimal characters
  (0-9a-f). This is specified as a query string parameter.

- `plugin_id` `(string: "")` - Specifies a string to filter volumes
  based on a plugin ID prefix. Because the value is decoded to bytes,
  the prefix must have an even number of hexadecimal characters
  (0-9a-f). This is specified as a query string parameter.

- `next_token` `(string: "")` - This endpoint supports paging. The `next_token`
  parameter accepts a string which identifies the next expected volume. This
  value can be obtained from the `X-Nomad-NextToken` header from the previous
  response.

- `per_page` `(int: 0)` - Specifies a maximum number of volumes to return for
  this request. If omitted, the response is not paginated. The value of the
  `X-Nomad-NextToken` header of the last response can be used as the
  `next_token` of the next request to fetch additional pages.

- `filter` `(string: "")` - Specifies the [expression](/api-docs#filtering)
  used to filter the results. Consider using pagination or a query parameter to
  reduce resource used to serve the request.

### Sample Request

```shell-session
$ curl \
    https://localhost:4646/v1/volumes?type=csi&node_id=foo&plugin_id=plugin-id1
```

### Sample Response

```json
[
  {
    "ID": "volume-id1",
    "ExternalID": "vol-abcdef",
    "Namespace": "default",
    "Name": "volume id1",
    "Topologies": [
      {
        "foo": "bar"
      }
    ],
    "AccessMode": "multi-node-single-writer",
    "AttachmentMode": "file-system",
    "CurrentReaders": 2,
    "CurrentWriters": 1,
    "Schedulable": true,
    "PluginID": "plugin-id1",
    "Provider": "ebs",
    "ControllerRequired": true,
    "ControllersHealthy": 3,
    "ControllersExpected": 3,
    "NodesHealthy": 15,
    "NodesExpected": 18,
    "ResourceExhausted": 0,
    "CreateIndex": 42,
    "ModifyIndex": 64
  }
]
```

## Read Volume

This endpoint reads information about a specific volume.

| Method | Path                        | Produces           |
| ------ | --------------------------- | ------------------ |
| `GET`  | `/v1/volume/csi/:volume_id` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                |
| ---------------- | --------------------------- |
| `YES`            | `namespace:csi-read-volume` |

### Parameters

- `:volume_id` `(string: <required>)` - Specifies the ID of the
  volume. This must be the full ID. This is specified as part of the
  path.

### Sample Request

```shell-session
$ curl \
    https://localhost:4646/v1/volume/csi/volume-id1
```

### Sample Response

```json
{
  "ID": "volume-id1",
  "Name": "volume id1",
  "Namespace": "default",
  "ExternalID": "vol-abcdef",
  "Topologies": [{ "foo": "bar" }],
  "AccessMode": "multi-node-single-writer",
  "AttachmentMode": "file-system",
  "RequestedCapabilities": [
    {
      "AccessMode": "single-node-writer",
      "AttachmentMode": "file-system"
    },
    {
      "AccessMode": "single-node-writer",
      "AttachmentMode": "block-device"
    }
  ],
  "Allocations": [
    {
      "ID": "a8198d79-cfdb-6593-a999-1e9adabcba2e",
      "EvalID": "5456bd7a-9fc0-c0dd-6131-cbee77f57577",
      "Name": "example.cache[0]",
      "NodeID": "fb2170a8-257d-3c64-b14d-bc06cc94e34c",
      "PreviousAllocation": "516d2753-0513-cfc7-57ac-2d6fac18b9dc",
      "NextAllocation": "cd13d9b9-4f97-7184-c88b-7b451981616b",
      "RescheduleTracker": {
        "Events": [
          {
            "PrevAllocID": "516d2753-0513-cfc7-57ac-2d6fac18b9dc",
            "PrevNodeID": "9230cd3b-3bda-9a3f-82f9-b2ea8dedb20e",
            "RescheduleTime": 1517434161192946200,
            "Delay": "5000000000"
          }
        ]
      },
      "JobID": "example",
      "TaskGroup": "cache",
      "DesiredStatus": "run",
      "DesiredDescription": "",
      "ClientStatus": "running",
      "ClientDescription": "",
      "TaskStates": {
        "redis": {
          "State": "running",
          "FinishedAt": "0001-01-01T00:00:00Z",
          "LastRestart": "0001-01-01T00:00:00Z",
          "Restarts": 0,
          "StartedAt": "2017-07-25T23:36:26.106431265Z",
          "Failed": false,
          "Events": [
            {
              "Type": "Received",
              "Time": 1495747371795703800,
              "FailsTask": false,
              "RestartReason": "",
              "SetupError": "",
              "DriverError": "",
              "ExitCode": 0,
              "Signal": 0,
              "Message": "",
              "KillTimeout": 0,
              "KillError": "",
              "KillReason": "",
              "StartDelay": 0,
              "DownloadError": "",
              "ValidationError": "",
              "DiskLimit": 0,
              "FailedSibling": "",
              "VaultError": "",
              "TaskSignalReason": "",
              "TaskSignal": "",
              "DriverMessage": ""
            },
            {
              "Type": "Driver",
              "Time": 1495747371798867200,
              "FailsTask": false,
              "RestartReason": "",
              "SetupError": "",
              "DriverError": "",
              "ExitCode": 0,
              "Signal": 0,
              "Message": "",
              "KillTimeout": 0,
              "KillError": "",
              "KillReason": "",
              "StartDelay": 0,
              "DownloadError": "",
              "ValidationError": "",
              "DiskLimit": 0,
              "FailedSibling": "",
              "VaultError": "",
              "TaskSignalReason": "",
              "TaskSignal": "",
              "DriverMessage": "Downloading image redis:7"
            },
            {
              "Type": "Started",
              "Time": 1495747379525667800,
              "FailsTask": false,
              "RestartReason": "",
              "SetupError": "",
              "DriverError": "",
              "ExitCode": 0,
              "Signal": 0,
              "Message": "",
              "KillTimeout": 0,
              "KillError": "",
              "KillReason": "",
              "StartDelay": 0,
              "DownloadError": "",
              "ValidationError": "",
              "DiskLimit": 0,
              "FailedSibling": "",
              "VaultError": "",
              "TaskSignalReason": "",
              "TaskSignal": "",
              "DriverMessage": ""
            }
          ]
        }
      },
      "CreateIndex": 54,
      "ModifyIndex": 57,
      "CreateTime": 1495747371794276400,
      "ModifyTime": 1495747371794276400
    }
  ],
  "ReadAllocs": {
    "a8198d79-cfdb-6593-a999-1e9adabcba2e": null
  },
  "WriteAllocs": {},
  "Schedulable": true,
  "PluginID": "plugin-id1",
  "Provider": "ebs",
  "Version": "1.0.1",
  "ControllerRequired": true,
  "ControllersHealthy": 3,
  "ControllersExpected": 3,
  "NodesHealthy": 15,
  "NodesExpected": 18,
  "ResourceExhausted": 0,
  "CreateIndex": 42,
  "ModifyIndex": 64
}
```

## Register Volume

This endpoint registers an external volume with Nomad. The volume must exist
in the external storage provider (see [Create Volume] below). It is an error
to register an already registered volume.

| Method | Path                        | Produces           |
| ------ | --------------------------- | ------------------ |
| `PUT`  | `/v1/volume/csi/:volume_id` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                 |
| ---------------- | ---------------------------- |
| `NO`             | `namespace:csi-write-volume` |

### Parameters

- `:volume_id` `(string: <required>)` - Specifies the ID of the
  volume. This must be the full ID. This is specified as part of the
  path.

### Sample Payload

The payload must include a JSON document that describes the volume's
parameters.

```json
{
  "Volumes": [
    {
      "ExternalID": "vol-abcdef",
      "ID": "volume-id1",
      "Name": "volume one",
      "Namespace": "default",
      "PluginID": "plugin-id1",
      "RequestedCapabilities": [
        {
          "AccessMode": "single-node-writer",
          "AttachmentMode": "file-system"
        },
        {
          "AccessMode": "single-node-writer",
          "AttachmentMode": "block-device"
        }
      ],
      "Secrets": {
        "password": "xyzzy"
      },
      "Topologies": [{ "foo": "bar" }]
    }
  ]
}
```

### Sample Request

```shell-session
$ curl \
    --request PUT \
    --data @payload.json \
    https://localhost:4646/v1/volume/csi/volume-id1
```

## Create Volume

This endpoint creates a volume in an external storage provider and registers
it with Nomad. It is an error to create an already-registered volume. Only CSI
plugins that implement the [Controller][csi_plugins_internals] interface with
the `CREATE_DELETE_VOLUME` capability support this endpoint.

| Method | Path                               | Produces           |
| ------ | ---------------------------------- | ------------------ |
| `PUT`  | `/v1/volume/csi/:volume_id/create` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                 |
| ---------------- | ---------------------------- |
| `NO`             | `namespace:csi-write-volume` |

### Parameters

- `:volume_id` `(string: <required>)` - Specifies the ID of the
  volume. This must be the full ID. This is specified as part of the
  path.

### Sample Payload

The payload must include a JSON document that describes the volume's
parameters.

```json
{
  "Volumes": [
    {
      "ID": "volume-id1",
      "Name": "volume one",
      "Namespace": "default",
      "PluginID": "plugin-id1",
      "MountOptions": {
        "FsType": "ext4",
        "MountFlags": ["ro", "noatime"],
      },
      "RequestedCapacityMin": 10737418240,
      "RequestedCapacityMax": 21474836480,
      "RequestedCapabilities": [
        {
          "AccessMode": "single-node-writer",
          "AttachmentMode": "file-system"
        },
        {
          "AccessMode": "single-node-writer",
          "AttachmentMode": "block-device"
        }
      ],
      "Secrets": {
        "password": "xyzzy"
      },
      "SnapshotID": "snap-12345",
      "Topologies": [{ "foo": "bar" }]
    }
  ]
}
```

### Sample Request

```shell-session
$ curl \
    --request PUT \
    --data @payload.json \
    https://localhost:4646/v1/volume/csi/volume-id1
```


## Deregister Volume

This endpoint deregisters an external volume with Nomad. It is an error to
deregister a volume that is in use.

| Method   | Path                        | Produces           |
| -------- | --------------------------- | ------------------ |
| `DELETE` | `/v1/volume/csi/:volume_id` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                 |
| ---------------- | ---------------------------- |
| `NO`             | `namespace:csi-write-volume` |

### Parameters

- `:volume_id` `(string: <required>)` - Specifies the ID of the
  volume. This must be the full ID. This is specified as part of the
  path.

- `force` `(bool: false)` - Force deregistration of the volume and immediately
  drop claims for terminal allocations. Returns an error if the volume has
  running allocations. This does not detach the volume from client nodes.
  This is specified as a query string parameter.

### Sample Request

```shell-session
$ curl \
    --request DELETE \
    https://localhost:4646/v1/volume/csi/volume-id1?force=false
```

## Delete Volume

This endpoint deletes an external volume from the storage provider, and
deregisters it from Nomad. It is an error to delete a volume that is in
use. Only CSI plugins that implement the [Controller][csi_plugins_internals]
interface with the `CREATE_DELETE_VOLUME` capability support this endpoint.

| Method   | Path                               | Produces           |
| -------- | ---------------------------------- | ------------------ |
| `DELETE` | `/v1/volume/csi/:volume_id/delete` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                 |
| ---------------- | ---------------------------- |
| `NO`             | `namespace:csi-write-volume` |

This endpoint accepts a `X-Nomad-CSI-Secrets` header to set secrets
for deleting the volume as comma-separated key-value pairs (see the
example below). These secrets will be merged with any secrets already
stored when the CSI volume was created.

### Parameters

- `:volume_id` `(string: <required>)` - Specifies the ID of the
  volume. This must be the full ID. This is specified as part of the
  path.

### Sample Request

```shell-session
$ curl \
    --request DELETE \
    -H "X-Nomad-CSI-Secrets: secret-key-1=value-1,secret-key-2=value-2" \
    https://localhost:4646/v1/volume/csi/volume-id1/delete
```

## Detach Volume

This endpoint detaches an external volume from a Nomad client node. It is an
error to detach a volume that is in use.

| Method   | Path                               | Produces           |
| -------- | ---------------------------------- | ------------------ |
| `DELETE` | `/v1/volume/csi/:volume_id/detach` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                 |
| ---------------- | ---------------------------- |
| `NO`             | `namespace:csi-write-volume` |

### Parameters

- `:volume_id` `(string: <required>)` - Specifies the ID of the
  volume. This must be the full ID. This is specified as part of the
  path.

- `node` `(string: <required>)` - The node to detach the volume from.
  This is specified as a query string parameter.

### Sample Request

```shell-session
$ curl \
    --request DELETE \
    https://localhost:4646/v1/volume/csi/volume-id/detach?node=00000000-0000-0000-0000-000000000000
```

## List External Volumes

This endpoint lists storage volumes that are known to the external storage
provider but may not be registered with Nomad.  Only CSI plugins that
implement the [Controller][csi_plugins_internals] interface with the
`LIST_VOLUMES` capability support this endpoint.

| Method | Path                   | Produces           |
|--------|------------------------|--------------------|
| `GET`  | `/v1/volumes/external` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                 |
| ---------------- | ---------------------------- |
| `YES`             | `namespace:csi-write-volume` |

### Parameters

- `plugin_id` `(string: "")` - Specifies a string to filter volumes
  based on a plugin ID prefix. Because the value is decoded to bytes,
  the prefix must have an even number of hexadecimal characters
  (0-9a-f). This is specified as a query string parameter.

- `next_token` `(string: "")` - This endpoint supports paging. The
  `next_token` parameter accepts a string returned in a previous response's
  `NextToken` field to request the next page of results.

- `per_page` `(int: <required>)` - Specifies a maximum number of snapshots to
  return for this request. The response will include a `NextToken` field that
  can be passed to the next request to fetch additional pages.

### Sample Request

```shell-session
$ curl \
    https://localhost:4646/v1/volumes/external?&plugin_id=plugin-id1&per_page=2
```

### Sample Response

```json
{
  "NextToken": "eyJ2IjoiMiIsImMiOiJLZ",
  "Volumes": [
    {
      "ExternalID": "vol-37ac485e",
      "CapacityBytes": 1000000,
      "SnapshotID": "snap-12345",
      "PublishedExternalNodeIDs": ["i-12345", "i-abcde"],
      "IsAbnormal": false,
      "Status": ""
    },
    {
      "ExternalID": "vol-10ac4879",
      "CapacityBytes": 1000000,
      "SnapshotID": "snap-abcdef",
      "PublishedExternalNodeIDs": ["i-12345", "i-abcde"],
      "IsAbnormal": true,
      "Status": "example error message from provider"
    }
  ]
}
```

## Create Snapshot

This endpoint creates a snapshot of a volume on the external storage
provider. Only CSI plugins that implement the
[Controller][csi_plugins_internals] interface with the
`CREATE_DELETE_SNAPSHOT` capability support this endpoint.

| Method   | Path                              | Produces           |
| -------- | --------------------------------- | ------------------ |
| `POST`    | `/v1/volumes/snapshot`           | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                 |
| ---------------- | ---------------------------- |
| `NO`             | `namespace:csi-write-volume` |


### Sample Payload

The payload must include a JSON document that describes the snapshot's
parameters.

```json
{
  "Snapshots": [
    {
      "SourceVolumeID": "volume-id1",
      "PluginID": "plugin-id1",
      "Name": "mysnap",
      "Secrets": {
        "password": "xyzzy"
      },
      "Parameters": {
        "example": "infrequent_access"
      }
    }
  ]
}
```

### Sample Request

```shell-session
$ curl \
    --request POST \
    --data @payload.json \
    https://localhost:4646/v1/volumes/snapshot
```

### Sample Response

```json
{
  "Snapshots": [
    {
      "ID": "snap-031f5f7e3406d594a",
      "SizeBytes": 10737418240,
      "CreateTime": 1617909982,
      "IsReady": false,
      "SourceVolumeID": "volume-id1"
    }
  ]
}
```

## Delete Snapshot

This endpoint deletes a volume snapshot from the external storage
provider. Only CSI plugins that implement the
[Controller][csi_plugins_internals] interface with the
`CREATE_DELETE_SNAPSHOT` capability support this endpoint.

| Method   | Path                   | Produces           |
| -------- | ---------------------- | ------------------ |
| `DELETE` | `/v1/volumes/snapshot` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                 |
| ---------------- | ---------------------------- |
| `NO`             | `namespace:csi-write-volume` |

This endpoint accepts a `X-Nomad-CSI-Secrets` header to set secrets
for deleting the snapshot as comma-separated key-value pairs (see the
example below). These secrets will be merged with any secrets already
stored when the CSI snapshot was created.

### Parameters

- `plugin_id` `(string: <required>)` - Specifies the prefix of a CSI plugin ID
  to perform the delete. Because the value is decoded to bytes, the prefix
  must have an even number of hexadecimal characters (0-9a-f). This is
  specified as a query string parameter.

- `snapshot_id` `(string: <required>)` - Specifies the snapshot ID to
  delete. This is specified as a query parameter.

### Sample Request

```shell-session
$ curl \
    --request DELETE \
    -H "X-Nomad-CSI-Secrets: secret-key-1=value-1,secret-key-2=value-2" \
    https://localhost:4646/v1/volumes/snapshot
```

## List Snapshots

This endpoint lists volume snapshots on the external storage provider. Only
CSI plugins that implement the [Controller][csi_plugins_internals] interface
with the `LIST_SNAPSHOTS` capability support this endpoint.


| Method | Path                   | Produces           |
| ------ | ---------------------- | ------------------ |
| `GET`  | `/v1/volumes/snapshot` | `application/json` |

The table below shows this endpoint's support for
[blocking queries](/api-docs#blocking-queries) and
[required ACLs](/api-docs#acls).

| Blocking Queries | ACL Required                |
| ---------------- | --------------------------- |
| `YES`            | `namespace:csi-list-volume` |

This endpoint accepts a `X-Nomad-CSI-Secrets` header to set secrets
for deleting the snapshot as comma-separated key-value pairs (see the
example below). These secrets will be merged with any secrets already
stored when the CSI snapshot was created.

### Parameters

- `plugin_id` `(string: <required>)` - Specifies the prefix of a CSI plugin ID
  to perform the list. Because the value is decoded to bytes, the prefix must
  have an even number of hexadecimal characters (0-9a-f). This is specified as
  a query string parameter.

- `next_token` `(string: "")` - This endpoint supports paging. The
  `next_token` parameter accepts a string returned in a previous response's
  `NextToken` field to request the next page of results.

- `per_page` `(int: <required>)` - Specifies a maximum number of snapshots to
  return for this request. The response will include a `NextToken` field that
  can be passed to the next request to fetch additional pages.

### Sample Request

```shell-session
$ curl \
    -H "X-Nomad-CSI-Secrets: secret-key-1=value-1,secret-key-2=value-2" \
    https://localhost:4646/v1/volumes/snapshot?plugin_id=plugin-id1&per_page=2
```

### Sample Response

```json
{
  "NextToken": "eyJ2IjoiMiIsImMiOiJLZ",
  "Snapshots": [
    {
      "CreateTime": 1227088980,
      "ExternalSourceVolumeID": "vol-37ac485e",
      "ID": "snap-9df717f4",
      "IsReady": true,
      "Parameters": null,
      "PluginID": "aws-ebs0",
      "SizeBytes": 2147483648
    },
    {
      "CreateTime": 1227091232,
      "ExternalSourceVolumeID": "vol-10ac4879",
      "ID": "snap-5cf81835",
      "IsReady": true,
      "Parameters": null,
      "PluginID": "aws-ebs0",
      "SizeBytes": 11811160064
    }
  ]
}
```

[csi]: https://github.com/container-storage-interface/spec
[csi_plugin]: /docs/job-specification/csi_plugin
[csi_plugins_internals]: /docs/concepts/plugins/csi#csi-plugins
[Create Volume]: #create-volume