67 lines
2.5 KiB
Markdown
67 lines
2.5 KiB
Markdown
|
# Openstack Ceph-CSI Plugin
|
||
|
|
|
||
|
|
The configuration here is for the Ceph RBD driver, migrated from the k8s config [documentation](https://github.com/ceph/ceph-csi/blob/master/docs/deploy-rbd.md). It can be easily modified for the CephFS Driver, as used [here](https://github.com/ceph/ceph-csi/blob/master/docs/deploy-cephfs.md).
|
||
|
|
|
||
|
|
## Requirements
|
||
|
|
|
||
|
|
The example plugin job creates a file at `local/cloud.conf` using a [`template`](https://www.nomadproject.io/docs/job-specification/template) stanza which pulls the necessary credentials from a [Vault kv-v2](https://www.vaultproject.io/docs/secrets/kv/kv-v2) secrets store.
|
||
|
|
|
||
|
|
|
||
|
|
### Docker Privileged Mode
|
||
|
|
|
||
|
|
The Ceph CSI Node task requires that [`privileged = true`](https://www.nomadproject.io/docs/drivers/docker#privileged) be set. This is not needed for the Controller task.
|
||
|
|
|
||
|
|
## Container Arguments
|
||
|
|
|
||
|
|
Refer to the official plugin [guide](https://github.com/ceph/ceph-csi/blob/master/docs/deploy-rbd.md).
|
||
|
|
|
||
|
|
- `--type=rbd`
|
||
|
|
|
||
|
|
- Driver type `rbd` (or alternately `cephfs`)
|
||
|
|
|
||
|
|
- `--endpoint=unix:///csi/csi.sock`
|
||
|
|
|
||
|
|
- This option must match the `mount_dir` specified in the `csi_plugin` stanza for the task.
|
||
|
|
|
||
|
|
- `--nodeid=${node.unique.name}`
|
||
|
|
|
||
|
|
- A unique ID for the node the task is running on. Recommend using `${node.unique.name}`
|
||
|
|
|
||
|
|
- `--cluster=${NOMAD_DC}`
|
||
|
|
|
||
|
|
- The cluster the Controller/Node is a part of. Recommend using `${NOMAD_DC}`
|
||
|
|
|
||
|
|
- `--instanceid=${attr.unique.platform.aws.instance-id}`
|
||
|
|
|
||
|
|
- Unique ID distinguishing this instance of Ceph CSI among other instances, when sharing Ceph clusters across CSI instances for provisioning. Used for topology-aware deployments.
|
||
|
|
|
||
|
|
## Deployment
|
||
|
|
|
||
|
|
### Plugin
|
||
|
|
|
||
|
|
```bash
|
||
|
|
export NOMAD_ADDR=https://nomad.example.com:4646
|
||
|
|
export NOMAD_TOKEN=34534-3sdf3-szfdsafsdf3423-zxdfsd3
|
||
|
|
nomad job run ceph-csi-plugin.hcl
|
||
|
|
```
|
||
|
|
|
||
|
|
### Volume Registration
|
||
|
|
|
||
|
|
The `external_id` value for the volume must be strictly formatted, see `ceph_csi.tf`. Based on [Ceph-CSI ID Format](https://github.com/ceph/ceph-csi/blob/71ddf51544be498eee03734573b765eb04480bb9/internal/util/volid.go#L27), see [examples](https://github.com/ceph/ceph-csi/blob/71ddf51544be498eee03734573b765eb04480bb9/internal/util/volid_test.go#L33).
|
||
|
|
|
||
|
|
The `secrets` block will be populated with values pulled from `/etc/ceph/ceph.client.<user>.keyring`, e.g.
|
||
|
|
```
|
||
|
|
userid = "<user>"
|
||
|
|
userkey = "AWBg/BtfJInSFBATOrrnCh6UGE3QB3nYakdF+g=="
|
||
|
|
```
|
||
|
|
|
||
|
|
```bash
|
||
|
|
export NOMAD_ADDR=https://nomad.example.com:4646
|
||
|
|
export NOMAD_TOKEN=34534-3sdf3-szfdsafsdf3423-zxdfsd3
|
||
|
|
nomad volume register example_volume.hcl
|
||
|
|
```
|
||
|
|
|
||
|
|
## Ceph CSI Driver Source
|
||
|
|
|
||
|
|
- https://github.com/ceph/ceph-csi
|