Commit graph

17524 commits

Author SHA1 Message Date
Danielle Lancashire 51270ae0f4 csi: Support for NodeUnpublishVolume RPCs 2020-03-23 13:58:29 -04:00
Danielle Lancashire a4b96aff33 csi: Nil check ToCSIRepresentation implementations 2020-03-23 13:58:29 -04:00
Danielle Lancashire 02c4612e65 csi: Add NodePublishVolume RPCs 2020-03-23 13:58:29 -04:00
Danielle Lancashire 98f00a9220 csi: Add NodeUnstageVolume RPCs to CSIPlugin 2020-03-23 13:58:29 -04:00
Danielle Lancashire 5c447396fa csi: Add NodeUnstageVolume as a CSI Dependency 2020-03-23 13:58:29 -04:00
Danielle Lancashire f208770e94 csi: Add NodeStageVolume to fake client 2020-03-23 13:58:29 -04:00
Danielle Lancashire 07651a5231 csi: Add NodeStageVolume RPC 2020-03-23 13:58:29 -04:00
Danielle Lancashire 317b680744 csi: Add csi.NodeStageVolume to the NodeClient
Implements a fake version of NodeStageVolume as a dependency of
implementing the client.NodeStageVolume request
2020-03-23 13:58:29 -04:00
Danielle Lancashire ab1edd4e24 csi: Add Nomad Model for VolumeCapabilities
This commit introduces a nomad model for interacting with CSI
VolumeCapabilities as a pre-requisite for implementing NodeStageVolume
and NodeMountVolume correctly.

These fields have a few special characteristics that I've tried to model
here - specificially, we make a basic attempt to avoid printing data
that should be redacted during debug logs (additional mount flags), and
also attempt to make debuggability of other integer fields easier by
implementing the fmt.Stringer and fmt.GoStringer interfaces as
necessary.

We do not currnetly implement a CSI Protobuf -> Nomad implementation
transformation as this is currently not needed by any used RPCs.
2020-03-23 13:58:29 -04:00
Lang Martin 61cfc806ad csi_volume_endpoint_test: gen uuids to avoid t.Parallel conflicts 2020-03-23 13:58:29 -04:00
Lang Martin af857ae660 docs contributing: checklist-command 2020-03-23 13:58:29 -04:00
Lang Martin 334979a754 nomad/rpc: indicate missing region in error message 2020-03-23 13:58:29 -04:00
Lang Martin 2f646fa5e9 agent: csi endpoint 2020-03-23 13:58:29 -04:00
Lang Martin 6106a388e6 api: csi 2020-03-23 13:58:29 -04:00
Lang Martin 5b31b140c3 csi: do not use namespace specific identifiers 2020-03-23 13:58:29 -04:00
Lang Martin e922531aaf structs: move the content of csi_volumes into csi 2020-03-23 13:58:29 -04:00
Lang Martin 41cbd55657 client structs: use nstructs rather than s for nomad/structs 2020-03-23 13:58:29 -04:00
Lang Martin 04b6e7c7fb server: rpc register CSIVolume 2020-03-23 13:58:29 -04:00
Lang Martin 8f33fb9a6d csi volume endpoint: new RPC endpoint for CSI volumes 2020-03-23 13:58:29 -04:00
Lang Martin 03817e16d4 acl: new NamespaceCapabilityCSIAccess, CSICreateVolume 2020-03-23 13:58:29 -04:00
Lang Martin 4bb4dd98eb state_store: CSIVolume insert, get, delete, claim
state_store: change claim counts

state_store: get volumes by all, by driver

state_store: process volume claims

state_store: csi volume register error on update
2020-03-23 13:58:29 -04:00
Lang Martin 0422b967db schema: csi_volumes schema 2020-03-23 13:58:29 -04:00
Lang Martin 857cd37ab5 fsm: dispatch CSIVolume register, deregister, claim 2020-03-23 13:58:29 -04:00
Lang Martin 2eb288426c docs: update checklist-rpc-endpoint 2020-03-23 13:58:29 -04:00
Lang Martin f9d9faf673 structs: eliminate MaxReaders & MaxWriters 2020-03-23 13:58:29 -04:00
Lang Martin 3a7e1b6d14 client structs: move CSIVolumeAttachmentMode and CSIVolumeAccessMode 2020-03-23 13:58:29 -04:00
Lang Martin 637ce9dfad structs: new CSIVolume, request types 2020-03-23 13:58:29 -04:00
Danielle Lancashire de5d373001 csi: Setup gRPC Clients with a logger 2020-03-23 13:58:29 -04:00
Danielle Lancashire 973a61935e helper: Add initial grpc logging middleware 2020-03-23 13:58:29 -04:00
Danielle Lancashire 8fb312e48e node_status: Add CSI Summary info
This commit introduces two new fields to the basic output of `nomad
node status <node-id>`.

1) "CSI Controllers", which displays the names of registered controller
plugins.

2) "CSI Drivers", which displays the names of registered CSI Node
plugins.

However, it does not implement support for verbose output, such as
including health status or other fingerprinted data.
2020-03-23 13:58:29 -04:00
Danielle Lancashire 78b7784f2b api: Include CSI metadata on nodes 2020-03-23 13:58:29 -04:00
Danielle Lancashire 57ae1d2cd6 csimanager: Fingerprint Node Service capabilities 2020-03-23 13:58:29 -04:00
Danielle Lancashire c16812280c csi: Add NodeGetCapabilities RPC 2020-03-23 13:58:29 -04:00
Danielle Lancashire 564f5cec93 csimanager: Fingerprint controller capabilities 2020-03-23 13:58:29 -04:00
Danielle Lancashire 05525c98ae plugins_csi: Add GetControllerCapabilities RPC 2020-03-23 13:58:28 -04:00
Danielle Lancashire 9a23e27439 client_csi: Validate Access/Attachment modes 2020-03-23 13:58:28 -04:00
Danielle Lancashire 2fc65371a8 csi: ClientCSIControllerPublish* -> ClientCSIControllerAttach* 2020-03-23 13:58:28 -04:00
Danielle Lancashire 259852b05f csi: Model Attachment and Access modes 2020-03-23 13:58:28 -04:00
Danielle Lancashire 2c29b1c53d client: Setup CSI RPC Endpoint
This commit introduces a new set of endpoints to a Nomad Client:
ClientCSI.

ClientCSI is responsible for mediating requests from a Nomad Server to
a CSI Plugin running on a Nomad Client. It should only really be used to
make controller RPCs.
2020-03-23 13:58:28 -04:00
Danielle Lancashire 72ee2d4c1c csi: Add initial plumbing for controller rpcs 2020-03-23 13:58:28 -04:00
Danielle Lancashire 426c26d7c0 CSI Plugin Registration (#6555)
This changeset implements the initial registration and fingerprinting
of CSI Plugins as part of #5378. At a high level, it introduces the
following:

* A `csi_plugin` stanza as part of a Nomad task configuration, to
  allow a task to expose that it is a plugin.

* A new task runner hook: `csi_plugin_supervisor`. This hook does two
  things. When the `csi_plugin` stanza is detected, it will
  automatically configure the plugin task to receive bidirectional
  mounts to the CSI intermediary directory. At runtime, it will then
  perform an initial heartbeat of the plugin and handle submitting it to
  the new `dynamicplugins.Registry` for further use by the client, and
  then run a lightweight heartbeat loop that will emit task events
  when health changes.

* The `dynamicplugins.Registry` for handling plugins that run
  as Nomad tasks, in contrast to the existing catalog that requires
  `go-plugin` type plugins and to know the plugin configuration in
  advance.

* The `csimanager` which fingerprints CSI plugins, in a similar way to
  `drivermanager` and `devicemanager`. It currently only fingerprints
  the NodeID from the plugin, and assumes that all plugins are
  monolithic.

Missing features

* We do not use the live updates of the `dynamicplugin` registry in
  the `csimanager` yet.

* We do not deregister the plugins from the client when they shutdown
  yet, they just become indefinitely marked as unhealthy. This is
  deliberate until we figure out how we should manage deploying new
  versions of plugins/transitioning them.
2020-03-23 13:58:28 -04:00
Lang Martin 55ed831c56 vendor: Add github.com/grpc-ecosystem/go-grpc-middleware/retry 2020-03-23 13:58:28 -04:00
Danielle Lancashire e2450bb251 vendor: go-hclog 0.10.1 -> 0.11.0 2020-03-23 13:58:28 -04:00
Danielle Lancashire 438657d822 vendor: Init csi protobufs 2020-03-23 13:58:28 -04:00
Drew Bailey b09abef332
Audit config, seams for enterprise audit features
allow oss to parse sink duration

clean up audit sink parsing

ent eventer config reload

fix typo

SetEnabled to eventer interface

client acl test

rm dead code

fix failing test
2020-03-23 13:47:42 -04:00
Mahmood Ali 17059d0a1e ci: use curl
It's installed by default in macOS unlike wget
2020-03-23 13:27:58 -04:00
Mahmood Ali 4f6a2cf65c ci: build darwin on CI
build darwin binaries in CI as a basic check.  Protects against
cross compile issues.
2020-03-23 13:27:24 -04:00
Mahmood Ali 8142d42f85
Merge pull request #7431 from hashicorp/ci-golang-1.14.1
use golang 14.1
2020-03-23 13:22:11 -04:00
Mahmood Ali 3f27ad14d8 use golang 14.1 2020-03-23 13:02:14 -04:00
Jonathan Neal 1375272507 Replace internal DocsPage with @hashicorp/react-docs-page 2020-03-23 12:28:00 -04:00