luxolus
/
open-nomad
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity

CSI: plugin supervisor prestart should not mark itself done (#12752) 

The task runner hook `Prestart` response object includes a `Done`
field that's intended to tell the client not to run the hook
again. The plugin supervisor creates mount points for the task during
prestart and saves these mounts in the hook resources. But if a client
restarts the hook resources will not be populated. If the plugin task
restarts at any time after the client restarts, it will fail to have
the correct mounts and crash loop until restart attempts run out.

Fix this by not returning `Done` in the response, just as we do for
the `volume_mount_hook`.
This commit is contained in:
Tim Gross 2022-04-22 13:07:47 -04:00 committed by GitHub
parent 24b499791d
commit 766025cde7
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 10 additions and 5 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.changelog/12752.txt Normal file
View File

@ -0,0 +1,3 @@
```release-note:bug
csi: Fixed a bug where plugins would not restart if they failed any time after a client restart
```

12
client/allocrunner/taskrunner/plugin_supervisor_hook.go
View File

@ -129,10 +129,10 @@ func (*csiPluginSupervisorHook) Name() string {
} }
// Prestart is called before the task is started including after every // Prestart is called before the task is started including after every
// restart (but not after restore). This requires that the mount paths // restart. This requires that the mount paths for a plugin be
// for a plugin be idempotent, despite us not knowing the name of the // idempotent, despite us not knowing the name of the plugin ahead of
// plugin ahead of time. Because of this, we use the allocid_taskname // time. Because of this, we use the allocid_taskname as the unique
// as the unique identifier for a plugin on the filesystem. // identifier for a plugin on the filesystem.
func (h *csiPluginSupervisorHook) Prestart(ctx context.Context, func (h *csiPluginSupervisorHook) Prestart(ctx context.Context,
req *interfaces.TaskPrestartRequest, resp *interfaces.TaskPrestartResponse) error { req *interfaces.TaskPrestartRequest, resp *interfaces.TaskPrestartResponse) error {
@ -191,9 +191,11 @@ func (h *csiPluginSupervisorHook) Prestart(ctx context.Context,
mounts = ensureMountpointInserted(mounts, volumeStagingMounts) mounts = ensureMountpointInserted(mounts, volumeStagingMounts)
mounts = ensureMountpointInserted(mounts, devMount) mounts = ensureMountpointInserted(mounts, devMount)
// we normally would set resp.Mounts here but without setting the
// hookResources before returning we can get a postrun hook that's
// missing resources.
h.runner.hookResources.setMounts(mounts) h.runner.hookResources.setMounts(mounts)
resp.Done = true
return nil return nil
} }