open-vault/sdk/plugin/backend.go

74 lines
1.9 KiB
Go
Raw Normal View History

Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
package plugin
import (
"context"
"sync/atomic"
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
"google.golang.org/grpc"
log "github.com/hashicorp/go-hclog"
"github.com/hashicorp/go-plugin"
feature: secrets/auth plugin multiplexing (#14946) * enable registering backend muxed plugins in plugin catalog * set the sysview on the pluginconfig to allow enabling secrets/auth plugins * store backend instances in map * store single implementations in the instances map cleanup instance map and ensure we don't deadlock * fix system backend unit tests move GetMultiplexIDFromContext to pluginutil package fix pluginutil test fix dbplugin ut * return error(s) if we can't get the plugin client update comments * refactor/move GetMultiplexIDFromContext test * add changelog * remove unnecessary field on pluginClient * add unit tests to PluginCatalog for secrets/auth plugins * fix comment * return pluginClient from TestRunTestPlugin * add multiplexed backend test * honor metadatamode value in newbackend pluginconfig * check that connection exists on cleanup * add automtls to secrets/auth plugins * don't remove apiclientmeta parsing * use formatting directive for fmt.Errorf * fix ut: remove tls provider func * remove tlsproviderfunc from backend plugin tests * use env var to prevent test plugin from running as a unit test * WIP: remove lazy loading * move non lazy loaded backend to new package * use version wrapper for backend plugin factory * remove backendVersionWrapper type * implement getBackendPluginType for plugin catalog * handle backend plugin v4 registration * add plugin automtls env guard * modify plugin factory to determine the backend to use * remove old pluginsets from v5 and log pid in plugin catalog * add reload mechanism via context * readd v3 and v4 to pluginset * call cleanup from reload if non-muxed * move v5 backend code to new package * use context reload for for ErrPluginShutdown case * add wrapper on v5 backend * fix run config UTs * fix unit tests - use v4/v5 mapping for plugin versions - fix test build err - add reload method on fakePluginClient - add multiplexed cases for integration tests * remove comment and update AutoMTLS field in test * remove comment * remove errwrap and unused context * only support metadatamode false for v5 backend plugins * update plugin catalog errors * use const for env variables * rename locks and remove unused * remove unneeded nil check * improvements based on staticcheck recommendations * use const for single implementation string * use const for context key * use info default log level * move pid to pluginClient struct * remove v3 and v4 from multiplexed plugin set * return from reload when non-multiplexed * update automtls env string * combine getBackend and getBrokeredClient * update comments for plugin reload, Backend return val and log * revert Backend return type * allow non-muxed plugins to serve v5 * move v5 code to existing sdk plugin package * do next export sdk fields now that we have removed extra plugin pkg * set TLSProvider in ServeMultiplex for backwards compat * use bool to flag multiplexing support on grpc backend server * revert userpass main.go * refactor plugin sdk - update comments - make use of multiplexing boolean and single implementation ID const * update comment and use multierr * attempt v4 if dispense fails on getPluginTypeForUnknown * update comments on sdk plugin backend
2022-08-30 02:42:26 +00:00
"github.com/hashicorp/vault/sdk/helper/pluginutil"
"github.com/hashicorp/vault/sdk/logical"
"github.com/hashicorp/vault/sdk/plugin/pb"
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
)
var (
_ plugin.Plugin = (*GRPCBackendPlugin)(nil)
_ plugin.GRPCPlugin = (*GRPCBackendPlugin)(nil)
)
// GRPCBackendPlugin is the plugin.Plugin implementation that only supports GRPC
// transport
type GRPCBackendPlugin struct {
Factory logical.Factory
MetadataMode bool
Logger log.Logger
feature: secrets/auth plugin multiplexing (#14946) * enable registering backend muxed plugins in plugin catalog * set the sysview on the pluginconfig to allow enabling secrets/auth plugins * store backend instances in map * store single implementations in the instances map cleanup instance map and ensure we don't deadlock * fix system backend unit tests move GetMultiplexIDFromContext to pluginutil package fix pluginutil test fix dbplugin ut * return error(s) if we can't get the plugin client update comments * refactor/move GetMultiplexIDFromContext test * add changelog * remove unnecessary field on pluginClient * add unit tests to PluginCatalog for secrets/auth plugins * fix comment * return pluginClient from TestRunTestPlugin * add multiplexed backend test * honor metadatamode value in newbackend pluginconfig * check that connection exists on cleanup * add automtls to secrets/auth plugins * don't remove apiclientmeta parsing * use formatting directive for fmt.Errorf * fix ut: remove tls provider func * remove tlsproviderfunc from backend plugin tests * use env var to prevent test plugin from running as a unit test * WIP: remove lazy loading * move non lazy loaded backend to new package * use version wrapper for backend plugin factory * remove backendVersionWrapper type * implement getBackendPluginType for plugin catalog * handle backend plugin v4 registration * add plugin automtls env guard * modify plugin factory to determine the backend to use * remove old pluginsets from v5 and log pid in plugin catalog * add reload mechanism via context * readd v3 and v4 to pluginset * call cleanup from reload if non-muxed * move v5 backend code to new package * use context reload for for ErrPluginShutdown case * add wrapper on v5 backend * fix run config UTs * fix unit tests - use v4/v5 mapping for plugin versions - fix test build err - add reload method on fakePluginClient - add multiplexed cases for integration tests * remove comment and update AutoMTLS field in test * remove comment * remove errwrap and unused context * only support metadatamode false for v5 backend plugins * update plugin catalog errors * use const for env variables * rename locks and remove unused * remove unneeded nil check * improvements based on staticcheck recommendations * use const for single implementation string * use const for context key * use info default log level * move pid to pluginClient struct * remove v3 and v4 from multiplexed plugin set * return from reload when non-multiplexed * update automtls env string * combine getBackend and getBrokeredClient * update comments for plugin reload, Backend return val and log * revert Backend return type * allow non-muxed plugins to serve v5 * move v5 code to existing sdk plugin package * do next export sdk fields now that we have removed extra plugin pkg * set TLSProvider in ServeMultiplex for backwards compat * use bool to flag multiplexing support on grpc backend server * revert userpass main.go * refactor plugin sdk - update comments - make use of multiplexing boolean and single implementation ID const * update comment and use multierr * attempt v4 if dispense fails on getPluginTypeForUnknown * update comments on sdk plugin backend
2022-08-30 02:42:26 +00:00
MultiplexingSupport bool
// Embeding this will disable the netRPC protocol
plugin.NetRPCUnsupportedPlugin
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
}
func (b GRPCBackendPlugin) GRPCServer(broker *plugin.GRPCBroker, s *grpc.Server) error {
feature: secrets/auth plugin multiplexing (#14946) * enable registering backend muxed plugins in plugin catalog * set the sysview on the pluginconfig to allow enabling secrets/auth plugins * store backend instances in map * store single implementations in the instances map cleanup instance map and ensure we don't deadlock * fix system backend unit tests move GetMultiplexIDFromContext to pluginutil package fix pluginutil test fix dbplugin ut * return error(s) if we can't get the plugin client update comments * refactor/move GetMultiplexIDFromContext test * add changelog * remove unnecessary field on pluginClient * add unit tests to PluginCatalog for secrets/auth plugins * fix comment * return pluginClient from TestRunTestPlugin * add multiplexed backend test * honor metadatamode value in newbackend pluginconfig * check that connection exists on cleanup * add automtls to secrets/auth plugins * don't remove apiclientmeta parsing * use formatting directive for fmt.Errorf * fix ut: remove tls provider func * remove tlsproviderfunc from backend plugin tests * use env var to prevent test plugin from running as a unit test * WIP: remove lazy loading * move non lazy loaded backend to new package * use version wrapper for backend plugin factory * remove backendVersionWrapper type * implement getBackendPluginType for plugin catalog * handle backend plugin v4 registration * add plugin automtls env guard * modify plugin factory to determine the backend to use * remove old pluginsets from v5 and log pid in plugin catalog * add reload mechanism via context * readd v3 and v4 to pluginset * call cleanup from reload if non-muxed * move v5 backend code to new package * use context reload for for ErrPluginShutdown case * add wrapper on v5 backend * fix run config UTs * fix unit tests - use v4/v5 mapping for plugin versions - fix test build err - add reload method on fakePluginClient - add multiplexed cases for integration tests * remove comment and update AutoMTLS field in test * remove comment * remove errwrap and unused context * only support metadatamode false for v5 backend plugins * update plugin catalog errors * use const for env variables * rename locks and remove unused * remove unneeded nil check * improvements based on staticcheck recommendations * use const for single implementation string * use const for context key * use info default log level * move pid to pluginClient struct * remove v3 and v4 from multiplexed plugin set * return from reload when non-multiplexed * update automtls env string * combine getBackend and getBrokeredClient * update comments for plugin reload, Backend return val and log * revert Backend return type * allow non-muxed plugins to serve v5 * move v5 code to existing sdk plugin package * do next export sdk fields now that we have removed extra plugin pkg * set TLSProvider in ServeMultiplex for backwards compat * use bool to flag multiplexing support on grpc backend server * revert userpass main.go * refactor plugin sdk - update comments - make use of multiplexing boolean and single implementation ID const * update comment and use multierr * attempt v4 if dispense fails on getPluginTypeForUnknown * update comments on sdk plugin backend
2022-08-30 02:42:26 +00:00
server := backendGRPCPluginServer{
broker: broker,
factory: b.Factory,
instances: make(map[string]backendInstance),
// We pass the logger down into the backend so go-plugin will
// forward logs for us.
logger: b.Logger,
feature: secrets/auth plugin multiplexing (#14946) * enable registering backend muxed plugins in plugin catalog * set the sysview on the pluginconfig to allow enabling secrets/auth plugins * store backend instances in map * store single implementations in the instances map cleanup instance map and ensure we don't deadlock * fix system backend unit tests move GetMultiplexIDFromContext to pluginutil package fix pluginutil test fix dbplugin ut * return error(s) if we can't get the plugin client update comments * refactor/move GetMultiplexIDFromContext test * add changelog * remove unnecessary field on pluginClient * add unit tests to PluginCatalog for secrets/auth plugins * fix comment * return pluginClient from TestRunTestPlugin * add multiplexed backend test * honor metadatamode value in newbackend pluginconfig * check that connection exists on cleanup * add automtls to secrets/auth plugins * don't remove apiclientmeta parsing * use formatting directive for fmt.Errorf * fix ut: remove tls provider func * remove tlsproviderfunc from backend plugin tests * use env var to prevent test plugin from running as a unit test * WIP: remove lazy loading * move non lazy loaded backend to new package * use version wrapper for backend plugin factory * remove backendVersionWrapper type * implement getBackendPluginType for plugin catalog * handle backend plugin v4 registration * add plugin automtls env guard * modify plugin factory to determine the backend to use * remove old pluginsets from v5 and log pid in plugin catalog * add reload mechanism via context * readd v3 and v4 to pluginset * call cleanup from reload if non-muxed * move v5 backend code to new package * use context reload for for ErrPluginShutdown case * add wrapper on v5 backend * fix run config UTs * fix unit tests - use v4/v5 mapping for plugin versions - fix test build err - add reload method on fakePluginClient - add multiplexed cases for integration tests * remove comment and update AutoMTLS field in test * remove comment * remove errwrap and unused context * only support metadatamode false for v5 backend plugins * update plugin catalog errors * use const for env variables * rename locks and remove unused * remove unneeded nil check * improvements based on staticcheck recommendations * use const for single implementation string * use const for context key * use info default log level * move pid to pluginClient struct * remove v3 and v4 from multiplexed plugin set * return from reload when non-multiplexed * update automtls env string * combine getBackend and getBrokeredClient * update comments for plugin reload, Backend return val and log * revert Backend return type * allow non-muxed plugins to serve v5 * move v5 code to existing sdk plugin package * do next export sdk fields now that we have removed extra plugin pkg * set TLSProvider in ServeMultiplex for backwards compat * use bool to flag multiplexing support on grpc backend server * revert userpass main.go * refactor plugin sdk - update comments - make use of multiplexing boolean and single implementation ID const * update comment and use multierr * attempt v4 if dispense fails on getPluginTypeForUnknown * update comments on sdk plugin backend
2022-08-30 02:42:26 +00:00
}
if b.MultiplexingSupport {
// Multiplexing is enabled for this plugin, register the server so we
// can tell the client in Vault.
pluginutil.RegisterPluginMultiplexingServer(s, pluginutil.PluginMultiplexingServerImpl{
Supported: true,
})
server.multiplexingSupport = true
}
pb.RegisterBackendServer(s, &server)
logical.RegisterPluginVersionServer(s, &server)
return nil
}
func (b *GRPCBackendPlugin) GRPCClient(ctx context.Context, broker *plugin.GRPCBroker, c *grpc.ClientConn) (interface{}, error) {
ret := &backendGRPCPluginClient{
client: pb.NewBackendClient(c),
versionClient: logical.NewPluginVersionClient(c),
broker: broker,
cleanupCh: make(chan struct{}),
doneCtx: ctx,
metadataMode: b.MetadataMode,
}
// Create the value and set the type
ret.server = new(atomic.Value)
ret.server.Store((*grpc.Server)(nil))
return ret, nil
}