open-vault/sdk/plugin/serve.go

165 lines
4.3 KiB
Go
Raw Normal View History

Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
package plugin
import (
"crypto/tls"
"math"
"os"
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
"google.golang.org/grpc"
log "github.com/hashicorp/go-hclog"
plugin "github.com/hashicorp/go-plugin"
"github.com/hashicorp/vault/sdk/helper/pluginutil"
"github.com/hashicorp/vault/sdk/logical"
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
)
// BackendPluginName is the name of the plugin that can be
// dispensed from the plugin server.
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
const BackendPluginName = "backend"
2018-03-20 18:54:10 +00:00
type TLSProviderFunc func() (*tls.Config, error)
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
type ServeOpts struct {
BackendFactoryFunc logical.Factory
2018-03-20 18:54:10 +00:00
TLSProviderFunc TLSProviderFunc
Logger log.Logger
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
}
Lazy-load plugin mounts (#3255) * Lazy load plugins to avoid setup-unwrap cycle * Remove commented blocks * Refactor NewTestCluster, use single core cluster on basic plugin tests * Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly * Add special path to mock plugin * Move ensureCoresSealed to vault/testing.go * Use same method for EnsureCoresSealed and Cleanup * Bump ensureCoresSealed timeout to 60s * Correctly handle nil opts on NewTestCluster * Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap * Check metadata flag directly on the plugin process * Plumb isMetadataMode down to PluginRunner * Add NOOP shims when running in metadata mode * Remove unused flag from the APIMetadata object * Remove setupSecretPlugins and setupCredentialPlugins functions * Move when we setup rollback manager to after the plugins are initialized * Fix tests * Fix merge issue * start rollback manager after the credential setup * Add guards against running certain client and server functions while in metadata mode * Call initialize once a plugin is loaded on the fly * Add more tests, update basic secret/auth plugin tests to trigger lazy loading * Skip mount if plugin removed from catalog * Fixup * Remove commented line on LookupPlugin * Fail on mount operation if plugin is re-added to catalog and mount is on existing path * Check type and special paths on startBackend * Fix merge conflicts * Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 05:02:03 +00:00
// Serve is a helper function used to serve a backend plugin. This
// should be ran on the plugin's main process.
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
func Serve(opts *ServeOpts) error {
logger := opts.Logger
if logger == nil {
logger = log.New(&log.LoggerOptions{
Level: log.Trace,
Output: os.Stderr,
JSONFormat: true,
})
}
// pluginMap is the map of plugins we can dispense.
pluginSets := map[int]plugin.PluginSet{
// Version 3 used to supports both protocols. We want to keep it around
// since it's possible old plugins built against this version will still
// work with gRPC. There is currently no difference between version 3
// and version 4.
3: {
"backend": &GRPCBackendPlugin{
Factory: opts.BackendFactoryFunc,
Logger: logger,
},
},
4: {
"backend": &GRPCBackendPlugin{
Factory: opts.BackendFactoryFunc,
Logger: logger,
},
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
},
feature: secrets/auth plugin multiplexing (#14946) * enable registering backend muxed plugins in plugin catalog * set the sysview on the pluginconfig to allow enabling secrets/auth plugins * store backend instances in map * store single implementations in the instances map cleanup instance map and ensure we don't deadlock * fix system backend unit tests move GetMultiplexIDFromContext to pluginutil package fix pluginutil test fix dbplugin ut * return error(s) if we can't get the plugin client update comments * refactor/move GetMultiplexIDFromContext test * add changelog * remove unnecessary field on pluginClient * add unit tests to PluginCatalog for secrets/auth plugins * fix comment * return pluginClient from TestRunTestPlugin * add multiplexed backend test * honor metadatamode value in newbackend pluginconfig * check that connection exists on cleanup * add automtls to secrets/auth plugins * don't remove apiclientmeta parsing * use formatting directive for fmt.Errorf * fix ut: remove tls provider func * remove tlsproviderfunc from backend plugin tests * use env var to prevent test plugin from running as a unit test * WIP: remove lazy loading * move non lazy loaded backend to new package * use version wrapper for backend plugin factory * remove backendVersionWrapper type * implement getBackendPluginType for plugin catalog * handle backend plugin v4 registration * add plugin automtls env guard * modify plugin factory to determine the backend to use * remove old pluginsets from v5 and log pid in plugin catalog * add reload mechanism via context * readd v3 and v4 to pluginset * call cleanup from reload if non-muxed * move v5 backend code to new package * use context reload for for ErrPluginShutdown case * add wrapper on v5 backend * fix run config UTs * fix unit tests - use v4/v5 mapping for plugin versions - fix test build err - add reload method on fakePluginClient - add multiplexed cases for integration tests * remove comment and update AutoMTLS field in test * remove comment * remove errwrap and unused context * only support metadatamode false for v5 backend plugins * update plugin catalog errors * use const for env variables * rename locks and remove unused * remove unneeded nil check * improvements based on staticcheck recommendations * use const for single implementation string * use const for context key * use info default log level * move pid to pluginClient struct * remove v3 and v4 from multiplexed plugin set * return from reload when non-multiplexed * update automtls env string * combine getBackend and getBrokeredClient * update comments for plugin reload, Backend return val and log * revert Backend return type * allow non-muxed plugins to serve v5 * move v5 code to existing sdk plugin package * do next export sdk fields now that we have removed extra plugin pkg * set TLSProvider in ServeMultiplex for backwards compat * use bool to flag multiplexing support on grpc backend server * revert userpass main.go * refactor plugin sdk - update comments - make use of multiplexing boolean and single implementation ID const * update comment and use multierr * attempt v4 if dispense fails on getPluginTypeForUnknown * update comments on sdk plugin backend
2022-08-30 02:42:26 +00:00
5: {
"backend": &GRPCBackendPlugin{
Factory: opts.BackendFactoryFunc,
MultiplexingSupport: false,
Logger: logger,
},
},
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
}
err := pluginutil.OptionallyEnableMlock()
if err != nil {
return err
}
serveOpts := &plugin.ServeConfig{
feature: secrets/auth plugin multiplexing (#14946) * enable registering backend muxed plugins in plugin catalog * set the sysview on the pluginconfig to allow enabling secrets/auth plugins * store backend instances in map * store single implementations in the instances map cleanup instance map and ensure we don't deadlock * fix system backend unit tests move GetMultiplexIDFromContext to pluginutil package fix pluginutil test fix dbplugin ut * return error(s) if we can't get the plugin client update comments * refactor/move GetMultiplexIDFromContext test * add changelog * remove unnecessary field on pluginClient * add unit tests to PluginCatalog for secrets/auth plugins * fix comment * return pluginClient from TestRunTestPlugin * add multiplexed backend test * honor metadatamode value in newbackend pluginconfig * check that connection exists on cleanup * add automtls to secrets/auth plugins * don't remove apiclientmeta parsing * use formatting directive for fmt.Errorf * fix ut: remove tls provider func * remove tlsproviderfunc from backend plugin tests * use env var to prevent test plugin from running as a unit test * WIP: remove lazy loading * move non lazy loaded backend to new package * use version wrapper for backend plugin factory * remove backendVersionWrapper type * implement getBackendPluginType for plugin catalog * handle backend plugin v4 registration * add plugin automtls env guard * modify plugin factory to determine the backend to use * remove old pluginsets from v5 and log pid in plugin catalog * add reload mechanism via context * readd v3 and v4 to pluginset * call cleanup from reload if non-muxed * move v5 backend code to new package * use context reload for for ErrPluginShutdown case * add wrapper on v5 backend * fix run config UTs * fix unit tests - use v4/v5 mapping for plugin versions - fix test build err - add reload method on fakePluginClient - add multiplexed cases for integration tests * remove comment and update AutoMTLS field in test * remove comment * remove errwrap and unused context * only support metadatamode false for v5 backend plugins * update plugin catalog errors * use const for env variables * rename locks and remove unused * remove unneeded nil check * improvements based on staticcheck recommendations * use const for single implementation string * use const for context key * use info default log level * move pid to pluginClient struct * remove v3 and v4 from multiplexed plugin set * return from reload when non-multiplexed * update automtls env string * combine getBackend and getBrokeredClient * update comments for plugin reload, Backend return val and log * revert Backend return type * allow non-muxed plugins to serve v5 * move v5 code to existing sdk plugin package * do next export sdk fields now that we have removed extra plugin pkg * set TLSProvider in ServeMultiplex for backwards compat * use bool to flag multiplexing support on grpc backend server * revert userpass main.go * refactor plugin sdk - update comments - make use of multiplexing boolean and single implementation ID const * update comment and use multierr * attempt v4 if dispense fails on getPluginTypeForUnknown * update comments on sdk plugin backend
2022-08-30 02:42:26 +00:00
HandshakeConfig: HandshakeConfig,
VersionedPlugins: pluginSets,
TLSProvider: opts.TLSProviderFunc,
Logger: logger,
// A non-nil value here enables gRPC serving for this plugin...
GRPCServer: func(opts []grpc.ServerOption) *grpc.Server {
opts = append(opts, grpc.MaxRecvMsgSize(math.MaxInt32))
opts = append(opts, grpc.MaxSendMsgSize(math.MaxInt32))
return plugin.DefaultGRPCServer(opts)
},
}
plugin.Serve(serveOpts)
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
return nil
}
feature: secrets/auth plugin multiplexing (#14946) * enable registering backend muxed plugins in plugin catalog * set the sysview on the pluginconfig to allow enabling secrets/auth plugins * store backend instances in map * store single implementations in the instances map cleanup instance map and ensure we don't deadlock * fix system backend unit tests move GetMultiplexIDFromContext to pluginutil package fix pluginutil test fix dbplugin ut * return error(s) if we can't get the plugin client update comments * refactor/move GetMultiplexIDFromContext test * add changelog * remove unnecessary field on pluginClient * add unit tests to PluginCatalog for secrets/auth plugins * fix comment * return pluginClient from TestRunTestPlugin * add multiplexed backend test * honor metadatamode value in newbackend pluginconfig * check that connection exists on cleanup * add automtls to secrets/auth plugins * don't remove apiclientmeta parsing * use formatting directive for fmt.Errorf * fix ut: remove tls provider func * remove tlsproviderfunc from backend plugin tests * use env var to prevent test plugin from running as a unit test * WIP: remove lazy loading * move non lazy loaded backend to new package * use version wrapper for backend plugin factory * remove backendVersionWrapper type * implement getBackendPluginType for plugin catalog * handle backend plugin v4 registration * add plugin automtls env guard * modify plugin factory to determine the backend to use * remove old pluginsets from v5 and log pid in plugin catalog * add reload mechanism via context * readd v3 and v4 to pluginset * call cleanup from reload if non-muxed * move v5 backend code to new package * use context reload for for ErrPluginShutdown case * add wrapper on v5 backend * fix run config UTs * fix unit tests - use v4/v5 mapping for plugin versions - fix test build err - add reload method on fakePluginClient - add multiplexed cases for integration tests * remove comment and update AutoMTLS field in test * remove comment * remove errwrap and unused context * only support metadatamode false for v5 backend plugins * update plugin catalog errors * use const for env variables * rename locks and remove unused * remove unneeded nil check * improvements based on staticcheck recommendations * use const for single implementation string * use const for context key * use info default log level * move pid to pluginClient struct * remove v3 and v4 from multiplexed plugin set * return from reload when non-multiplexed * update automtls env string * combine getBackend and getBrokeredClient * update comments for plugin reload, Backend return val and log * revert Backend return type * allow non-muxed plugins to serve v5 * move v5 code to existing sdk plugin package * do next export sdk fields now that we have removed extra plugin pkg * set TLSProvider in ServeMultiplex for backwards compat * use bool to flag multiplexing support on grpc backend server * revert userpass main.go * refactor plugin sdk - update comments - make use of multiplexing boolean and single implementation ID const * update comment and use multierr * attempt v4 if dispense fails on getPluginTypeForUnknown * update comments on sdk plugin backend
2022-08-30 02:42:26 +00:00
// ServeMultiplex is a helper function used to serve a backend plugin. This
// should be ran on the plugin's main process.
func ServeMultiplex(opts *ServeOpts) error {
logger := opts.Logger
if logger == nil {
logger = log.New(&log.LoggerOptions{
Level: log.Info,
Output: os.Stderr,
JSONFormat: true,
})
}
// pluginMap is the map of plugins we can dispense.
pluginSets := map[int]plugin.PluginSet{
// Version 3 used to supports both protocols. We want to keep it around
// since it's possible old plugins built against this version will still
// work with gRPC. There is currently no difference between version 3
// and version 4.
3: {
"backend": &GRPCBackendPlugin{
Factory: opts.BackendFactoryFunc,
Logger: logger,
},
},
4: {
"backend": &GRPCBackendPlugin{
Factory: opts.BackendFactoryFunc,
Logger: logger,
},
},
5: {
"backend": &GRPCBackendPlugin{
Factory: opts.BackendFactoryFunc,
MultiplexingSupport: true,
Logger: logger,
},
},
}
err := pluginutil.OptionallyEnableMlock()
if err != nil {
return err
}
serveOpts := &plugin.ServeConfig{
HandshakeConfig: HandshakeConfig,
VersionedPlugins: pluginSets,
Logger: logger,
// A non-nil value here enables gRPC serving for this plugin...
GRPCServer: func(opts []grpc.ServerOption) *grpc.Server {
opts = append(opts, grpc.MaxRecvMsgSize(math.MaxInt32))
opts = append(opts, grpc.MaxSendMsgSize(math.MaxInt32))
return plugin.DefaultGRPCServer(opts)
},
// TLSProvider is required to support v3 and v4 plugins.
// It will be ignored for v5 which uses AutoMTLS
TLSProvider: opts.TLSProviderFunc,
}
plugin.Serve(serveOpts)
return nil
}
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
// handshakeConfigs are used to just do a basic handshake between
// a plugin and host. If the handshake fails, a user friendly error is shown.
// This prevents users from executing bad plugins or executing a plugin
// directory. It is a UX feature, not a security feature.
feature: secrets/auth plugin multiplexing (#14946) * enable registering backend muxed plugins in plugin catalog * set the sysview on the pluginconfig to allow enabling secrets/auth plugins * store backend instances in map * store single implementations in the instances map cleanup instance map and ensure we don't deadlock * fix system backend unit tests move GetMultiplexIDFromContext to pluginutil package fix pluginutil test fix dbplugin ut * return error(s) if we can't get the plugin client update comments * refactor/move GetMultiplexIDFromContext test * add changelog * remove unnecessary field on pluginClient * add unit tests to PluginCatalog for secrets/auth plugins * fix comment * return pluginClient from TestRunTestPlugin * add multiplexed backend test * honor metadatamode value in newbackend pluginconfig * check that connection exists on cleanup * add automtls to secrets/auth plugins * don't remove apiclientmeta parsing * use formatting directive for fmt.Errorf * fix ut: remove tls provider func * remove tlsproviderfunc from backend plugin tests * use env var to prevent test plugin from running as a unit test * WIP: remove lazy loading * move non lazy loaded backend to new package * use version wrapper for backend plugin factory * remove backendVersionWrapper type * implement getBackendPluginType for plugin catalog * handle backend plugin v4 registration * add plugin automtls env guard * modify plugin factory to determine the backend to use * remove old pluginsets from v5 and log pid in plugin catalog * add reload mechanism via context * readd v3 and v4 to pluginset * call cleanup from reload if non-muxed * move v5 backend code to new package * use context reload for for ErrPluginShutdown case * add wrapper on v5 backend * fix run config UTs * fix unit tests - use v4/v5 mapping for plugin versions - fix test build err - add reload method on fakePluginClient - add multiplexed cases for integration tests * remove comment and update AutoMTLS field in test * remove comment * remove errwrap and unused context * only support metadatamode false for v5 backend plugins * update plugin catalog errors * use const for env variables * rename locks and remove unused * remove unneeded nil check * improvements based on staticcheck recommendations * use const for single implementation string * use const for context key * use info default log level * move pid to pluginClient struct * remove v3 and v4 from multiplexed plugin set * return from reload when non-multiplexed * update automtls env string * combine getBackend and getBrokeredClient * update comments for plugin reload, Backend return val and log * revert Backend return type * allow non-muxed plugins to serve v5 * move v5 code to existing sdk plugin package * do next export sdk fields now that we have removed extra plugin pkg * set TLSProvider in ServeMultiplex for backwards compat * use bool to flag multiplexing support on grpc backend server * revert userpass main.go * refactor plugin sdk - update comments - make use of multiplexing boolean and single implementation ID const * update comment and use multierr * attempt v4 if dispense fails on getPluginTypeForUnknown * update comments on sdk plugin backend
2022-08-30 02:42:26 +00:00
var HandshakeConfig = plugin.HandshakeConfig{
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
MagicCookieKey: "VAULT_BACKEND_PLUGIN",
MagicCookieValue: "6669da05-b1c8-4f49-97d9-c8e5bed98e20",
}