open-vault/sdk/logical/logical.go

179 lines
6.9 KiB
Go
Raw Normal View History

package logical
import (
"context"
log "github.com/hashicorp/go-hclog"
)
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
// BackendType is the type of backend that is being implemented
type BackendType uint32
// The these are the types of backends that can be derived from
// logical.Backend
const (
TypeUnknown BackendType = 0 // This is also the zero-value for BackendType
TypeLogical BackendType = 1
TypeCredential BackendType = 2
)
// Stringer implementation
func (b BackendType) String() string {
switch b {
case TypeLogical:
return "secret"
case TypeCredential:
return "auth"
}
return "unknown"
}
// Backend interface must be implemented to be "mountable" at
// a given path. Requests flow through a router which has various mount
// points that flow to a logical backend. The logic of each backend is flexible,
// and this is what allows materialized keys to function. There can be specialized
// logical backends for various upstreams (Consul, PostgreSQL, MySQL, etc) that can
// interact with remote APIs to generate keys dynamically. This interface also
// allows for a "procfs" like interaction, as internal state can be exposed by
// acting like a logical backend and being mounted.
type Backend interface {
AWS upgrade role entries (#7025) * upgrade aws roles * test upgrade aws roles * Initialize aws credential backend at mount time * add a TODO * create end-to-end test for builtin/credential/aws * fix bug in initializer * improve comments * add Initialize() to logical.Backend * use Initialize() in Core.enableCredentialInternal() * use InitializeRequest to call Initialize() * improve unit testing for framework.Backend * call logical.Backend.Initialize() from all of the places that it needs to be called. * implement backend.proto changes for logical.Backend.Initialize() * persist current role storage version when upgrading aws roles * format comments correctly * improve comments * use postUnseal funcs to initialize backends * simplify test suite * improve test suite * simplify logic in aws role upgrade * simplify aws credential initialization logic * simplify logic in aws role upgrade * use the core's activeContext for initialization * refactor builtin/plugin/Backend * use a goroutine to upgrade the aws roles * misc improvements and cleanup * do not run AWS role upgrade on DR Secondary * always call logical.Backend.Initialize() when loading a plugin. * improve comments * on standbys and DR secondaries we do not want to run any kind of upgrade logic * fix awsVersion struct * clarify aws version upgrade * make the upgrade logic for aws auth more explicit * aws upgrade is now called from a switch * fix fallthrough bug * simplify logic * simplify logic * rename things * introduce currentAwsVersion const to track aws version * improve comments * rearrange things once more * conglomerate things into one function * stub out aws auth initialize e2e test * improve aws auth initialize e2e test * finish aws auth initialize e2e test * tinker with aws auth initialize e2e test * tinker with aws auth initialize e2e test * tinker with aws auth initialize e2e test * fix typo in test suite * simplify logic a tad * rearrange assignment * Fix a few lifecycle related issues in #7025 (#7075) * Fix panic when plugin fails to load
2019-07-05 23:55:40 +00:00
// Initialize is used to initialize a plugin after it has been mounted.
Initialize(context.Context, *InitializationRequest) error
// HandleRequest is used to handle a request and generate a response.
// The backends must check the operation type and handle appropriately.
HandleRequest(context.Context, *Request) (*Response, error)
2015-03-31 00:46:18 +00:00
// SpecialPaths is a list of paths that are special in some way.
// See PathType for the types of special paths. The key is the type
// of the special path, and the value is a list of paths for this type.
// This is not a regular expression but is an exact match. If the path
// ends in '*' then it is a prefix-based match. The '*' can only appear
// at the end.
SpecialPaths() *Paths
// System provides an interface to access certain system configuration
// information, such as globally configured default and max lease TTLs.
System() SystemView
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
// Logger provides an interface to access the underlying logger. This
// is useful when a struct embeds a Backend-implemented struct that
// contains a private instance of logger.
Logger() log.Logger
// HandleExistenceCheck is used to handle a request and generate a response
// indicating whether the given path exists or not; this is used to
// understand whether the request must have a Create or Update capability
2016-01-12 20:09:16 +00:00
// ACL applied. The first bool indicates whether an existence check
// function was found for the backend; the second indicates whether, if an
// existence check function was found, the item exists or not.
HandleExistenceCheck(context.Context, *Request) (bool, bool, error)
// Cleanup is invoked during an unmount of a backend to allow it to
// handle any cleanup like connection closing or releasing of file handles.
Cleanup(context.Context)
// InvalidateKey may be invoked when an object is modified that belongs
// to the backend. The backend can use this to clear any caches or reset
// internal state as needed.
InvalidateKey(context.Context, string)
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
// Setup is used to set up the backend based on the provided backend
// configuration.
Setup(context.Context, *BackendConfig) error
Backend plugin system (#2874) * Add backend plugin changes * Fix totp backend plugin tests * Fix logical/plugin InvalidateKey test * Fix plugin catalog CRUD test, fix NoopBackend * Clean up commented code block * Fix system backend mount test * Set plugin_name to omitempty, fix handleMountTable config parsing * Clean up comments, keep shim connections alive until cleanup * Include pluginClient, disallow LookupPlugin call from within a plugin * Add wrapper around backendPluginClient for proper cleanup * Add logger shim tests * Add logger, storage, and system shim tests * Use pointer receivers for system view shim * Use plugin name if no path is provided on mount * Enable plugins for auth backends * Add backend type attribute, move builtin/plugin/package * Fix merge conflict * Fix missing plugin name in mount config * Add integration tests on enabling auth backend plugins * Remove dependency cycle on mock-plugin * Add passthrough backend plugin, use logical.BackendType to determine lease generation * Remove vault package dependency on passthrough package * Add basic impl test for passthrough plugin * Incorporate feedback; set b.backend after shims creation on backendPluginServer * Fix totp plugin test * Add plugin backends docs * Fix tests * Fix builtin/plugin tests * Remove flatten from PluginRunner fields * Move mock plugin to logical/plugin, remove totp and passthrough plugins * Move pluginMap into newPluginClient * Do not create storage RPC connection on HandleRequest and HandleExistenceCheck * Change shim logger's Fatal to no-op * Change BackendType to uint32, match UX backend types * Change framework.Backend Setup signature * Add Setup func to logical.Backend interface * Move OptionallyEnableMlock call into plugin.Serve, update docs and comments * Remove commented var in plugin package * RegisterLicense on logical.Backend interface (#3017) * Add RegisterLicense to logical.Backend interface * Update RegisterLicense to use callback func on framework.Backend * Refactor framework.Backend.RegisterLicense * plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs * plugin: Revert BackendType to remove TypePassthrough and related references * Fix typo in plugin backends docs
2017-07-20 17:28:40 +00:00
// Type returns the BackendType for the particular backend
Type() BackendType
}
// BackendConfig is provided to the factory to initialize the backend
type BackendConfig struct {
// View should not be stored, and should only be used for initialization
StorageView Storage
// The backend should use this logger. The log should not contain any secrets.
2016-08-19 20:45:17 +00:00
Logger log.Logger
// System provides a view into a subset of safe system information that
// is useful for backends, such as the default/max lease TTLs
System SystemView
// BackendUUID is a unique identifier provided to this backend. It's useful
// when a backend needs a consistent and unique string without using storage.
BackendUUID string
// Config is the opaque user configuration provided when mounting
Config map[string]string
}
// Factory is the factory function to create a logical backend.
type Factory func(context.Context, *BackendConfig) (Backend, error)
2015-03-31 00:46:18 +00:00
// Paths is the structure of special paths that is used for SpecialPaths.
type Paths struct {
// Root are the API paths that require a root token to access
2015-03-31 00:46:18 +00:00
Root []string
// Unauthenticated are the API paths that can be accessed without any auth.
2021-10-13 16:51:20 +00:00
// These can't be regular expressions, it is either exact match, a prefix
// match and/or a wildcard match. For prefix match, append '*' as a suffix.
// For a wildcard match, use '+' in the segment to match any identifier
// (e.g. 'foo/+/bar'). Note that '+' can't be adjacent to a non-slash.
2015-03-31 00:46:18 +00:00
Unauthenticated []string
// LocalStorage are storage paths (prefixes) that are local to this cluster;
// this indicates that these paths should not be replicated across performance clusters
// (DR replication is unaffected).
LocalStorage []string
2017-10-23 20:42:56 +00:00
// SealWrapStorage are storage paths that, when using a capable seal,
// should be seal wrapped with extra encryption. It is exact matching
// unless it ends with '/' in which case it will be treated as a prefix.
SealWrapStorage []string
Add path based primary write forwarding (PBPWF) - OSS (#18735) * Add WriteForwardedStorage to sdk's plugin, logical in OSS This should allow backends to specify paths to forward write (storage.Put(...) and storage.Delete(...)) operations for. Notably, these semantics are subject to change and shouldn't yet be relied on. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Collect paths for write forwarding in OSS This adds a path manager to Core, allowing tracking across all Vault versions of paths which could use write forwarding if available. In particular, even on OSS offerings, we'll need to template {{clusterId}} into the paths, in the event of later upgrading to Enterprise. If we didn't, we'd end up writing paths which will no longer be accessible post-migration, due to write forwarding now replacing the sentinel with the actual cluster identifier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add forwarded writer implementation to OSS Here, for paths given to us, we determine if we need to do cluster translation and perform local writing. This is the OSS variant. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Wire up mount-specific request forwarding in OSS Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify that state lock needs to be held to call HAState in OSS Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Move cluster sentinel constant to sdk/logical Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expose ClusterID to Plugins via SystemView This will let plugins learn what the Cluster's ID is, without having to resort to hacks like writing a random string to its cluster-prefixed namespace and then reading it once it has replicated. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add GRPC ClusterID implementation For any external plugins which wish to use it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-20 21:36:18 +00:00
// WriteForwardedStorage are storage paths that, when running on a PR
// Secondary cluster, cause a GRPC call up to the PR Primary cluster's
// active node to handle storage.Put(...) and storage.Delete(...) events.
// These paths MUST include a {{clusterId}} literal, which the write layer
// will resolve to this cluster's UUID ("replication set" identifier).
// storage.List(...) and storage.Get(...) operations occur from the
// locally replicated data set, but can use path template expansion to be
// identifier agnostic.
//
// These paths require careful considerations by developers to use. In
// particular, writes on secondary clusters will not appear (when a
// corresponding read is issued immediately after a write) until the
// replication from primary->secondary has occurred. This replication
// triggers an InvalidateKey(...) call on the secondary, which can be
// used to detect the write has finished syncing. However, this will
// likely occur after the request has finished, so it is important to
// not block on this occurring.
//
// On standby nodes, like all storage write operations, this will trigger
// an ErrReadOnly return.
WriteForwardedStorage []string
}
type Auditor interface {
AuditRequest(ctx context.Context, input *LogInput) error
AuditResponse(ctx context.Context, input *LogInput) error
}
// Externaler allows us to check if a backend is running externally (i.e., over GRPC)
type Externaler interface {
IsExternal() bool
}
type PluginVersion struct {
Version string
}
// PluginVersioner is an optional interface to return version info.
type PluginVersioner interface {
// PluginVersion returns the version for the backend
PluginVersion() PluginVersion
}
var EmptyPluginVersion = PluginVersion{""}