Check if plugin version matches running version
When registering a plugin, we check if the request version matches the
self-reported version from the plugin. If these do not match, we log a
warning.
This uncovered a few missing pieces for getting the database version
code fully working.
We added an environment variable that helps us unit test the running
version behavior as well, but only for approle, postgresql, and consul
plugins.
Return 400 on plugin not found or version mismatch
Populate the running SHA256 of plugins in the mount and auth tables (#17217)
Add plugin version to GRPC interface
Added a version interface in the sdk/logical so that it can be shared between all plugin types, and then wired it up to RunningVersion in the mounts, auth list, and database systems.
I've tested that this works with auth, database, and secrets plugin types, with the following logic to populate RunningVersion:
If a plugin has a PluginVersion() method implemented, then that is used
If not, and the plugin is built into the Vault binary, then the go.mod version is used
Otherwise, the it will be the empty string.
My apologies for the length of this PR.
* Placeholder backend should be external
We use a placeholder backend (previously a framework.Backend) before a
GRPC plugin is lazy-loaded. This makes us later think the plugin is a
builtin plugin.
So we added a `placeholderBackend` type that overrides the
`IsExternal()` method so that later we know that the plugin is external,
and don't give it a default builtin version.
* Support version selection for database plugins
* Don't consider unversioned plugins for version selection algorithm
* Added version to 'plugin not found' error
* Add PluginFactoryVersion function to avoid changing sdk/ API
* enable registering backend muxed plugins in plugin catalog
* set the sysview on the pluginconfig to allow enabling secrets/auth plugins
* store backend instances in map
* store single implementations in the instances map
cleanup instance map and ensure we don't deadlock
* fix system backend unit tests
move GetMultiplexIDFromContext to pluginutil package
fix pluginutil test
fix dbplugin ut
* return error(s) if we can't get the plugin client
update comments
* refactor/move GetMultiplexIDFromContext test
* add changelog
* remove unnecessary field on pluginClient
* add unit tests to PluginCatalog for secrets/auth plugins
* fix comment
* return pluginClient from TestRunTestPlugin
* add multiplexed backend test
* honor metadatamode value in newbackend pluginconfig
* check that connection exists on cleanup
* add automtls to secrets/auth plugins
* don't remove apiclientmeta parsing
* use formatting directive for fmt.Errorf
* fix ut: remove tls provider func
* remove tlsproviderfunc from backend plugin tests
* use env var to prevent test plugin from running as a unit test
* WIP: remove lazy loading
* move non lazy loaded backend to new package
* use version wrapper for backend plugin factory
* remove backendVersionWrapper type
* implement getBackendPluginType for plugin catalog
* handle backend plugin v4 registration
* add plugin automtls env guard
* modify plugin factory to determine the backend to use
* remove old pluginsets from v5 and log pid in plugin catalog
* add reload mechanism via context
* readd v3 and v4 to pluginset
* call cleanup from reload if non-muxed
* move v5 backend code to new package
* use context reload for for ErrPluginShutdown case
* add wrapper on v5 backend
* fix run config UTs
* fix unit tests
- use v4/v5 mapping for plugin versions
- fix test build err
- add reload method on fakePluginClient
- add multiplexed cases for integration tests
* remove comment and update AutoMTLS field in test
* remove comment
* remove errwrap and unused context
* only support metadatamode false for v5 backend plugins
* update plugin catalog errors
* use const for env variables
* rename locks and remove unused
* remove unneeded nil check
* improvements based on staticcheck recommendations
* use const for single implementation string
* use const for context key
* use info default log level
* move pid to pluginClient struct
* remove v3 and v4 from multiplexed plugin set
* return from reload when non-multiplexed
* update automtls env string
* combine getBackend and getBrokeredClient
* update comments for plugin reload, Backend return val and log
* revert Backend return type
* allow non-muxed plugins to serve v5
* move v5 code to existing sdk plugin package
* do next export sdk fields now that we have removed extra plugin pkg
* set TLSProvider in ServeMultiplex for backwards compat
* use bool to flag multiplexing support on grpc backend server
* revert userpass main.go
* refactor plugin sdk
- update comments
- make use of multiplexing boolean and single implementation ID const
* update comment and use multierr
* attempt v4 if dispense fails on getPluginTypeForUnknown
* update comments on sdk plugin backend
Adds support for using semantic version information when registering
and managing plugins. New `detailed` field in the response data for listing
plugins and new `version` field in the response data for reading a
single plugin.
* fix dev-plugin-dir when backend is builtin
* use builtinRegistry.Contains
* revert aa76337
* use correct plugin type for logical backend after revert
* fix factory func default setting after revert
* add ut coverage for builtin plugin with plugin directory set
* add coverage for secrets plugin type
* use totp in tests to avoid test import cycle in ssh package
* use nomad in tests to avoid test import cycle
* remove secrets mount tests due to unavoidable test import cycle
* Update plugin-portal.mdx (#13229)
Add a Vault plugin to allow authentication via SSH certificates and public keys
* oss changes
Co-authored-by: Wim <wim@42.be>
* adds development workflow to mirage config
* adds mirage handler and factory for mfa workflow
* adds mfa handling to auth service and cluster adapter
* moves auth success logic from form to controller
* adds mfa form component
* shows delayed auth message for all methods
* adds new code delay to mfa form
* adds error views
* fixes merge conflict
* adds integration tests for mfa-form component
* fixes auth tests
* updates mfa response handling to align with backend
* updates mfa-form to handle multiple methods and constraints
* adds noDefault arg to Select component
* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios
* adds tests
* flaky test fix attempt
* reverts test fix attempt
* adds changelog entry
* updates comments for todo items
* removes faker from mfa mirage factory and handler
* adds number to word helper
* fixes tests
* Revert "Merge branch 'main' into ui/mfa"
This reverts commit 8ee6a6aaa1b6c9ec16b985c10d91c3806819ec40, reversing
changes made to 2428dd6cca07bb41cda3f453619646ca3a88bfd0.
* format-ttl helper fix from main
* oss part of vault 2399
* Update vault/quotas/quotas.go
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
* use OSS PR number as changelog entry as indicated by the changelog guide
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
Checking if a plugin is a builtin by comparing it's type to those in the builtin
registry allows for a custom plugin loaded with the same name to be considered a
builtin during error handling of the mounting process. This can cause the vault
installation to brick itself because it cannot be unsealed to register a new
sha256 or file path for a previously loaded custom plugin. Improve this logic
by checking the plugin catalog rather than the builtin registry.
Fixes#11687
* first commit
* update
* removed some ent features from backport
* final refactor
* backport patch
Co-authored-by: Hridoy Roy <hridoyroy@Hridoys-MacBook-Pro.local>
Co-authored-by: Hridoy Roy <hridoyroy@Hridoys-MBP.hitronhub.home>
* upgrade aws roles
* test upgrade aws roles
* Initialize aws credential backend at mount time
* add a TODO
* create end-to-end test for builtin/credential/aws
* fix bug in initializer
* improve comments
* add Initialize() to logical.Backend
* use Initialize() in Core.enableCredentialInternal()
* use InitializeRequest to call Initialize()
* improve unit testing for framework.Backend
* call logical.Backend.Initialize() from all of the places that it needs to be called.
* implement backend.proto changes for logical.Backend.Initialize()
* persist current role storage version when upgrading aws roles
* format comments correctly
* improve comments
* use postUnseal funcs to initialize backends
* simplify test suite
* improve test suite
* simplify logic in aws role upgrade
* simplify aws credential initialization logic
* simplify logic in aws role upgrade
* use the core's activeContext for initialization
* refactor builtin/plugin/Backend
* use a goroutine to upgrade the aws roles
* misc improvements and cleanup
* do not run AWS role upgrade on DR Secondary
* always call logical.Backend.Initialize() when loading a plugin.
* improve comments
* on standbys and DR secondaries we do not want to run any kind of upgrade logic
* fix awsVersion struct
* clarify aws version upgrade
* make the upgrade logic for aws auth more explicit
* aws upgrade is now called from a switch
* fix fallthrough bug
* simplify logic
* simplify logic
* rename things
* introduce currentAwsVersion const to track aws version
* improve comments
* rearrange things once more
* conglomerate things into one function
* stub out aws auth initialize e2e test
* improve aws auth initialize e2e test
* finish aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* fix typo in test suite
* simplify logic a tad
* rearrange assignment
* Fix a few lifecycle related issues in #7025 (#7075)
* Fix panic when plugin fails to load
When unmounting, the router entry would be tainted, preventing routing.
However, we would then unmount the router before clearing storage, so if
an error occurred the router would have forgotten the path. For auth
mounts this isn't a problem since they had a secondary check, but
regular mounts didn't (not sure why, but this is true back to at least
0.2.0). This meant you could then create a duplicate mount using the
same path which would then not conflict in the router until postUnseal.
This adds the extra check to regular mounts, and also moves the location
of the router unmount.
This also ensures that on the next router.Mount, tainted is set to the
mount entry's tainted status.
Fixes#6769
This lets other parts of Vault that can't depend on the vault package
take advantage of the subview functionality.
This also allows getting rid of BarrierStorage and vault.Entry, two
totally redundant abstractions.
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
* plumbing request context to expiration manager
* moar context
* address feedback
* only using active context for revoke prefix
* using active context for revoke commands
* cancel tidy on active context
* address feedback
This change makes it so that if a lease is revoked through user action,
we set the expiration time to now and update pending, just as we do with
tokens. This allows the normal retry logic to apply in these cases as
well, instead of just erroring out immediately. The idea being that once
you tell Vault to revoke something it should keep doing its darndest to
actually make that happen.
A few notes:
* We exert positive control over singletons and they usually need to
perform some (known, validated) writes, so this excludes singletons --
they are simply limited to the end of the mount function as before.
* I'm not sure how to test this _specifically_; I've done some testing
of e.g. sealing vault and unsealing and ensuring that I can write to a
KV mount. I think this is tested by every dev server though, since for a
dev server Vault is inited, the default mounts are mounted, then it's
sealed, then it's unsealed for the user, so it already goes through this
code path. The mere fact that you can write to secret/ on a dev server
means it was successfully set read-write.
* logbridge with hclog and identical output
* Initial search & replace
This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.
* strip logbridge
* fix majority of tests
* update logxi aliases
* WIP fixing tests
* more test fixes
* Update test to hclog
* Fix format
* Rename hclog -> log
* WIP making hclog and logxi love each other
* update logger_test.go
* clean up merged comments
* Replace RawLogger interface with a Logger
* Add some logger names
* Replace Trace with Debug
* update builtin logical logging patterns
* Fix build errors
* More log updates
* update log approach in command and builtin
* More log updates
* update helper, http, and logical directories
* Update loggers
* Log updates
* Update logging
* Update logging
* Update logging
* Update logging
* update logging in physical
* prefixing and lowercase
* Update logging
* Move phyisical logging name to server command
* Fix som tests
* address jims feedback so far
* incorporate brians feedback so far
* strip comments
* move vault.go to logging package
* update Debug to Trace
* Update go-plugin deps
* Update logging based on review comments
* Updates from review
* Unvendor logxi
* Remove null_logger.go
