Add plugin version to GRPC interface
Added a version interface in the sdk/logical so that it can be shared between all plugin types, and then wired it up to RunningVersion in the mounts, auth list, and database systems.
I've tested that this works with auth, database, and secrets plugin types, with the following logic to populate RunningVersion:
If a plugin has a PluginVersion() method implemented, then that is used
If not, and the plugin is built into the Vault binary, then the go.mod version is used
Otherwise, the it will be the empty string.
My apologies for the length of this PR.
* Placeholder backend should be external
We use a placeholder backend (previously a framework.Backend) before a
GRPC plugin is lazy-loaded. This makes us later think the plugin is a
builtin plugin.
So we added a `placeholderBackend` type that overrides the
`IsExternal()` method so that later we know that the plugin is external,
and don't give it a default builtin version.
* Support version selection for database plugins
* Don't consider unversioned plugins for version selection algorithm
* Added version to 'plugin not found' error
* Add PluginFactoryVersion function to avoid changing sdk/ API
* core: Handle deprecated mounts on enable and unseal
* changelog: Deprecation Status handling
* core: Add Pending Removal override var
* core: Add some documentation for Pending Removal override
- When we added new tests that validate the RSA PSS feature, they
work properly on normal Go builds, but tests underneath the Boring
Crypto fips implementations fail due to a lack of SHA3 support in
FIPS 140-2.
* Get import correct
* limits, docs
* changelog
* unit tests
* And fix import for hmac unit test
* typo
* Update website/content/api-docs/secret/transit.mdx
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Update builtin/logical/transit/path_keys.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Validate key sizes a bit more carefully
* Update sdk/helper/keysutil/policy.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* accommodate salt lengths for RSA PSS
* address feedback
* generalise salt length to an int
* fix error reporting
* Revert "fix error reporting"
This reverts commit 8adfc15fe3303b8fdf9f094ea246945ab1364077.
* fix a faulty check
* check for min/max salt lengths
* stringly-typed HTTP param
* unit tests for sign/verify HTTP requests
also, add marshaling for both SDK and HTTP requests
* randomly sample valid salt length
* add changelog
* add documentation
* enable registering backend muxed plugins in plugin catalog
* set the sysview on the pluginconfig to allow enabling secrets/auth plugins
* store backend instances in map
* store single implementations in the instances map
cleanup instance map and ensure we don't deadlock
* fix system backend unit tests
move GetMultiplexIDFromContext to pluginutil package
fix pluginutil test
fix dbplugin ut
* return error(s) if we can't get the plugin client
update comments
* refactor/move GetMultiplexIDFromContext test
* add changelog
* remove unnecessary field on pluginClient
* add unit tests to PluginCatalog for secrets/auth plugins
* fix comment
* return pluginClient from TestRunTestPlugin
* add multiplexed backend test
* honor metadatamode value in newbackend pluginconfig
* check that connection exists on cleanup
* add automtls to secrets/auth plugins
* don't remove apiclientmeta parsing
* use formatting directive for fmt.Errorf
* fix ut: remove tls provider func
* remove tlsproviderfunc from backend plugin tests
* use env var to prevent test plugin from running as a unit test
* WIP: remove lazy loading
* move non lazy loaded backend to new package
* use version wrapper for backend plugin factory
* remove backendVersionWrapper type
* implement getBackendPluginType for plugin catalog
* handle backend plugin v4 registration
* add plugin automtls env guard
* modify plugin factory to determine the backend to use
* remove old pluginsets from v5 and log pid in plugin catalog
* add reload mechanism via context
* readd v3 and v4 to pluginset
* call cleanup from reload if non-muxed
* move v5 backend code to new package
* use context reload for for ErrPluginShutdown case
* add wrapper on v5 backend
* fix run config UTs
* fix unit tests
- use v4/v5 mapping for plugin versions
- fix test build err
- add reload method on fakePluginClient
- add multiplexed cases for integration tests
* remove comment and update AutoMTLS field in test
* remove comment
* remove errwrap and unused context
* only support metadatamode false for v5 backend plugins
* update plugin catalog errors
* use const for env variables
* rename locks and remove unused
* remove unneeded nil check
* improvements based on staticcheck recommendations
* use const for single implementation string
* use const for context key
* use info default log level
* move pid to pluginClient struct
* remove v3 and v4 from multiplexed plugin set
* return from reload when non-multiplexed
* update automtls env string
* combine getBackend and getBrokeredClient
* update comments for plugin reload, Backend return val and log
* revert Backend return type
* allow non-muxed plugins to serve v5
* move v5 code to existing sdk plugin package
* do next export sdk fields now that we have removed extra plugin pkg
* set TLSProvider in ServeMultiplex for backwards compat
* use bool to flag multiplexing support on grpc backend server
* revert userpass main.go
* refactor plugin sdk
- update comments
- make use of multiplexing boolean and single implementation ID const
* update comment and use multierr
* attempt v4 if dispense fails on getPluginTypeForUnknown
* update comments on sdk plugin backend
* Allow generation of up-to-date delta CRLs
While switching to periodic rebuilds of CRLs alleviates the constant
rebuild pressure on Vault during times of high revocation, the CRL
proper becomes stale. One response to this is to switch to OCSP, but not
every system has support for this. Additionally, OCSP usually requires
connectivity and isn't used to augment a pre-distributed CRL (and is
instead used independently).
By generating delta CRLs containing only new revocations, an existing
CRL can be supplemented with newer revocations without requiring Vault
to rebuild all complete CRLs. Admins can periodically fetch the delta
CRL and add it to the existing CRL and applications should be able to
support using serials from both.
Because delta CRLs are emptied when the next complete CRL is rebuilt, it
is important that applications fetch the delta CRL and correlate it to
their complete CRL; if their complete CRL is older than the delta CRL's
extension number, applications MUST fetch the newer complete CRL to
ensure they have a correct combination.
This modifies the revocation process and adds several new configuration
options, controlling whether Delta CRLs are enabled and when we'll
rebuild it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for delta CRLs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation on delta CRLs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback: fix several bugs
Thanks Steve!
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly invoke periodic func on active nodes
We need to ensure we read the updated config (in case of OCSP request
handling on standby nodes), but otherwise want to avoid CRL/DeltaCRL
re-building.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Adds support for using semantic version information when registering
and managing plugins. New `detailed` field in the response data for listing
plugins and new `version` field in the response data for reading a
single plugin.
* Allow Proof of Possession based revocation
Revocation by proof of possession ensures that we have a private key
matching the (provided or stored) certificate. This allows callers to
revoke certificate they own (as proven by holding the corresponding
private key), without having an admin create innumerable ACLs around
the serial_number parameter for every issuance/user.
We base this on Go TLS stack's verification of certificate<->key
matching, but extend it where applicable to ensure curves match, the
private key is indeed valid, and has the same structure as the
corresponding public key from the certificate.
This endpoint currently is authenticated, allowing operators to disable
the endpoint if it isn't desirable to use, via ACL policies.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify error message on ParseDERKey
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Leave revoke-with-key authenticated
After some discussion, given the potential for DoS (via submitting a lot
of keys/certs to validate, including invalid pairs), it seems best to
leave this as an authenticated endpoint. Presently in Vault, there's no
way to have an authenticated-but-unauthorized path (i.e., one which
bypasses ACL controls), so it is recommended (but not enforced) to make
this endpoint generally available by permissive ACL policies.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add API documentation on PoP
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add acceptance tests for Proof of Possession
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Exercise negative cases in PoP tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- Found by @trishankatdatadog in PR #16549, we were masking errors
coming out of the rsa verification calls as verfication errors and
not returning when they were usage errors.
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* Add PSS signature support to Vault PKI engine
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use issuer's RevocationSigAlg for CRL signing
We introduce a new parameter on issuers, revocation_signature_algorithm
to control the signature algorithm used during CRL signing. This is
because the SignatureAlgorithm value from the certificate itself is
incorrect for this purpose: a RSA root could sign an ECDSA intermediate
with say, SHA256WithRSA, but when the intermediate goes to sign a CRL,
it must use ECDSAWithSHA256 or equivalent instead of SHA256WithRSA. When
coupled with support for PSS-only keys, allowing the user to set the
signature algorithm value as desired seems like the best approach.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add use_pss, revocation_signature_algorithm docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add PSS to signature role issuance test matrix
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow roots to self-identify revocation alg
When using PSS support with a managed key, sometimes the underlying
device will not support PKCS#1v1.5 signatures. This results in CRL
building failing, unless we update the entry's signature algorithm
prior to building the CRL for the new root.
With a RSA-type key and use_pss=true, we use the signature bits value to
decide which hash function to use for PSS support.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add clearer error message on failed import
When CRL building fails during cert/key import, due to PSS failures,
give a better indication to the user that import succeeded its just CRL
building that failed. This tells them the parameter to adjust on the
issuer and warns that CRL building will fail until this is fixed.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add case insensitive SigAlgo matching
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Convert UsePSS back to regular bool
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor PSS->certTemplate into helper function
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Proper string output on rev_sig_alg display
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Copy root's SignatureAlgorithm for CRL building
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow old certs to be cross-signed
In Vault 1.11, we introduced cross-signing support, but the earlier SKID
field change in Vault 1.10 causes problems: notably, certs created on
older versions of Vault (<=1.9) or outside of Vault (with a different
SKID method) cannot be cross-signed and validated in OpenSSL.
In particular, OpenSSL appears to be unique in requiring a SKID/AKID
match for chain building. If AKID and SKID are present on an otherwise
valid client/parent cert pair and the values are different, OpenSSL will
not build a valid path over those two, whereas most other chain
validation implementations will.
Regardless, to have proper cross-signing support, we really aught to
support copying an SKID. This adds such support to the sign-intermediate
endpoint. Support for the /issue endpoint is not added, as cross-signing
leaf certs isn't generally useful and can accept random SKIDs.
Resolves: #16461
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback, fix tests
Also adds a known-answer test using LE R3 CA's SKID.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback regarding separators
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* use automtls for v5 secrets/auth plugins
* add automtls env guard
* start backend without metadata mode
* use PluginClientConfig for backend's NewPluginClient param
refactor
* - fix pluginutil test
- do not expect plugin to be unloaded in UT
- fix pluginutil tests --need new env var
- use require in UT
- fix lazy load test
* add changelog
* prioritize automtls; improve comments
* user multierror; refactor pluginSet for v4 unit test
* add test cases for v4 and v5 plugin versions
* remove unnecessary call to AutoMTLSSupported
* update comment on pluginSets
* use runconfig directly in sdk newpluginclient
* use automtls without metadatamode for v5 backend plugin registration
* use multierror for plugin runconfig calls
* remove some unnecessary code
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core
* Fix body handling
* Role resolution for rate limit quotas
* VAULT-6613 update precedence test
* Add changelog
* VAULT-6614 start of changes for roles in LCQs
* Expiration changes for leases
* Add role information to RequestAuth
* VAULT-6614 Test updates
* VAULT-6614 Add expiration test with roles
* VAULT-6614 fix comment
* VAULT-6614 Protobuf on OSS
* VAULT-6614 Add rlock to determine role code
* VAULT-6614 Try lock instead of rlock
* VAULT-6614 back to rlock while I think about this more
* VAULT-6614 Additional safety for nil dereference
* VAULT-6614 Use %q over %s
* VAULT-6614 Add overloading to plugin backends
* VAULT-6614 RLocks instead
* VAULT-6614 Fix return for backend factory
* add func to set level for specific logger
* add endpoints to modify log level
* initialize base logger with IndependentLevels
* test to ensure other loggers remain unchanged
* add DELETE loggers endpoints to revert back to config
* add API docs page
* add changelog entry
* remove extraneous line
* add log level field to Core struct
* add godoc for getLogLevel
* add some loggers to c.allLoggers
* Add parsing for NSS-wrapped Ed25519 keys
NSS wraps Ed25519 using the PKCS#8 standard structure. The Go standard
library as of Go 1.18.x doesn't support parsing this key type with the
OID used by NSS; it requires the 1.3.101.112/RFC 8410 format, rather
than the RFC 5915-esque structure supported here.
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add integration test with NSS-created wrapped key
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Rachel Culpepper <84159930+rculpepper@users.noreply.github.com>
* Support for CPS URLs in Custom Policy Identifiers.
* go fmt
* Add Changelog
* Fix panic in test-cases.
* Update builtin/logical/pki/path_roles.go
Fix intial nil identifiers.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Make valid policy OID so don't break ASN parse in test.
* Add test cases.
* go fmt.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Note that this only really applies to the SSH engine, nothing else uses
this helper. Other go.mod's updated for consistency.
See also: https://github.com/hashicorp/go-secure-stdlib/pull/40
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* WIP replacing lib/pq
* change timezome param to be URI format
* add changelog
* add changelog for redshift
* update changelog
* add test for DSN style connection string
* more parseurl and quoteidentify to sdk; include copyright and license
* call dbutil.ParseURL instead, fix import ordering
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Fix handling of username_as_alias during LDAP authentication
There is a bug that was introduced in the LDAP authentication method by https://github.com/hashicorp/vault/pull/11000.
It was thought to be backward compatible but has broken a number of users. Later
a new parameter `username_as_alias` was introduced in https://github.com/hashicorp/vault/pull/14324
to make it possible for operators to restore the previous behavior.
The way it is currently working is not completely backward compatible thought
because when username_as_alias is set, a call to GetUserAliasAttributeValue() will
first be made, then this value is completely discarded in pathLogin() and replaced
by the username as expected.
This is an issue because it makes useless calls to the LDAP server and will break
backward compatibility if one of the constraints in GetUserAliasAttributeValue()
is not respected, even though the resulting value will be discarded anyway.
In order to maintain backward compatibility here we have to only call
GetUserAliasAttributeValue() if necessary.
Since this change of behavior was introduced in 1.9, this fix will need to be
backported to the 1.9, 1.10 and 1.11 branches.
* Add changelog
* Add tests
* Format code
* Update builtin/credential/ldap/backend.go
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Format and fix declaration
* Reword changelog
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
Steven Clark