* Add support for PKCSv1_5_NoOID signatures
This assumes a pre-hashed input has been provided to Vault, but we do
not write the hash's OID into the signature stream. This allows us to
generate the alternative PKCSv1_5_NoOID signature type rather than the
existing PKCSv1_5_DERnull signature type we presently use.
These are specified in RFC 3447 Section 9.2.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Exclude new none type from PSS based tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for PKCS#1v1.5 signatures
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow passing AssociatedData factories in keysutil
This allows the high-level, algorithm-agnostic Encrypt/Decrypt with
Factory to pass in AssociatedData, and potentially take multiple
factories (to allow KMS keys to work). On AEAD ciphers with a relevant
factory, an AssociatedData factory will be used to populate the
AdditionalData field of the SymmetricOpts struct, using it in the AEAD
Seal process.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add associated_data to Transit Encrypt/Decrypt API
This allows passing the associated_data (the last AD in AEAD) to
Transit's encrypt/decrypt when using an AEAD cipher (currently
aes128-gcm96, aes256-gcm96, and chacha20-poly1305). We err if this
parameter is passed on non-AEAD ciphers presently.
This associated data can be safely transited in plaintext, without risk
of modifications. In the event of tampering with either the ciphertext
or the associated data, decryption will fail.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add to documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
The SSH secrets engine previously split the `validPrincipals` field
on comma, then if user templating is enabled, evaluated the
templates on each substring. This meant the identity template was only
ever allowed to return a single principal. There are use cases
where it would be helpful for identity metadata to contain a list
of valid principals and for the identity template to be able to inject
all of those as valid principals.
This change inverts the order of processing. First the template
is evaluated, and then the resulting string is split on commas.
This allows the identity template to return a single comma-separated
string with multiple permitted principals.
There is a potential security implication here, that if a user is
allowed to update their own identity metadata, they may be able to
elevate privileges where previously this was not possible.
Fixes#11038
* Add tests for zlint-clean CA building
This test ensures that we can consistently pass ZLint's CA linting
tests on a root certificate generated by Vault. In particular, nominal
requirements are placed on the structure on the issuer's Subject, which
we supply, and the remaining requirements pass.
The one exception is we include both RFC and CA/BF BR lints in the
default zlint checks; this means ECDSA P-521 (which isn't accepted by
Mozilla's root store policies) is rejected, so we ignore to lints
related to that.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add options to copy to/from container, fix stopping
Stopping the container takes a bit of time for some unknown reason so
I've instead opted to shorten the sleep in the zlint tests to avoid
consuming resources too long after the test finish.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Make zlint tests execute in parallel
This improves the overall test time of the zlint tests, making the
container build up front once (provisioning zlint), and then copying the
cert into the new container image later.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* make fmt
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor Docker command execution
This refactor will allow others to interact with containers more easily,
providing two interfaces (RunCmdWithOutput and RunCmdInBackground) for
executing commands in running containers if they don't wish to do so
manually.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow building containerfiles in tests
By adding image building capabilities to testhelpers (and coupled with
the better command execution support), we can begin to build better,
more reliable integration tests on top of public base images without
needing to maintain separate forks of these images out-of-tree for any
shortcomings they might have.
In particular, rather than doing the rather messy echo hack for writing
clients.conf, it is far better to provision this via a slim
Containerfile overlay on top of the stock jumanjiman/radiusd:latest
image.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly parse stdout/stderr in RunCmdWithOutput
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* ctx -> bCtx for BuildContext
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update errors to use %w instead of %v
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix tidy-status, tidy-cancel on PR Secondaries
PKI's tidy-status included a bug that prevented PR secondary nodes from
responding with the status of the running tidy operation: while the
operation constructor correctly forwarded the node on PR standby
instances, the handler itself forwarded also on PR secondary nodes.
This is incorrect as the PR secondary nodes are the active node in the
local PR cluster, and run tidy operations otherwise.
This meant that while auto-tidy and tidy operations would run, there was
no insight into the process.
When implementing tidy-cancel, tidy-status's handler logic was reused,
duplicating the bug there as well.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
When revoking an issuer, we immediately force a full rebuild of all CRLs
(complete and delta). However, we had forgotten to guard the delta CRL's
inclusion of augmented issuers, resulting in double-listing the issuer's
serial number on both the complete and the delta CRL. This isn't
necessary as the delta's referenced complete CRL number has incremented
to the point where the issuer itself was included on the complete CRL.
Avoid this double reference and don't include issuers on delta CRLs;
they should always appear only on the complete CRL.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- Add some unit tests around the OCSP response validation that we
are using the proper signature algorithms.
- Add in test cases as well to validate SHA384 and SHA512 requested hash support
* Fix for duplicate SANs in signed certificates when othernames are present in the CSR SAN extension and UseCSRValues is true.
When UseCSRValues is true (as is the case on the sign-verbatim endpoint), all extensions including Subject Alternative Names are copied from the CSR to the final certificate.
If the Subject Alternative Name in question contains any othernames (such as a Microsoft UPN) the SAN extension is added again as a workaround for an encoding issue (in function HandleOtherSANs).
Having duplicate x509v3 extensions is invalid and is rejected by openssl on Ubuntu 20.04, and also by Go since https://github.com/golang/go/issues/50988 (including in Go 1.19).
In this fix I do not add the extension from the CSR if it will be added during HandleOtherSANs.
* Added unittest and changelog entry.
* Fix RevocationSigAlg provisioning in GCP
GCP restricts keys to a certain type of signature, including hash
algorithm, so we must provision our RevocationSigAlg from the root
itself unconditionally in order for GCP to work.
This does change the default, but only for newly created certificates.
Additionally, we clarify that CRL building is not fatal to the import
process.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add inverse mapping for SignatureAlgorithm
By default we'd use .String() on x509.SignatureAlgorithm, but this
doesn't round-trip. Switch to a custom map that is round-trippable
and matches the constant name as there is no other way to get this info
presently.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test to ensure root creation sets rev_sig_alg
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Test round-tripping of SigAlgoNames, InvSigAlgoNames
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix failing Default Update test
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
When requesting a SSH certificate with default_extension templating
enabled, if the request lacks entity information and a particular
extension requires templating, just these extensions will be elided.
Other extensions (if present) will still be on the final certificate.
Add a warning in the event of missing entity information and at least
one extension that was skipped as a result.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow OCSP to use issuer's RevocationSigAlgo
When an issuer specifies a RevocationSigAlgo, we should largely follow
this for both CRLs and OCSP. However, x/crypto/ocsp lacks support for
PSS signatures, so we drop these down to PKCS#1v1.5 instead.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning when issuer has PSS-based RevSigAlgo
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add note about OCSP and PSS support
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* PKI: Add support for signature_bits param to the intermediate/generate api
- Mainly to work properly with GCP backed managed keys, we need to
issue signatures that would match the GCP key algorithm.
- At this time due to https://github.com/golang/go/issues/45990 we
can't issue PSS signed CSRs, as the libraries in Go always request
a PKCS1v15.
- Add an extra check in intermediate/generate that validates the CSR's
signature before providing it back to the client in case we generated
a bad signature such as if an end-user used a GCP backed managed key
with a RSA PSS algorithm.
- GCP ignores the requested signature type and always signs with the
key's algorithm which can lead to a CSR that says it is signed with
a PKCS1v15 algorithm but is actually a RSA PSS signature
* Add cl
* PR feedback
* PKI: Do not load revoked certificates if CRL has been disabled
- Restore the prior behavior of not reading in all revoked certificates
if the CRL has been disabled as there might be performance issues
if a customer had or is still revoking a lot of certificates.
* Add cl
When adding delta CRL support, we unconditionally added the delta
indicator extension to the main CRL. We shouldn't have done this, and
instead only added it conditionally when we were building delta CRLs.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
The periodic function only runs every 50ms, so waiting 60ms means we
might not be done fetching the CRL on slower CI systems or with high
test parallelism.
Tested with:
> untilfail -parallel=-9 ../../../cert.test -test.run=TestCRLFetch -test.count=1 -test.v
And shown to reliably fail before, fixed after.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Check if plugin version matches running version
When registering a plugin, we check if the request version matches the
self-reported version from the plugin. If these do not match, we log a
warning.
This uncovered a few missing pieces for getting the database version
code fully working.
We added an environment variable that helps us unit test the running
version behavior as well, but only for approle, postgresql, and consul
plugins.
Return 400 on plugin not found or version mismatch
Populate the running SHA256 of plugins in the mount and auth tables (#17217)
* Fix interoperability concerns with PSS
When Go parses a certificate with rsaPSS OID, it will accept this
certificate but not parse the SubjectPublicKeyInfo, leaving the
PublicKeyAlgorithm and PublicKey fields blank, but otherwise not erring.
The same behavior occurs with rsaPSS OID CSRs.
On the other hand, when Go parses rsaPSS OID PKCS8 private keys, these
keys will fail to parse completely.
Thus, detect and fail on any empty PublicKey certs and CSRs, warning the
user that we cannot parse these correctly and thus refuse to operate.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Run more PKI tests in parallel
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add notes about PSS shortcomings to considerations
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Basics of Cert-Count Telemetry, changelog, "best attempt" slice to capture (and test for) duplicates, Move sorting of possibleDoubleCountedRevokedSerials to after compare of entries. Add values to counter when still initializing.
Set lists to nil after use, Fix atomic2 import, Delay reporting metrics until after deduplication has completed,
The test works now, Move string slice to helper function; Add backendUUID to gauge name.
* Don't race for CRL rebuilding capability check
Core has recently seen some data races during SystemView/replication
updates between them and the PKI subsystem. This is because this
SystemView access occurs outside of a request (during invalidation
handling) and thus the proper lock isn't held.
Because replication status cannot change within the lifetime of a plugin
(and instead, if a node switches replication status, the entire plugin
instance will be torn down and recreated), it is safe to cache this
once, at plugin startup, and use it throughout its lifetime.
Thus, we replace this SystemView access with a stored boolean variable
computed ahead of time.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update builtin/logical/pki/backend.go
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
There were two races here:
1. Tests racing against periodic func on updating the backend.
2. Tests racing internally to itself, to access the http-served
CRL data.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fetch CRLs from a user defined CDP (PoC)
* Handle no param sent
* Move CRL fetch to a periodFunc. Use configured CA certs + system root as trusted certs for CRL fetch
* comments
* changelog
* Just use root trust
* cdp->url in api
* Store CRL and populate it initially in cdlWrite
* Update docs
* Update builtin/credential/cert/path_crls.go
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Handle pre-verification of a CRL url better
* just in case
* Fix crl write locking
* Add a CRL fetch unit test
* Remove unnecessary validity clear
* Better func name
* Don't exit early updating CRLs
* lock in updateCRLs
* gofumpt
* err-
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Add plugin version to GRPC interface
Added a version interface in the sdk/logical so that it can be shared between all plugin types, and then wired it up to RunningVersion in the mounts, auth list, and database systems.
I've tested that this works with auth, database, and secrets plugin types, with the following logic to populate RunningVersion:
If a plugin has a PluginVersion() method implemented, then that is used
If not, and the plugin is built into the Vault binary, then the go.mod version is used
Otherwise, the it will be the empty string.
My apologies for the length of this PR.
* Placeholder backend should be external
We use a placeholder backend (previously a framework.Backend) before a
GRPC plugin is lazy-loaded. This makes us later think the plugin is a
builtin plugin.
So we added a `placeholderBackend` type that overrides the
`IsExternal()` method so that later we know that the plugin is external,
and don't give it a default builtin version.
This option was elided from the default value for the usage field. This
results in issuers "losing" ocsp-signing when they're POST updated. Most
issuers will want OCSP signing by default, so it makes sense to add this
as the default.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Support version selection for database plugins
* Don't consider unversioned plugins for version selection algorithm
* Added version to 'plugin not found' error
* Add PluginFactoryVersion function to avoid changing sdk/ API
* Allow exposing access to the underlying container
This exposes the Container response from the Docker API, allowing
consumers of the testhelper to interact with the newly started running
container instance. This will be useful for two reasons:
1. Allowing radiusd container to start its own daemon after modifying
its configuration.
2. For loading certificates into a future similar integration test
using the PKI secrets engine.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow any client to connect to test radiusd daemon
This fixes test failures of the following form:
> 2022-09-07T10:46:19.332-0400 [TRACE] core: adding local paths: paths=[]
> 2022-09-07T10:46:19.333-0400 [INFO] core: enabled credential backend: path=mnt/ type=test
> 2022-09-07T10:46:19.334-0400 [WARN] Executing test step: step_number=1
> 2022-09-07T10:46:19.334-0400 [WARN] Executing test step: step_number=2
> 2022-09-07T10:46:29.334-0400 [WARN] Executing test step: step_number=3
> 2022-09-07T10:46:29.335-0400 [WARN] Executing test step: step_number=4
> 2022-09-07T10:46:39.336-0400 [WARN] Requesting RollbackOperation
> --- FAIL: TestBackend_acceptance (28.56s)
> testing.go:364: Failed step 4: erroneous response:
>
> &logical.Response{Secret:<nil>, Auth:<nil>, Data:map[string]interface {}{"error":"context deadline exceeded"}, Redirect:"", Warnings:[]string(nil), WrapInfo:(*wrapping.ResponseWrapInfo)(nil), Headers:map[string][]string(nil)}
> FAIL
> FAIL github.com/hashicorp/vault/builtin/credential/radius 29.238s
In particular, radiusd container ships with a default clients.conf which
restricts connections to ranges associated with the Docker daemon. When
creating new networks (such as in CircleCI) or when running via Podman
(which has its own set of network ranges), this initial config will no
longer be applicable. We thus need to write a new config into the image;
while we could do this by rebuilding a new image on top of the existing
layers (provisioning our config), we then need to manage these changes
and give hooks for the service setup to build it.
Thus, post-startup modification is probably easier to execute in our
case.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- When we added new tests that validate the RSA PSS feature, they
work properly on normal Go builds, but tests underneath the Boring
Crypto fips implementations fail due to a lack of SHA3 support in
FIPS 140-2.
* Get import correct
* limits, docs
* changelog
* unit tests
* And fix import for hmac unit test
* typo
* Update website/content/api-docs/secret/transit.mdx
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Update builtin/logical/transit/path_keys.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Validate key sizes a bit more carefully
* Update sdk/helper/keysutil/policy.go
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
* Add fields 'ttl' and 'num_uses' to SecretID generation.
Add fields 'ttl' and 'num_uses' when generating/obtaining a SecretID.
Rather than just being able to use the Role's SecretID ttl and num uses. #14390
* Add secret_id_num_uses response field to generating SecretID
Add the response field secret_id_num_uses to the endpoints for generating
SecretIDs. Used in testing but also to supply the vendor with this variable.
* Add tests for new ttl and num_uses SecretID generation fields
Add tests to assert the new TTL and NumUses option in the SecretID entry.
Separate test for testing with just parameters vs a -force example.
* Patch up test for ttl and num_uses fields
* Add changelog entry for auth/approle 'ttl' and 'num_uses' fields
* Add fields to API Docs and AppRole Auth Docs example
* Correct error message for failing test on missing field.
Change the error message produced when a test fails due to a missing field.
Previous values did not map to correct fields.
* Remove unnecessary int cast to int "secret_id_num_uses" field.
Unnecessary cast to int where type already is int.
* Move numUses field check to after assignment.
* Remove metadata entry in sample payload to limit change to changes made.
Remove metadata entry in sample payload for custom-secret-id. The metadata was not
changed in the features pull request.
* Bind fields 'ttl' and 'num_uses' to role's configuration.
Rather than implicitly overriding, error when the ttl is lower than and the num
uses higher than the role's configuration. #14390
* Update changelog 14474 with a more detailed description.
More elaborate description for the changelog. Specifying the per-request based fields.
* Elaborate more on the bounds of the 'ttl' and 'num_uses' field.
Specify in both the api-docs and the CLI the limits of the fields.
Specify that the role's configuration is still the leading factor.
* Upper bound ttl with role secret id ttl
Upper bound ttl with role secret id ttl when creating a secret id
Adding test cases for infinite ttl and num uses
Adding test cases for negative ttl and num uses
Validation on infinite ttl and num uses
* Formatting issues. Removed unnecessary newline
* Update documentation for AppRole Secret ID and Role
Changed that TTL is not allowed to be shorter to longer
* Cleanup approle secret ID test and impl
* Define ttl and num_uses in every test
Define ttl and num_uses in every test despite them not being tested.
This is to ensure that no unexpected behaviour comes to mind.
* Rename test RoleSecretID -> RoleSecretIDWithoutFields
* Test secret id generation defaults to Role's config
Test secret id generation defaults to Role's configuration entries.
* Change finit -> finite
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
* Rephrase comments to the correct validation check
* Rephrase role-secret-id option description
* Remove "default" incorrect statement about ttl
* Remove "default" incorrect statement about ttl for custom secret id
* Touch up approle.mdx to align more with path_role documentation
Co-authored-by: Remco Buddelmeijer <r.buddelmeijer@fullstaq.com>
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
* Add path to manually rebuild delta CRLs
The crl/rotate-delta path behaves like crl/rotate, triggering a
cluster-local rebuild of just the delta CRL. This is useful for when
delta CRLs are enabled with a longer-than-desired auto-rebuild period
after some high-profile revocations occur.
In the event delta CRLs are not enabled, this becomes a no-op.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for Delta CRL rebuilding
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update documentation about Delta CRLs
Also fixes a omission in the If-Modified-Since docs to mention that the
response header should probably also be passed through.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow tidy operations to be cancelled
When tidy operations take a long time to execute (and especially when
executing them automatically), having the ability to cancel them becomes
useful to reduce strain on Vault clusters (and let them be rescheduled
at a later time).
To this end, we add the /tidy-cancel write endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing auto-tidy synopsis / description
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add a pause duration between tidying certificates
By setting pause_duration, operators can have a little control over the
resource utilization of a tidy operation. While the list of certificates
remain in memory throughout the entire operation, a pause is added
between processing certificates and the revocation lock is released.
This allows other operations to occur during this gap and potentially
allows the tidy operation to consume less resources per unit of time
(due to the sleep -- though obviously consumes the same resources over
the time of the operation).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for cancellation, pause
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add API docs on pause_duration, /tidy-cancel
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add lock releasing around tidy pause
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Reset cancel guard, return errors
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Alexander Scheel