* Use a UUID for ACME kid instead of a key fingerprint
* PR feedback
- Calculate thumbprint within CreateAccount instead of passing it in
- Reorder writes within CreateAccount to now write out thumbprint entry
first as we can easily recover/overwrite it if we fail mid-way
- Change back LoadAccount in acme to return an error if it fails to
lookup the entry
* Clearify comment within ACME CreateAccount
* Rework ACME workflow test to leverage Golang's ACME client library
- Instead of testing manually, leverage the Golang ACME library
to test against our implementation from the unit tests.
* Add tests for new-account and misc fixes
- Set and return the account status for registration
- Add handlers for the account/ api/updates
- Switch acme/ to cluster local storage
- Disable terms of service checks for now as we don't set the url
* PR feedback
- Implement account deactivation
- Create separate account update handler, to not mix account creation
logic
- Add kid field to account update definition
- Add support to update contact details on an existing account
* Correctly find certificates for unified delta CRL
When building the unified delta CRL, WAL entries from the non-primary
cluster were ignored. This resulted in an incomplete delta CRL,
preventing some entries from appearing.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly rebuild unified delta CRLs
When deciding if the Unified Delta CRL should be rebuilt, we need to
check the status of all clusters and their last revoked serial numbers.
If any new serial has been revoked on any cluster, we should rebuild the
unified delta CRLs.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly persist Unified Delta CRL build entries
When building the unified CRL, we need to read the last seen serial
number from all clusters, not just the present cluster, and write it
to the last built serial for that cluster's unified delta WAL entry.
This prevents us from continuously rebuilding unified CRLs now that we
have fixed our rebuild heuristic.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix getLastWALSerial for unified delta CRLs
getLastWALSerial ignored its path argument, preventing it from reading
the specified cluster-specific WAL entry. On the primary cluster, this
was mostly equivalent, but now that we're correctly reading WAL entries
and revocations for other clusters, we need to handle reading these
entries correctly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Copy delta WAL entries in event of failure
Any local delta WAL should be persisted to unified delta WAL space as
well. If such unified persistence fails, we need to ensure that they get
eventually moved up, otherwise they'll remain missing until the next
full CRL rebuild occurs, which might be significantly longer than when
the next delta CRL rebuild would otherwise occur. runUnifiedTransfer
already handles this for us, but it lacked logic for delta WAL serials.
The only interesting catch here is that we refuse to copy any entries
whose full unified revocation entry has not also been written.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Make doUnifiedTransferMissingLocalSerials log an error
This message is mostly an error and would always be helpful information
to have when troubleshooting failures.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Warn on cross-cluster write failures during revoke
When revoking certificates, we log cross-cluster revocation failures,
but we should really expose this information to the caller, that their
local revocation was successful, but their cross-cluster revocation
failed.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ensure unified delta WAL entry has full entry
Delta WAL entries are empty files whose only information (a revoked
serial number) is contained in the file path. These depend implicitly on
a full revocation entry existing for this file (whether a cross-cluster
unified entry or a local entry).
We should not write unified delta WAL entries without the corresponding
full unified revocation entry existing. Add a warning in this case.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
When reading the config, we attempt to detect if the running Vault
instance has been changed from its Enterprise status on write.
Similarly, we should detect if the mount is a local mount instead. While
this isn't changeable at runtime, using sys/raw to side-load an invalid
config could be possible.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Log, don't err, on unified delta WAL write failure
When the PBPWF fails on the Active node of a PR Secondary cluster with a
read-only failure, there is no value in forwarding this request up to
the Active node of the PR Primary cluster: it does not have the local
revocation context necessary to write a Delta WAL entry for this
request, and would likely end up writing a cross-cluster revocation
entry (if it is enabled) or else erring completely.
Instead, log this error like we do when failing to write unified CRL
entries. Switch both to using Error instead of Debug for this type of
failure.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Please see #19319 for more details on how this will affect the generated OpenAPI schema.
___
The following OperationID's will be generated for Nomad plugin:
nomad-read-access-configuration
nomad-configure-access
nomad-delete-access-configuration
nomad-read-lease-configuration
nomad-configure-lease
nomad-delete-lease-configuration
nomad-generate-credentials
nomad-list-roles
nomad-read-role
nomad-write-role
nomad-delete-role
* Enable creation of accounts
- Refactors many methods to take an acmeContext, which holds the
storageContext on it.
- Updates the core ACME Handlers to use *acmeContext, to avoid
copying structs.
- Makes JWK exported so the JSON parser can find it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Finish ACME account creation
- This ensures a Kid is created when one doesn't exist
- Expands the parsed handler capabilities, to format the response and
set required headers.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- Instead of using tests that just test the plugin storage/interface
layer, use a full Vault instance to validate that we can send/receive
the proper headers and responses back to a client.
- Found an issue with HEAD new-nounce api calls returning 500 errors.
- Add the /acme/ suffix to the baseUrl in the acme context so we don't
have to keep adding it a bit everywhere.
* Squash pki/acme package down to pki folder
Without refactoring most of PKI to export the storage layer, which we
were initially hesitant about, it would be nearly impossible to have the
ACME layer handle its own storage while being in the acme/ subpackage
under the pki package.
Thus, merge the two packages together again.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Properly format errors for missing parameters
When missing required ACME request parameters, don't return Vault-level
errors, but drop into the PKI package to return properly-formatted ACME
error messages.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Error type clarifications
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix GetOk with type conversion calls
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Identify whether JWKs existed or were created, set KIDs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Reclassify ErrAccountDoesNotExist as 400 per spec
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional stub methods for ACME accounts
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Start adding ACME newAccount handlers
This handler supports two pieces of functionality:
1. Searching for whether an existing account already exists.
2. Creating a new account.
One side effect of our JWS parsing logic is that we needed a way to
differentiate between whether a JWK existed on disk from an account or
if it was specified in the request. This technically means we're
potentially responding to certain requests with positive results (e.g.,
key search based on kid) versus erring earlier like other
implementations do.
No account storage has been done as part of this commit.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Unify path fields handling, fix newAccount method
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new PKI ACME subpackage to test_packages list
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Restrict JWS keys to specified algorithms
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ACME package to provide a nonce service
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add JWS parsing helper
Using go-jose v2, we start building a JWS parsing helper, ensuring that
fields are properly validated w.r.t. the ACME spec's intersection with
JWS.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add error context information
Start adding the ability to wrap errors returned by Vault to
ACME-specific errors.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Make ACMEState exported
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix for PKI.TestStandby_Operations test to work in ENT
- Remove wait call to testhelpers.WaitForActiveNodeAndStandbys and
leverage testhelpers.WaitForStandbyNode instead.
* Use InmemBackendSetup for a proper HA backend in ENT
* Forward PKI revocation requests received by standby nodes to active node
- A refactoring that occurred in 1.13 timeframe removed what was
considered a specific check for standby nodes that wasn't required
as a writes should be returning ErrReadOnly.
- That sadly exposed a long standing bug where the errors from the
storage layer were not being properly wrapped, hiding the ErrReadOnly
coming from a write and failing the request.
* Add cl
* Add test for basic PKI operations against standby nodes
* Add support for importing RSA-PSS keys in Transit
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* plugin/auth: enable multiplexing
- the plugin will be multiplexed when run as an external plugin
by vault versions that support secrets/auth plugin multiplexing (> 1.12)
- we continue to set the TLSProviderFunc to maintain backwards
compatibility with vault versions that don't support AutoMTLS (< 1.12)
* enable multiplexing for secrets engines
* add changelog
* revert call to ServeMultiplex for pki and transit
* Revert "revert call to ServeMultiplex for pki and transit"
This reverts commit 755be28d14b4c4c4d884d3cf4d2ec003dda579b9.
* Telemetry Metrics Configuration.
* Err Shadowing Fix (woah, semgrep is cool).
* Fix TestBackend_RevokePlusTidy_Intermediate
* Add Changelog.
* Fix memory leak. Code cleanup as suggested by Steve.
* Turn off metrics by default, breaking-change.
* Show on tidy-status before start-up.
* Fix tests
* make fmt
* Add emit metrics to periodicFunc
* Test not delivering unavailable metrics + fix.
* Better error message.
* Fixing the false-error bug.
* make fmt.
* Try to fix race issue, remove confusing comments.
* Switch metric counter variables to an atomic.Uint32
- Switch the metric counter variables to an atomic variable type
so that we are forced to properly load/store values to it
* Fix race-issue better by trying until the metric is sunk.
* make fmt.
* empty commit to retrigger non-race tests that all pass locally
---------
Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
- This fix was incorrect as now the tests and program are double
URL encoding the OCSP GET requests, so the base64 + characters
when using Vault proper are becoming space characters.
* Use the unified CRL on legacy CRL paths if UnifiedCRLOnExistingPaths is set
- If the crl configuration option unified_crl_on_existing_paths is set
to true along with the unified_crl feature, provide the unified crl
on the existing CRL paths.
- Added some test helpers to help debugging, they are being used by
the ENT test that validates this feature.
* Rename method to shouldLocalPathsUseUnified
* Use UTC for leaf exceeding CA's notAfter
When generating a leaf which exceeds the CA's validity period, Vault's
error message was confusing as the leaf would use the server's time
zone, but the CA's notAfter date would use UTC. This could cause
user confusion as the leaf's expiry might look before the latter, due
to using different time zones. E.g.:
> cannot satisfy request, as TTL would result in notAfter
> 2023-03-06T16:41:09.757694-08:00 that is beyond the expiration of
> the CA certificate at 2023-03-07T00:29:52Z
Consistently use UTC for this instead.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Apply URL encoding/unencoding to OCSP Get requests
- Missed this during development and sadly the unit tests were written
at a level that did not expose this issue originally, there are
certain combinations of issuer cert + serial that lead to base64
data containing a '/' which will lead to the OCSP handler not getting
the full parameter.
- Do as the spec says, this should be treated as url-encoded data.
* Add cl
* Add higher level PKI OCSP GET/POST tests
* Rename PKI ocsp files to path_ocsp to follow naming conventions
* make fmt
* Add ability to clean up host keys for dynamic keys
This adds a new endpoint, tidy/dynamic-keys that removes any stale host
keys still present on the mount. This does not clean up any pending
dynamic key leases and will not remove these keys from systems with
authorized hosts entries created by Vault.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* PKI Unified CRL/OCSP apis should be ent only
- Do not enable any of the unified crl/ocsp related apis on OSS.
* Rollback refactoring of pathFetchCRLViaCertPath
- As pointed out in the PR, this method isn't actually being used at
the moment with the <serial> handler, pathFetchValid, matching
everything under the cert/XXXX path.
* Fix schema for ent/oss diff
- Define the OSS vs ENT urls we want to see within the schema
definition even if they aren't really going to be used in the end.
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package. Consolidate on a single "noop audit" implementation.
Steven Clark