* Refactor setting local addresses
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate wildcard domains in ACME test suite
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add locking to DNS resolver
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Better removal semantics for records
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Export DockerAPI for use by other consumers
As usage of DockerCluster gets more advanced, some users may want to
interact with the container nodes of the cluster. While, if you already
have a DockerAPI instance lying around you can reuse that safely, for
use cases where an existing e.g., docker/testhelpers's runner instance
is not available, reusing the existing cluster's DockerAPI is easiest.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ability to exec commands without runner
When modifying DockerTestCluster's containers manually, we might not
have a Runner instance; instead, expose the ability to run commands via
a DockerAPI instance directly, as they're awfully convenient.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS resolver into ACME tests
This updates the pkiext_binary tests to use an adjacent DNS resolver,
allowing these tests to eventually be extended to solve DNS challenges,
as modifying the /etc/hosts file does not allow this.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix loading DNS resolver onto network
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix bug with DNS configuration validation
Both conditionals here were inverted: address being empty means a bad
specification was given, and the parse being nil means that it was not a
valid IP address.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix specifying TXT records, allow removing records
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- Do not serialize the entire internal object, instead return
just the Type and Value fields back to the caller.
- Also within authorization responses, return the base domain
on wildcard queries, dropping the *. as the RFC requests.
- Update tests to reflect/test this logic.
- We have a known issue that is difficult to address in released versions of Vault that OCSP GET requests can contain consecutive / characters which the Golang HTTP mux will force a redirection.
- Instead of failing various PRs and runs with this known issue, check to see if we are about to trigger it and if so skip the test. We have already at this point tested the POST version of the API.
- When someone is fetching the order to get it's status, compute if we
need to bump the status to Ready like we do in finalize handler
- Add a wait state to the ACME docker test suite to deal with a race
condition
* Initial refactoring of ACME PKI binary tests
- Rework test suite to use a single Vault cluster with
different mounts.
- Refactor convenience methods to write PKI tests.
* Add ACME test cases for mixed IP and DNS, along with IP only identifier requests
* Parallelize the Vault PKI test suite
* Add missing tidy-status state values
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add docs on auto-tidy reading
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing tidy status field revocation_queue_safety_buffer
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Include pause_duration in tidy-status docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add date of last auto-tidy operation to status
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional existing keys response field
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update tests for validating existing keys
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update docs for import to include new fields
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update website/content/api-docs/secret/pki.mdx
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Validate identifiers against role when creating order
Perform some initial validation against the order's requested
identifiers during creation; this gives a client a heads up that their
request might be rejected by the server before they have to solve
challenges for these, only to find out during CSR submission time that
there is no way to request the specified certificate.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add unit tests and switch errors to ErrRejectedIdentifier
- Change the error messages from validating identifiers against the
role to ErrRejectedIdentifier errors if they do occur
- Add unit tests to validate that we validate against the various
roles somewhat okay.
* go doc
* Fix typo in test godoc
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add default ACME configuration, invalidate on write
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add enforcment of ACME enabled
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate requested role against ACME config
Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add validation of issuer restrictions with ACME
Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add validation around allowed config lenghts
Co-authored-by: kitography <khaines@mit.edu>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Prune later deemed unnecessary config options
Co-authored-by: kitography <khaines@mit.edu>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* make fmt
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: kitography <khaines@mit.edu>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add PKI ACME IP SANS test case
- Leveraging the new Vault docker based tests along with nginx and
the Go ACME library, verify that Vault can properly perform ACME
validations for IP identifiers
* Formatting
* Handle caching of ACME config
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS resolvers to ACME configuration
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add custom DNS resolver to challenge verification
This required plumbing through the config, reloading it when necessary,
and creating a custom net.Resolver instance.
Not immediately clear is how we'd go about building a custom DNS
validation mechanism that supported multiple resolvers. Likely we'd need
to rely on meikg/dns and handle the resolution separately for each
container and use a custom Dialer that assumes the address is already
pre-resolved.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Improvements to Docker harness
- Expose additional service information, allowing callers to figure out
both the local address and the network-specific address of the
service container, and
- Allow modifying permissions on uploaded container files.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add infrastructure to run Bind9 in a container for tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate DNS-01 challenge works
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add the ability to disable ACME through an OS environment variable
- Disable ACME through the VAULT_DISABLE_PUBLIC_ACME environment
variable.
* PR feedback
- Switch to using t.Setenv instead of manually doing it
- Remove t.Parallel from the test not to influence others
* make fmt
* Enforce ACME accounts to a specific directory path
- Accounts and correspondingly orders, authz should not cross
the path boundaries. So we now tag an ACME account with a specific
directory based on the requested role/issuer values in the path.
- If an operation occurs on a different acme directory path it will
cause a failure of the request.
- Add some go doc to a few places and reorder the methods in the
acme_wrappers.go class to highlight the wrappers and not intertwine
the helper functions
- Rename path_acme_new_account.go to path_acme_account.go as it has
several account related methods now.
* Get rid of bad test case
- The previous commit contained a bug fix for us properly
loading issuers within the ACME path, that exposed
this broken/bad test case. Simply remove it.
* Add ACME revocation handlers
This refactors path_revoke to expose Proof of Possession verification,
which is reused by ACME to allow methods 1 and 2:
1. Revocation of a certificate issued by the account, using account
signature as sufficient proof.
2. Revocation of a certificate via proving possession of its private
key, using this private key to create the JWS signature.
We do not support the third mechanism, completing challenges equivalent
to those on the existing certificate and then performing a revocation
under an account which didn't issue the certificate but which did solve
those challenges.
We additionally create another map account->cert->order, allowing us to
quickly look up if a cert was issued by this particular account. Note
that the inverse lookup of cert->(account, order) lookup isn't yet
possible due to Vault's storage structure.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update ACME pkiext tests to revoke certs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add auth handler checks
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add enable_aia_url_templating to read issuer
This field was elided from read issuer responses, though the value
otherwise persisted correctly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add comprehensive test for patching issuers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing OpenAPI scheme definition
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Start ACME containerized test suite
This starts a containerized ACME test suite using containers, running
both Vault and Certbot (in standalone mode) in the container to ensure
we successfully issue certificates.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Switch to using hashicorp mirror
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support to load roles and issuers within ACME wrapper
* Add missing go doc to new test
* PR feedback
- Move field definitions into fields.go
- Update wording and associated errors to some role failures.
- Add missing ':' to error messages
* Add additional fields to LIST issuers for Web UI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS challenge validation to ACME
This allows us to validate against wildcard domain names, as the HTTP-01
challenge does not support wildcard validation.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Properly delay challenge retry attempts
Previously we'd essentially semi-busy wait for a challenge to become
retry-able, as the queue itself had no knowledge of the retry after
value of the validation attempt. Now, we plumb through this value into
the queue itself, to aide selection of validations to attempt.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor wildcard validation checks
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add helper to determine if identifier is wildcard
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Better validate wildcard acceptance
This correctly validates wildcards to contain only a leading prefix
(*.) and must include another label, also ensuring that the remaining
domain components pass non-IP and IDNA validation, and removing them
from display on the authorization. Finally, we restrict the challenge
types available for various combinations of IP, DNS, and wildcard
addresses.
This then updates the tests to validate this as well.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* WIP: Implement ACME CSR signing and certificate retrieval
* Add some validations within the ACME finalize API
- Validate that the CSR we were given matches the DNS names
and IP addresses within the order
- Validate that the CSR does not share the same public as the
account
* Gate ACME finalize order validating all authorizations are in valid state
* Add infrastructure for warnings on CRL rebuilds
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning on issuer missing KU for CRL Signing
When an entire issuer equivalency class is missing CRL signing usage
(but otherwise has key material present), we should add a warning so
operators can either correct this issuer or create an equivalent version
with KU specified.
Resolves: https://github.com/hashicorp/vault/issues/20137
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for issuer warnings
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix return order of CRL builders
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow creating storageContext with timeout
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add challenge validation engine to ACME
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Initialize the ACME challenge validation engine
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Trigger challenge validation on endpoint submission
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix GetKeyThumbprint to use raw base64
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Point at localhost for testing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add cleanup of validation engine
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* remove intercepting helpText
* add subtext directly to StringList input component
* update tests and add coverage for new openapi-attrs util
* update test
* add warning validation to input
* lol is this right i dont know go
* literally no idea what im doing
* add Description to display attrs struct
* update struct comment
* add descriptions to remaining go fields
* add missing comma
* remaining commas..."
* add description to display attrs
* update tests
* update tests
* add changelog;
* Update ui/app/utils/openapi-to-attrs.js
* update tests following backend changes
* clearly name variable
* format files
* no longer need to test for modified tooltip since coming from backend now
* Return OCSP errors on cert auth login failures
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Switch to immediately returning the first match
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Minor follow-ups to #16865
Fix PKI issuer upgrade logic when upgrading to 1.12 or later, to
actually turn off the issuer crl-signing usage when it intended to.
Fix minor typo in docs.
* changelog
* Add HTTP challenge validator
This will attempt to safely validate HTTP challenges, following a
limited number of redirects and timing out after too much time has
passed.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test for ValidateKeyAuthorization
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test cases for ValidateHTTP01Challenge
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add token to HTTP challenge
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove extraneous certificate from OCSP response
Since the issuer used to sign the certificate also signs the OCSP
response, no additional information is added by sending the issuer again
in the certs field of the BasicOCSPResponse structure. Removing it saves
bytes and avoids confusing Go-based OCSP verifiers which cannot handle
the cert issuer being duplicated in the certs field.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- Add a helper function that can accept the final API path along with
the pattern function for an ACME api definition and generate the
various flavors for the given API
* Move all ACME wrappers into a dedicated go file
- Make it easier to figure out where the various wrappers for
ACME exist by locating them inside a dedicated go file instead
of spread out across the various path_acme_xxx files.
* Add missing copyright headers to PKI files
* Implement ACME new-order API
- This is a very rough draft for the new order ACME API
* Add ACME order list API
* Implement ACME Get order API
* Misc order related fixes
- Filter authorizations in GetOrders for valid
- Validate notBefore and notAfter dates make sense
- Add <order>/cert URL path to order response if set to valid
* Return account status within err authorized, if the account key verified
* revert STS lease changes, now create a lease for STS credentials but keep the ttl
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
* Distinguish POST-as-GET from POST-with-empty-body
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ACME authorization, identifier, and challenge types
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ability to load and save authorizations
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ACME authorizations path handling
This supports two methods: a fetch handler over the authorization, to
expose the underlying challenges, and a deactivate handler to revoke
the authorization and mark its challenges invalid.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ACME challenge path handling
These paths kick off processing and validation of the challenge by the
ACME client.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use a UUID for ACME kid instead of a key fingerprint
* PR feedback
- Calculate thumbprint within CreateAccount instead of passing it in
- Reorder writes within CreateAccount to now write out thumbprint entry
first as we can easily recover/overwrite it if we fail mid-way
- Change back LoadAccount in acme to return an error if it fails to
lookup the entry
* Clearify comment within ACME CreateAccount
* Rework ACME workflow test to leverage Golang's ACME client library
- Instead of testing manually, leverage the Golang ACME library
to test against our implementation from the unit tests.
* Add tests for new-account and misc fixes
- Set and return the account status for registration
- Add handlers for the account/ api/updates
- Switch acme/ to cluster local storage
- Disable terms of service checks for now as we don't set the url
* PR feedback
- Implement account deactivation
- Create separate account update handler, to not mix account creation
logic
- Add kid field to account update definition
- Add support to update contact details on an existing account
* Correctly find certificates for unified delta CRL
When building the unified delta CRL, WAL entries from the non-primary
cluster were ignored. This resulted in an incomplete delta CRL,
preventing some entries from appearing.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly rebuild unified delta CRLs
When deciding if the Unified Delta CRL should be rebuilt, we need to
check the status of all clusters and their last revoked serial numbers.
If any new serial has been revoked on any cluster, we should rebuild the
unified delta CRLs.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly persist Unified Delta CRL build entries
When building the unified CRL, we need to read the last seen serial
number from all clusters, not just the present cluster, and write it
to the last built serial for that cluster's unified delta WAL entry.
This prevents us from continuously rebuilding unified CRLs now that we
have fixed our rebuild heuristic.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix getLastWALSerial for unified delta CRLs
getLastWALSerial ignored its path argument, preventing it from reading
the specified cluster-specific WAL entry. On the primary cluster, this
was mostly equivalent, but now that we're correctly reading WAL entries
and revocations for other clusters, we need to handle reading these
entries correctly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Copy delta WAL entries in event of failure
Any local delta WAL should be persisted to unified delta WAL space as
well. If such unified persistence fails, we need to ensure that they get
eventually moved up, otherwise they'll remain missing until the next
full CRL rebuild occurs, which might be significantly longer than when
the next delta CRL rebuild would otherwise occur. runUnifiedTransfer
already handles this for us, but it lacked logic for delta WAL serials.
The only interesting catch here is that we refuse to copy any entries
whose full unified revocation entry has not also been written.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Make doUnifiedTransferMissingLocalSerials log an error
This message is mostly an error and would always be helpful information
to have when troubleshooting failures.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Warn on cross-cluster write failures during revoke
When revoking certificates, we log cross-cluster revocation failures,
but we should really expose this information to the caller, that their
local revocation was successful, but their cross-cluster revocation
failed.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ensure unified delta WAL entry has full entry
Delta WAL entries are empty files whose only information (a revoked
serial number) is contained in the file path. These depend implicitly on
a full revocation entry existing for this file (whether a cross-cluster
unified entry or a local entry).
We should not write unified delta WAL entries without the corresponding
full unified revocation entry existing. Add a warning in this case.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
When reading the config, we attempt to detect if the running Vault
instance has been changed from its Enterprise status on write.
Similarly, we should detect if the mount is a local mount instead. While
this isn't changeable at runtime, using sys/raw to side-load an invalid
config could be possible.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Log, don't err, on unified delta WAL write failure
When the PBPWF fails on the Active node of a PR Secondary cluster with a
read-only failure, there is no value in forwarding this request up to
the Active node of the PR Primary cluster: it does not have the local
revocation context necessary to write a Delta WAL entry for this
request, and would likely end up writing a cross-cluster revocation
entry (if it is enabled) or else erring completely.
Instead, log this error like we do when failing to write unified CRL
entries. Switch both to using Error instead of Debug for this type of
failure.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Please see https://github.com/hashicorp/vault/pull/19319 for more details on how this will affect the generated OpenAPI schema.
____
### The following OperationID's will be generated for GitHub auth:
github-read-configuration
github-configure
github-log-in
github-read-teams
github-read-team-mapping
github-write-team-mapping
github-delete-team-mapping
github-read-users
github-read-user-mapping
github-write-user-mapping
github-delete-user-mapping
Please see #19319 for more details on how this will affect the generated OpenAPI schema.
___
The following OperationID's will be generated for Nomad plugin:
nomad-read-access-configuration
nomad-configure-access
nomad-delete-access-configuration
nomad-read-lease-configuration
nomad-configure-lease
nomad-delete-lease-configuration
nomad-generate-credentials
nomad-list-roles
nomad-read-role
nomad-write-role
nomad-delete-role
* Enable creation of accounts
- Refactors many methods to take an acmeContext, which holds the
storageContext on it.
- Updates the core ACME Handlers to use *acmeContext, to avoid
copying structs.
- Makes JWK exported so the JSON parser can find it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Finish ACME account creation
- This ensures a Kid is created when one doesn't exist
- Expands the parsed handler capabilities, to format the response and
set required headers.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- Instead of using tests that just test the plugin storage/interface
layer, use a full Vault instance to validate that we can send/receive
the proper headers and responses back to a client.
- Found an issue with HEAD new-nounce api calls returning 500 errors.
- Add the /acme/ suffix to the baseUrl in the acme context so we don't
have to keep adding it a bit everywhere.
* Squash pki/acme package down to pki folder
Without refactoring most of PKI to export the storage layer, which we
were initially hesitant about, it would be nearly impossible to have the
ACME layer handle its own storage while being in the acme/ subpackage
under the pki package.
Thus, merge the two packages together again.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Properly format errors for missing parameters
When missing required ACME request parameters, don't return Vault-level
errors, but drop into the PKI package to return properly-formatted ACME
error messages.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Error type clarifications
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix GetOk with type conversion calls
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Identify whether JWKs existed or were created, set KIDs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Reclassify ErrAccountDoesNotExist as 400 per spec
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional stub methods for ACME accounts
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Start adding ACME newAccount handlers
This handler supports two pieces of functionality:
1. Searching for whether an existing account already exists.
2. Creating a new account.
One side effect of our JWS parsing logic is that we needed a way to
differentiate between whether a JWK existed on disk from an account or
if it was specified in the request. This technically means we're
potentially responding to certain requests with positive results (e.g.,
key search based on kid) versus erring earlier like other
implementations do.
No account storage has been done as part of this commit.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Unify path fields handling, fix newAccount method
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add new PKI ACME subpackage to test_packages list
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Restrict JWS keys to specified algorithms
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ACME package to provide a nonce service
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add JWS parsing helper
Using go-jose v2, we start building a JWS parsing helper, ensuring that
fields are properly validated w.r.t. the ACME spec's intersection with
JWS.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add error context information
Start adding the ability to wrap errors returned by Vault to
ACME-specific errors.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Make ACMEState exported
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix for PKI.TestStandby_Operations test to work in ENT
- Remove wait call to testhelpers.WaitForActiveNodeAndStandbys and
leverage testhelpers.WaitForStandbyNode instead.
* Use InmemBackendSetup for a proper HA backend in ENT
* Forward PKI revocation requests received by standby nodes to active node
- A refactoring that occurred in 1.13 timeframe removed what was
considered a specific check for standby nodes that wasn't required
as a writes should be returning ErrReadOnly.
- That sadly exposed a long standing bug where the errors from the
storage layer were not being properly wrapped, hiding the ErrReadOnly
coming from a write and failing the request.
* Add cl
* Add test for basic PKI operations against standby nodes
* Add support for importing RSA-PSS keys in Transit
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* impr(auth/ldap): allow to dereference aliases in searches
* docs: add documentation for LDAP alias dereferencing
* chore(auth/ldap): add changelog entry for PR 18230
* chore: run formatter
* fix: update default LDAP configuration with new default
* Update website/content/docs/auth/ldap.mdx
Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
* docs(ldap): add alias dereferencing to API docs for LDAP
---------
Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
* add nil check for secret id entry on delete via accessor
* add changelog
* add godoc to test
* improve feedback on nil entry
* fix error reporting on invalid secret id accessor
* fix test to expect implemented error
* feat(auth/ldap): allow passing the LDAP password via an environment variable when authenticating via the CLI
* chore(auth/ldap): add changelog entry for PR 18225
* plugin/auth: enable multiplexing
- the plugin will be multiplexed when run as an external plugin
by vault versions that support secrets/auth plugin multiplexing (> 1.12)
- we continue to set the TLSProviderFunc to maintain backwards
compatibility with vault versions that don't support AutoMTLS (< 1.12)
* enable multiplexing for secrets engines
* add changelog
* revert call to ServeMultiplex for pki and transit
* Revert "revert call to ServeMultiplex for pki and transit"
This reverts commit 755be28d14b4c4c4d884d3cf4d2ec003dda579b9.
* Telemetry Metrics Configuration.
* Err Shadowing Fix (woah, semgrep is cool).
* Fix TestBackend_RevokePlusTidy_Intermediate
* Add Changelog.
* Fix memory leak. Code cleanup as suggested by Steve.
* Turn off metrics by default, breaking-change.
* Show on tidy-status before start-up.
* Fix tests
* make fmt
* Add emit metrics to periodicFunc
* Test not delivering unavailable metrics + fix.
* Better error message.
* Fixing the false-error bug.
* make fmt.
* Try to fix race issue, remove confusing comments.
* Switch metric counter variables to an atomic.Uint32
- Switch the metric counter variables to an atomic variable type
so that we are forced to properly load/store values to it
* Fix race-issue better by trying until the metric is sunk.
* make fmt.
* empty commit to retrigger non-race tests that all pass locally
---------
Co-authored-by: Steve Clark <steven.clark@hashicorp.com>
- This fix was incorrect as now the tests and program are double
URL encoding the OCSP GET requests, so the base64 + characters
when using Vault proper are becoming space characters.
* Use the unified CRL on legacy CRL paths if UnifiedCRLOnExistingPaths is set
- If the crl configuration option unified_crl_on_existing_paths is set
to true along with the unified_crl feature, provide the unified crl
on the existing CRL paths.
- Added some test helpers to help debugging, they are being used by
the ENT test that validates this feature.
* Rename method to shouldLocalPathsUseUnified
* Use UTC for leaf exceeding CA's notAfter
When generating a leaf which exceeds the CA's validity period, Vault's
error message was confusing as the leaf would use the server's time
zone, but the CA's notAfter date would use UTC. This could cause
user confusion as the leaf's expiry might look before the latter, due
to using different time zones. E.g.:
> cannot satisfy request, as TTL would result in notAfter
> 2023-03-06T16:41:09.757694-08:00 that is beyond the expiration of
> the CA certificate at 2023-03-07T00:29:52Z
Consistently use UTC for this instead.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix race accessing b.crls within cert auth
- Discovered by CircleCI the pathLogin, pathLoginRenew paths access
and reloads the b.crls member variable without a lock.
- Also discovered that pathLoginResolveRole never populated an empty
b.crls before usage within b.verifyCredentials
* Add cl
* Misc cleanup
- Introduce a login path wrapper instead of repeating in all the
various login methods the crl reloading
- Cleanup updatedConfig, never returned an error and nothing looked at
the error returned
- Make the test within TestCRLFetch a little less timing sensitive as
I was able to trigger a failure due to my machine taking more than
150ms to load the new CRL
* Apply URL encoding/unencoding to OCSP Get requests
- Missed this during development and sadly the unit tests were written
at a level that did not expose this issue originally, there are
certain combinations of issuer cert + serial that lead to base64
data containing a '/' which will lead to the OCSP handler not getting
the full parameter.
- Do as the spec says, this should be treated as url-encoded data.
* Add cl
* Add higher level PKI OCSP GET/POST tests
* Rename PKI ocsp files to path_ocsp to follow naming conventions
* make fmt
* Add ability to clean up host keys for dynamic keys
This adds a new endpoint, tidy/dynamic-keys that removes any stale host
keys still present on the mount. This does not clean up any pending
dynamic key leases and will not remove these keys from systems with
authorized hosts entries created by Vault.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* PKI Unified CRL/OCSP apis should be ent only
- Do not enable any of the unified crl/ocsp related apis on OSS.
* Rollback refactoring of pathFetchCRLViaCertPath
- As pointed out in the PR, this method isn't actually being used at
the moment with the <serial> handler, pathFetchValid, matching
everything under the cert/XXXX path.
* Fix schema for ent/oss diff
- Define the OSS vs ENT urls we want to see within the schema
definition even if they aren't really going to be used in the end.
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package. Consolidate on a single "noop audit" implementation.
* Remove dynamic keys from SSH Secrets Engine
This removes the functionality of Vault creating keys and adding them to
the authorized keys file on hosts.
This functionality has been deprecated since Vault version 0.7.2.
The preferred alternative is to use the SSH CA method, which also allows
key generation but places limits on TTL and doesn't require Vault reach
out to provision each key on the specified host, making it much more
secure.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove dynamic ssh references from documentation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove dynamic key secret type entirely
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify changelog language
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add removal notice to the website
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address pki::TestAutoRebuild flakiness
- Wait for a CRL change before progressing to the next step after
we change configuration. Prior to this we would be racing against
the CRL reloading from the configuration change.
* Read total cert counts with atomic.LoadUint32(...)
When generating the tidy status, we read the values of two backend
atomics, b.certCount and b.revokedCertCount, without using the atomic
load operation. This resulted in a data race when the status was read
at the same time as an on-going tidy operation:
WARNING: DATA RACE
Write at 0x00c00c77680c by goroutine 90522:
sync/atomic.AddInt32()
/usr/local/go/src/runtime/race_amd64.s:281 +0xb
sync/atomic.AddUint32()
<autogenerated>:1 +0x1a
github.com/hashicorp/vault/builtin/logical/pki.(*backend).tidyStatusIncRevokedCertCount()
/home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:1236 +0x107
github.com/hashicorp/vault/builtin/logical/pki.(*backend).doTidyRevocationStore()
/home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:525 +0x1404
github.com/hashicorp/vault/builtin/logical/pki.(*backend).startTidyOperation.func1.1()
/home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:290 +0x1a4
github.com/hashicorp/vault/builtin/logical/pki.(*backend).startTidyOperation.func1()
/home/circleci/go/src/github.com/hashicorp/vault/builtin/logical/pki/path_tidy.go:342 +0x278
Previous read at 0x00c00c77680c by goroutine 90528:
reflect.Value.Uint()
/usr/local/go/src/reflect/value.go:2584 +0x195
encoding/json.uintEncoder()
/usr/local/go/src/encoding/json/encode.go:562 +0x45
encoding/json.ptrEncoder.encode()
/usr/local/go/src/encoding/json/encode.go:944 +0x3c2
encoding/json.ptrEncoder.encode-fm()
<autogenerated>:1 +0x90
encoding/json.(*encodeState).reflectValue()
/usr/local/go/src/encoding/json/encode.go:359 +0x88
encoding/json.interfaceEncoder()
/usr/local/go/src/encoding/json/encode.go:715 +0x17b
encoding/json.mapEncoder.encode()
/usr/local/go/src/encoding/json/encode.go:813 +0x854
... more stack trace pointing into JSON encoding and http
handler...
In particular, because the tidy status was directly reading the uint
value without resorting to the atomic side, the JSON serialization could
race with a later atomic update.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Also use atomic load in tests
Because no tidy operation is running here, it should be safe to read the
pointed value directly, but use the safer atomic.Load for consistency.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move cert auth backend setup into initialize
In further review with new understanding after #18244, loading
configuration and CRLs within the backend's initialize function is the
ideal approach: Factory construction is strictly serial, resulting in
backend initialization blocking until config and CRLs are loaded.
By using an InitializeFunc(...), we delay loading until after all
backends are constructed (either right on startup in 1.12+, else during
the initial PeriodicFunc(...) invocation on 1.11 and earlier).
We also invoke initialize automatically on test Factory construction.
Resolves: #17847
Co-authored-by: valli_0x <personallune@mail.ru>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: valli_0x <personallune@mail.ru>
- This has been done to help diagnose errors in the future so that
we get the callers in the trace's when we fail and not just the
helper's trace output.
* Allow unification of revocations on other clusters
If a BYOC revocation occurred on cluster A, while the cert was initially
issued and stored on cluster B, we need to use the invalidation on the
unified entry to detect this: the revocation queues only work for
non-PoP, non-BYOC serial only revocations and thus this BYOC would be
immediately accepted on cluster A. By checking all other incoming
revocations for duplicates on a given cluster, we can ensure that
unified revocation is consistent across clusters.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use time-of-use locking for global revocation processing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Unified revocation migration code
- Add a periodic function that will list the local revocations
and if any are missing from the unified revocation area will
force a write to the unified revocation folder/remote instance.
* PR Feedback
- Do not transfer expired certificates to unified space from local
- Move new periodic code into a periodic.go file
- Add a flag so we only run this stuff once if all is good, with
a force flag if we encounter errors or if unified_crl is toggled
on
* PR feedback take 2
- Return a detailed reponse within the list api that an end-user can
use to determine what clusters revoked the certificate on from the
pki/certs/unified-revoked LIST api.
- Return colon delimited serial numbers from the certs/revocation-queue
LIST api
* Add new tidy operation for cross revoked certs
This operation allows tidying of the cross-cluster revocation storage.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix missing cancels, status values
Previous additions to tidy didn't have enough cancel operations and left
out some new values from the status operation.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify error on due to unsupported EC key bits
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove documentation about unsupported EC/224
Resolves: #18843
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- I missed this in the original review, that we were storing the
unified-crl in a cluster-local storage area so none of the other
hosts would receive it.
- Discovered while writing unit tests, the main cluster had the unified
crl but the other clusters would return an empty response
* The fields.
* UserID set, add to certificate
* Changelog.
* Fix test (set default).
* Add UserID constant to certutil, revert extension changes
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add user_ids as field for leaf signing
Presumably, this isn't necessary for CAs, given that CAs probably don't
have a user ID corresponding to them.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Support setting multiple user_ids in Subject
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow any User ID with sign-verbatim
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for User IDs in PKI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add docs about user_ids, allowed_user_ids
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
- It does not make sense to allow operators to enable the cross-cluster
revocation features on local mounts as they will never have a
corresponding mount on the other cluster.
* Add unified CRL config storage helpers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support to build unified CRLs
This allows us to build unified versions of both the complete and delta
CRLs. This mostly involved creating a new variant of the
unified-specific CRL builder, fetching certs from each cluster's storage
space.
Unlike OCSP, here we do not unify the node's local storage with the
cross-cluster storage: this node is the active of the performance
primary, so writes to unified storage happen exactly the same as
writes to cluster-local storage, meaning the two are always in
sync. Other performance secondaries do not rebuild the CRL, and hence
the out-of-sync avoidance that we'd like to solve with the OCSP
responder is not necessary to solve here.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ability to fetch unified CRLs
This adds to the path-fetch APIs the ability to return the unified CRLs.
We update the If-Modified-Since infrastructure to support querying the
unified CRL specific data and fetchCertBySerial to support all unified
variants. This works for both the default/global fetch APIs and the
issuer-specific fetch APIs.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rebuild CRLs on unified status changes
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Handle rebuilding CRLs due to either changing
This allows detecting if the Delta CRL needs to be rebuilt because
either the local or the unified CRL needs to be rebuilt. We never
trigger rebuilding the unified delta on a non-primary cluster.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Ensure serials aren't added to unified CRL twice
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Write delta WAL entries for unified CRLs
When we'd ordinarily write delta WALs for local CRLs, we also need to
populate the cross-cluster delta WAL. This could cause revocation to
appear to fail if the two clusters are disconnected, but notably regular
cross-cluster revocation would also fail.
Notably, this commit also changes us to not write Delta WALs when Delta
CRLs is disabled (versus previously doing it when auto rebuild is
enabled in case Delta CRLs were later asked for), and instead,
triggering rebuilding a complete CRL so we don't need up-to-date Delta
WAL info.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update IMS test for forced CRL rebuilds
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move comment about perf-primary only invalidation
Also remove noisy debug log.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove more noisy log statements during queue
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Skip revocation entries from our current cluster
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add locking and comment about tidying revoke queue
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Switch to time.Since for tidy
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor tidyStatuses into path_tidy.go
Leaving these in backend.go often causes us to miss adding useful values
to tidyStatus when we add a new config parameter.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Track the number of deleted revocation request
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow tidy to remove confirmed revocation requests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing field to tidy test
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add global, cross-cluster revocation queue to PKI
This adds a global, cross-cluster replicated revocation queue, allowing
operators to revoke certificates by serial number across any cluster. We
don't support revoking with private key (PoP) in the initial
implementation.
In particular, building on the PBPWF work, we add a special storage
location for handling non-local revocations which gets replicated up to
the active, primary cluster node and back down to all secondary PR
clusters. These then check the pending revocation entry and revoke the
serial locally if it exists, writing a cross-cluster confirmation entry.
Listing capabilities are present under pki/certs/revocation-queue,
allowing operators to see which certs are present. However, a future
improvement to the tidy subsystem will allow automatic cleanup of stale
entries.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow tidying revocation queue entries
No manual operator control of revocation queue entries are allowed.
However, entries are stored with their request time, allowing tidy to,
after a suitable safety buffer, remove these unconfirmed and presumably
invalid requests.
Notably, when a cluster goes offline, it will be unable to process
cross-cluster revocations for certificates it holds. If tidy runs,
potentially valid revocations may be removed. However, it is up to the
administrator to ensure the tidy window is sufficiently long that any
required maintenance is done (or, prior to maintenance when an issue is
first noticed, tidy is temporarily disabled).
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Only allow enabling global revocation queue on Vault Enterprise
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Use a locking queue to handle revocation requests
This queue attempts to guarantee that PKI's invalidateFunc won't have
to wait long to execute: by locking only around access to the queue
proper, and internally using a list, we minimize the time spent locked,
waiting for queue accesses.
Previously, we held a lock during tidy and processing that would've
prevented us from processing invalidateFunc calls.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* use_global_queue->cross_cluster_revocation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Grab revocation storage lock when processing queue
We need to grab the storage lock as we'll actively be revoking new
certificates in the revocation queue. This ensures nobody else is
competing for storage access, across periodic funcs, new revocations,
and tidy operations.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix expected tidy status test
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow probing RollbackManager directly in tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback on revocationQueue
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add more cancel checks, fix starting manual tidy
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor CRL building into separate functions
This will allow us to add the ability to add and build a unified CRL
across all clusters, reusing logic that is common to both, but letting
each have their own certificate lists.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename localCRLConfigEntry->internalCRLConfigEntry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Rename Delta WALs to Local Delta WALs
This adds clarity that we'll have a separate local and remote Delta CRL
and WALs for each.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
This PR modifies every test in `builtin/credentials/approle/path_role_test.go` with new validation checks to ensure that approle/path_role successful responses align with the declared response schema.
It also introduces a test helper in `sdk/helper/testhelpers`:
```go
func FindResponseSchema(t *testing.T, ...)
```
This test helper will be useful for all plugins that require similar response schema validation in tests.
### Background
This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
* Rename revokeCert variable to identify serial number formatting
* Refactor out lease specific behavior out of revokeCert
- Isolate the specific behavior regarding revoking lease specific
certificates outside of the revokeCert function and into the only
caller that leveraged used it.
- This allows us to simplify revokeCert a little bit and keeps the
function purely about revoking a certificate
* Within revokeCert short circuit the already revoked use-case
- Make the function a little easier to process by exiting early
if the certificate has already been revoked.
* Do not load certificates from storage multiple times during revocation
- Isolate the loading of a certificate and parsing of a certificate
into a single attempt, either when provided the certificate for BYOC
revocation or strictly from storage for the other revocation types.
* With BYOC write certificate entry using dashes not the legacy colon char
This PR relates to a feature request logged through HashiCorp commercial
support.
Vault lacks pagination in its APIs. As a result, certain list operations
can return **very** large responses. The user's chosen audit sinks may
experience difficulty consuming audit records that swell to tens of
megabytes of JSON.
In our case, one of the systems consuming audit log data could not cope,
and failed.
The responses of list operations are typically not very interesting, as
they are mostly lists of keys, or, even when they include a "key_info"
field, are not returning confidential information. They become even less
interesting once HMAC-ed by the audit system.
Some example Vault "list" operations that are prone to becoming very
large in an active Vault installation are:
auth/token/accessors/
identity/entity/id/
identity/entity-alias/id/
pki/certs/
In response, I've coded a new option that can be applied to audit
backends, `elide_list_responses`. When enabled, response data is elided
from audit logs, only when the operation type is "list".
For added safety, the elision only applies to the "keys" and "key_info"
fields within the response data - these are conventionally the only
fields present in a list response - see logical.ListResponse, and
logical.ListResponseWithInfo. However, other fields are technically
possible if a plugin author writes unusual code, and these will be
preserved in the audit log even with this option enabled.
The elision replaces the values of the "keys" and "key_info" fields with
an integer count of the number of entries. This allows even the elided
audit logs to still be useful for answering questions like "Was any data
returned?" or "How many records were listed?".
* Allow tidy to backup legacy CA bundles
With the new tidy_move_legacy_ca_bundle option, we'll use tidy to move
the legacy CA bundle from /config/ca_bundle to /config/ca_bundle.bak.
This does two things:
1. Removes ca_bundle from the hot-path of initialization after initial
migration has completed. Because this entry is seal wrapped, this
may result in performance improvements.
2. Allows recovery of this value in the event of some other failure
with migration.
Notably, this cannot occur during migration in the unlikely (and largely
unsupported) case that the operator immediately downgrades to Vault
<1.11.x. Thus, we reuse issuer_safety_buffer; while potentially long,
tidy can always be run manually with a shorter buffer (and only this
flag) to manually move the bundle if necessary.
In the event of needing to recover or undo this operation, it is
sufficient to use sys/raw to read the backed up value and subsequently
write it to its old path (/config/ca_bundle).
The new entry remains seal wrapped, but otherwise isn't used within the
code and so has better performance characteristics.
Performing a fat deletion (DELETE /root) will again remove the backup
like the old legacy bundle, preserving its wipe characteristics.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add documentation about new tidy parameter
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for migration scenarios
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clean up time comparisons
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Correctly distinguish empty issuer names
When using client.Logical().JSONMergePatch(...) with an empty issuer
name, patch incorrectly reports:
> issuer name contained invalid characters
In this case, both the error in getIssuerName(...) is incorrect and
patch should allow setting an empty issuer name explicitly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add cluster_aia_path templating variable
Per discussion with maxb, allow using a non-Vault distribution point
which may use an insecure transport for RFC 5280 compliance.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address feedback from Max
Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Max Bowsher <maxbowsher@gmail.com>
Alexander Scheel