* Add a last issued date on ACME accounts
- When we issue a new ACME certificate, attempt to update the account's last issued field
- Within ACME account tidy, use both account creation and last issue date to provide a buffer before we mark the account as revoked.
- Cleanup the cert serial to account tracker
- Misc formatting fixes in JSON objects
* Move account max-cert-expiry updates within tidy
- Perform the account update of max-cert-expiry within
the tidy operation as it has the account write lock
and is already iterating over all orders.
- With this the order path does not need any account
level locks
* Prefix ACME account status constants with AccountStatusX
* Add Vault APIS to create, list, delete ACME EAB keys
- Add Vault authenticated APIs to create, list and delete ACME
EAB keys.
- Add supporting tests for all new apis
* Add require_eab to acme configuration
* Add EAB support to ACME
* Add EAB support to ACME
* PR feedback 1
- Address missing err return within DeleteEab
- Move verifyEabPayload to acme_jws.go no code changes in this PR
- Update error message returned for error on account storage with EAB.
* PR feedback 2
- Verify JWK signature payload after signature verification
* Introduce an ACME eab_policy in configuration
- Instead of a boolean on/off for require_eab, introduce named policies for ACME behaviour enforcing eab.
- The default policy of always-required, will force new accounts to have an EAB, and all operations in the future, will make sure the account has an EAB associated with it.
- Two other policies, not-required will allow any anonymous users to use ACME within PKI and 'new-account-required' will enforce new accounts going forward to require an EAB, but existing accounts will still be allowed to use ACME if they don't have an EAB associated with the account.
- Having 'always-required' as a policy, will override the environment variable to disable public acme as well.
* Add missing go-docs to new tests.
* Add valid eab_policy values in error message.
* import rsa and ecdsa public keys
* allow import_version to update public keys - wip
* allow import_version to update public keys
* move check key fields into func
* put private/public keys in same switch cases
* fix method in UpdateKeyVersion
* move asymmetrics keys switch to its own method - WIP
* test import public and update it with private counterpart
* test import public keys
* use public_key to encrypt if RSAKey is not present and failed to decrypt
if key version does not have a private key
* move key to KeyEntry parsing from Policy to KeyEntry method
* move extracting of key from input fields into helper function
* change back policy Import signature to keep backwards compatibility and
add new method to import private or public keys
* test import with imported public rsa and ecdsa keys
* descriptions and error messages
* error messages, remove comments and unused code
* changelog
* documentation - wip
* suggested changes - error messages/typos and unwrap public key passed
* fix unwrap key error
* fail if both key fields have been set
* fix in extractKeyFromFields, passing a PolicyRequest wouldn't not work
* checks for read, sign and verify endpoints so they don't return errors when a private key was not imported and tests
* handle panic on "export key" endpoint if imported key is public
* fmt
* remove 'isPrivateKey' argument from 'UpdateKeyVersion' and
'parseFromKey' methods
also: rename 'UpdateKeyVersion' method to 'ImportPrivateKeyForVersion' and 'IsPublicKeyImported' to 'IsPrivateKeyMissing'
* delete 'RSAPublicKey' when private key is imported
* path_export: return public_key for ecdsa and rsa when there's no private key imported
* allow signed data validation with pss algorithm
* remove NOTE comment
* fix typo in EC public key export where empty derBytes was being used
* export rsa public key in pkcs8 format instead of pkcs1 and improve test
* change logic on how check for is private key missing is calculated
---------
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* use internal docker mirror for CI
* maybe it needs to be https
* no just kidding it's docker://
* apparently overriding it globally causes creates to fail. time to override each image individually lol
* maybe this works
* Structure of ACME Tidy.
* The tidy endpoints/call information.
* Counts for status plumbing.
* Update typo calls, add information saving date of account creation.
* Missed some field locations.
* Set-up of Tidy command.
* Proper tidy function; lock to work with
* Remove order safety buffer.
* Missed a field.
* Read lock for account creation; Write lock for tidy (account deletion)
* Type issues fixed.
* fix range operator.
* Fix path_tidy read.
* Add fields to auto-tidy config.
* Add (and standardize) Tidy Config Response
* Test pass, consistent fields
* Changes from PR-Reviews.
* Update test to updated default due to PR-Review.
* Refactor setting local addresses
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate wildcard domains in ACME test suite
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add locking to DNS resolver
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Better removal semantics for records
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Export DockerAPI for use by other consumers
As usage of DockerCluster gets more advanced, some users may want to
interact with the container nodes of the cluster. While, if you already
have a DockerAPI instance lying around you can reuse that safely, for
use cases where an existing e.g., docker/testhelpers's runner instance
is not available, reusing the existing cluster's DockerAPI is easiest.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add ability to exec commands without runner
When modifying DockerTestCluster's containers manually, we might not
have a Runner instance; instead, expose the ability to run commands via
a DockerAPI instance directly, as they're awfully convenient.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS resolver into ACME tests
This updates the pkiext_binary tests to use an adjacent DNS resolver,
allowing these tests to eventually be extended to solve DNS challenges,
as modifying the /etc/hosts file does not allow this.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix loading DNS resolver onto network
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix bug with DNS configuration validation
Both conditionals here were inverted: address being empty means a bad
specification was given, and the parse being nil means that it was not a
valid IP address.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix specifying TXT records, allow removing records
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- Do not serialize the entire internal object, instead return
just the Type and Value fields back to the caller.
- Also within authorization responses, return the base domain
on wildcard queries, dropping the *. as the RFC requests.
- Update tests to reflect/test this logic.
- We have a known issue that is difficult to address in released versions of Vault that OCSP GET requests can contain consecutive / characters which the Golang HTTP mux will force a redirection.
- Instead of failing various PRs and runs with this known issue, check to see if we are about to trigger it and if so skip the test. We have already at this point tested the POST version of the API.
- When someone is fetching the order to get it's status, compute if we
need to bump the status to Ready like we do in finalize handler
- Add a wait state to the ACME docker test suite to deal with a race
condition
* Initial refactoring of ACME PKI binary tests
- Rework test suite to use a single Vault cluster with
different mounts.
- Refactor convenience methods to write PKI tests.
* Add ACME test cases for mixed IP and DNS, along with IP only identifier requests
* Parallelize the Vault PKI test suite
* Add missing tidy-status state values
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add docs on auto-tidy reading
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing tidy status field revocation_queue_safety_buffer
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Include pause_duration in tidy-status docs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add date of last auto-tidy operation to status
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add additional existing keys response field
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update tests for validating existing keys
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update docs for import to include new fields
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update website/content/api-docs/secret/pki.mdx
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Validate identifiers against role when creating order
Perform some initial validation against the order's requested
identifiers during creation; this gives a client a heads up that their
request might be rejected by the server before they have to solve
challenges for these, only to find out during CSR submission time that
there is no way to request the specified certificate.
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add unit tests and switch errors to ErrRejectedIdentifier
- Change the error messages from validating identifiers against the
role to ErrRejectedIdentifier errors if they do occur
- Add unit tests to validate that we validate against the various
roles somewhat okay.
* go doc
* Fix typo in test godoc
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add default ACME configuration, invalidate on write
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add enforcment of ACME enabled
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate requested role against ACME config
Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add validation of issuer restrictions with ACME
Co-authored-by: kitography <khaines@mit.edu>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add validation around allowed config lenghts
Co-authored-by: kitography <khaines@mit.edu>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Prune later deemed unnecessary config options
Co-authored-by: kitography <khaines@mit.edu>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* make fmt
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: kitography <khaines@mit.edu>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
* Add PKI ACME IP SANS test case
- Leveraging the new Vault docker based tests along with nginx and
the Go ACME library, verify that Vault can properly perform ACME
validations for IP identifiers
* Formatting
* Handle caching of ACME config
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS resolvers to ACME configuration
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add custom DNS resolver to challenge verification
This required plumbing through the config, reloading it when necessary,
and creating a custom net.Resolver instance.
Not immediately clear is how we'd go about building a custom DNS
validation mechanism that supported multiple resolvers. Likely we'd need
to rely on meikg/dns and handle the resolution separately for each
container and use a custom Dialer that assumes the address is already
pre-resolved.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Improvements to Docker harness
- Expose additional service information, allowing callers to figure out
both the local address and the network-specific address of the
service container, and
- Allow modifying permissions on uploaded container files.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add infrastructure to run Bind9 in a container for tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Validate DNS-01 challenge works
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add the ability to disable ACME through an OS environment variable
- Disable ACME through the VAULT_DISABLE_PUBLIC_ACME environment
variable.
* PR feedback
- Switch to using t.Setenv instead of manually doing it
- Remove t.Parallel from the test not to influence others
* make fmt
* Enforce ACME accounts to a specific directory path
- Accounts and correspondingly orders, authz should not cross
the path boundaries. So we now tag an ACME account with a specific
directory based on the requested role/issuer values in the path.
- If an operation occurs on a different acme directory path it will
cause a failure of the request.
- Add some go doc to a few places and reorder the methods in the
acme_wrappers.go class to highlight the wrappers and not intertwine
the helper functions
- Rename path_acme_new_account.go to path_acme_account.go as it has
several account related methods now.
* Get rid of bad test case
- The previous commit contained a bug fix for us properly
loading issuers within the ACME path, that exposed
this broken/bad test case. Simply remove it.
* Add ACME revocation handlers
This refactors path_revoke to expose Proof of Possession verification,
which is reused by ACME to allow methods 1 and 2:
1. Revocation of a certificate issued by the account, using account
signature as sufficient proof.
2. Revocation of a certificate via proving possession of its private
key, using this private key to create the JWS signature.
We do not support the third mechanism, completing challenges equivalent
to those on the existing certificate and then performing a revocation
under an account which didn't issue the certificate but which did solve
those challenges.
We additionally create another map account->cert->order, allowing us to
quickly look up if a cert was issued by this particular account. Note
that the inverse lookup of cert->(account, order) lookup isn't yet
possible due to Vault's storage structure.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update ACME pkiext tests to revoke certs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add auth handler checks
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Address review feedback
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add enable_aia_url_templating to read issuer
This field was elided from read issuer responses, though the value
otherwise persisted correctly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add comprehensive test for patching issuers
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add missing OpenAPI scheme definition
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Start ACME containerized test suite
This starts a containerized ACME test suite using containers, running
both Vault and Certbot (in standalone mode) in the container to ensure
we successfully issue certificates.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Switch to using hashicorp mirror
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support to load roles and issuers within ACME wrapper
* Add missing go doc to new test
* PR feedback
- Move field definitions into fields.go
- Update wording and associated errors to some role failures.
- Add missing ':' to error messages
* Add additional fields to LIST issuers for Web UI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add DNS challenge validation to ACME
This allows us to validate against wildcard domain names, as the HTTP-01
challenge does not support wildcard validation.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Properly delay challenge retry attempts
Previously we'd essentially semi-busy wait for a challenge to become
retry-able, as the queue itself had no knowledge of the retry after
value of the validation attempt. Now, we plumb through this value into
the queue itself, to aide selection of validations to attempt.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor wildcard validation checks
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add helper to determine if identifier is wildcard
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Better validate wildcard acceptance
This correctly validates wildcards to contain only a leading prefix
(*.) and must include another label, also ensuring that the remaining
domain components pass non-IP and IDNA validation, and removing them
from display on the authorization. Finally, we restrict the challenge
types available for various combinations of IP, DNS, and wildcard
addresses.
This then updates the tests to validate this as well.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* WIP: Implement ACME CSR signing and certificate retrieval
* Add some validations within the ACME finalize API
- Validate that the CSR we were given matches the DNS names
and IP addresses within the order
- Validate that the CSR does not share the same public as the
account
* Gate ACME finalize order validating all authorizations are in valid state
* Add infrastructure for warnings on CRL rebuilds
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add warning on issuer missing KU for CRL Signing
When an entire issuer equivalency class is missing CRL signing usage
(but otherwise has key material present), we should add a warning so
operators can either correct this issuer or create an equivalent version
with KU specified.
Resolves: https://github.com/hashicorp/vault/issues/20137
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for issuer warnings
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix return order of CRL builders
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow creating storageContext with timeout
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add challenge validation engine to ACME
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Initialize the ACME challenge validation engine
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Trigger challenge validation on endpoint submission
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix GetKeyThumbprint to use raw base64
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Point at localhost for testing
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add cleanup of validation engine
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* remove intercepting helpText
* add subtext directly to StringList input component
* update tests and add coverage for new openapi-attrs util
* update test
* add warning validation to input
* lol is this right i dont know go
* literally no idea what im doing
* add Description to display attrs struct
* update struct comment
* add descriptions to remaining go fields
* add missing comma
* remaining commas..."
* add description to display attrs
* update tests
* update tests
* add changelog;
* Update ui/app/utils/openapi-to-attrs.js
* update tests following backend changes
* clearly name variable
* format files
* no longer need to test for modified tooltip since coming from backend now
* Return OCSP errors on cert auth login failures
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Switch to immediately returning the first match
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Minor follow-ups to #16865
Fix PKI issuer upgrade logic when upgrading to 1.12 or later, to
actually turn off the issuer crl-signing usage when it intended to.
Fix minor typo in docs.
* changelog
* Add HTTP challenge validator
This will attempt to safely validate HTTP challenges, following a
limited number of redirects and timing out after too much time has
passed.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test for ValidateKeyAuthorization
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add test cases for ValidateHTTP01Challenge
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add token to HTTP challenge
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove extraneous certificate from OCSP response
Since the issuer used to sign the certificate also signs the OCSP
response, no additional information is added by sending the issuer again
in the certs field of the BasicOCSPResponse structure. Removing it saves
bytes and avoids confusing Go-based OCSP verifiers which cannot handle
the cert issuer being duplicated in the certs field.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
---------
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
- Add a helper function that can accept the final API path along with
the pattern function for an ACME api definition and generate the
various flavors for the given API
* Move all ACME wrappers into a dedicated go file
- Make it easier to figure out where the various wrappers for
ACME exist by locating them inside a dedicated go file instead
of spread out across the various path_acme_xxx files.
* Add missing copyright headers to PKI files
* Implement ACME new-order API
- This is a very rough draft for the new order ACME API
* Add ACME order list API
* Implement ACME Get order API
* Misc order related fixes
- Filter authorizations in GetOrders for valid
- Validate notBefore and notAfter dates make sense
- Add <order>/cert URL path to order response if set to valid
* Return account status within err authorized, if the account key verified
* revert STS lease changes, now create a lease for STS credentials but keep the ttl
Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
