open-vault/builtin/logical/pki/path_fetch_issuers.go

1029 lines
32 KiB
Go
Raw Normal View History

Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
package pki
import (
"context"
Add PSS support to PKI Secrets Engine (#16519) * Add PSS signature support to Vault PKI engine Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use issuer's RevocationSigAlg for CRL signing We introduce a new parameter on issuers, revocation_signature_algorithm to control the signature algorithm used during CRL signing. This is because the SignatureAlgorithm value from the certificate itself is incorrect for this purpose: a RSA root could sign an ECDSA intermediate with say, SHA256WithRSA, but when the intermediate goes to sign a CRL, it must use ECDSAWithSHA256 or equivalent instead of SHA256WithRSA. When coupled with support for PSS-only keys, allowing the user to set the signature algorithm value as desired seems like the best approach. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add use_pss, revocation_signature_algorithm docs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add PSS to signature role issuance test matrix Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow roots to self-identify revocation alg When using PSS support with a managed key, sometimes the underlying device will not support PKCS#1v1.5 signatures. This results in CRL building failing, unless we update the entry's signature algorithm prior to building the CRL for the new root. With a RSA-type key and use_pss=true, we use the signature bits value to decide which hash function to use for PSS support. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add clearer error message on failed import When CRL building fails during cert/key import, due to PSS failures, give a better indication to the user that import succeeded its just CRL building that failed. This tells them the parameter to adjust on the issuer and warns that CRL building will fail until this is fixed. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add case insensitive SigAlgo matching Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Convert UsePSS back to regular bool Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor PSS->certTemplate into helper function Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Proper string output on rev_sig_alg display Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Copy root's SignatureAlgorithm for CRL building Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-03 16:42:24 +00:00
"crypto/x509"
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
"encoding/pem"
"fmt"
"strings"
Allow marking issuers as revoked (#16621) * Allow marking issuers as revoked This allows PKI's issuers to be considered revoked and appear on each others' CRLs. We disable issuance (via removing the usage) and prohibit modifying the usage via the regular issuer management interface. A separate endpoint is necessary because issuers (especially if signed by a third-party CA using incremental serial numbers) might share a serial number (e.g., an intermediate under cross-signing might share the same number as an external root or an unrelated intermediate). When the next CRL rebuild happens, this issuer will then appear on others issuers CRLs, if they validate this issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on revoking issuers Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for issuer revocation semantics Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Notate that CRLs will be rebuilt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix timestamp field from _utc -> to _rfc3339 Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Ensure serial-based accesses shows as revoked Thanks Kit! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning when revoking default issuer Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-18 22:08:31 +00:00
"time"
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
"github.com/hashicorp/vault/sdk/framework"
"github.com/hashicorp/vault/sdk/helper/certutil"
"github.com/hashicorp/vault/sdk/logical"
)
func pathListIssuers(b *backend) *framework.Path {
return &framework.Path{
Pattern: "issuers/?$",
Operations: map[logical.Operation]framework.OperationHandler{
logical.ListOperation: &framework.PathOperation{
Callback: b.pathListIssuersHandler,
},
},
HelpSynopsis: pathListIssuersHelpSyn,
HelpDescription: pathListIssuersHelpDesc,
}
}
func (b *backend) pathListIssuersHandler(ctx context.Context, req *logical.Request, _ *framework.FieldData) (*logical.Response, error) {
if b.useLegacyBundleCaStorage() {
return logical.ErrorResponse("Can not list issuers until migration has completed"), nil
}
var responseKeys []string
responseInfo := make(map[string]interface{})
sc := b.makeStorageContext(ctx, req.Storage)
entries, err := sc.listIssuers()
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
config, err := sc.getIssuersConfig()
if err != nil {
return nil, err
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
// For each issuer, we need not only the identifier (as returned by
// listIssuers), but also the name of the issuer. This means we have to
// fetch the actual issuer object as well.
for _, identifier := range entries {
issuer, err := sc.fetchIssuerById(identifier)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
responseKeys = append(responseKeys, string(identifier))
responseInfo[string(identifier)] = map[string]interface{}{
"issuer_name": issuer.Name,
"is_default": identifier == config.DefaultIssuerId,
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
}
}
return logical.ListResponseWithInfo(responseKeys, responseInfo), nil
}
const (
pathListIssuersHelpSyn = `Fetch a list of CA certificates.`
pathListIssuersHelpDesc = `
This endpoint allows listing of known issuing certificates, returning
their identifier and their name (if set).
`
)
func pathGetIssuer(b *backend) *framework.Path {
pattern := "issuer/" + framework.GenericNameRegex(issuerRefParam) + "(/der|/pem|/json)?"
return buildPathGetIssuer(b, pattern)
}
func buildPathGetIssuer(b *backend, pattern string) *framework.Path {
fields := map[string]*framework.FieldSchema{}
fields = addIssuerRefNameFields(fields)
// Fields for updating issuer.
fields["manual_chain"] = &framework.FieldSchema{
Type: framework.TypeCommaStringSlice,
Description: `Chain of issuer references to use to build this
issuer's computed CAChain field, when non-empty.`,
}
fields["leaf_not_after_behavior"] = &framework.FieldSchema{
Type: framework.TypeString,
Description: `Behavior of leaf's NotAfter fields: "err" to error
if the computed NotAfter date exceeds that of this issuer; "truncate" to
silently truncate to that of this issuer; or "permit" to allow this
issuance to succeed (with NotAfter exceeding that of an issuer). Note that
not all values will results in certificates that can be validated through
the entire validity period. It is suggested to use "truncate" for
intermediate CAs and "permit" only for root CAs.`,
Default: "err",
}
fields["usage"] = &framework.FieldSchema{
Type: framework.TypeCommaStringSlice,
Description: `Comma-separated list (or string slice) of usages for
this issuer; valid values are "read-only", "issuing-certificates",
"crl-signing", and "ocsp-signing". Multiple values may be specified. Read-only
is implicit and always set.`,
Default: []string{"read-only", "issuing-certificates", "crl-signing", "ocsp-signing"},
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
}
Add PSS support to PKI Secrets Engine (#16519) * Add PSS signature support to Vault PKI engine Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use issuer's RevocationSigAlg for CRL signing We introduce a new parameter on issuers, revocation_signature_algorithm to control the signature algorithm used during CRL signing. This is because the SignatureAlgorithm value from the certificate itself is incorrect for this purpose: a RSA root could sign an ECDSA intermediate with say, SHA256WithRSA, but when the intermediate goes to sign a CRL, it must use ECDSAWithSHA256 or equivalent instead of SHA256WithRSA. When coupled with support for PSS-only keys, allowing the user to set the signature algorithm value as desired seems like the best approach. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add use_pss, revocation_signature_algorithm docs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add PSS to signature role issuance test matrix Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow roots to self-identify revocation alg When using PSS support with a managed key, sometimes the underlying device will not support PKCS#1v1.5 signatures. This results in CRL building failing, unless we update the entry's signature algorithm prior to building the CRL for the new root. With a RSA-type key and use_pss=true, we use the signature bits value to decide which hash function to use for PSS support. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add clearer error message on failed import When CRL building fails during cert/key import, due to PSS failures, give a better indication to the user that import succeeded its just CRL building that failed. This tells them the parameter to adjust on the issuer and warns that CRL building will fail until this is fixed. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add case insensitive SigAlgo matching Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Convert UsePSS back to regular bool Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor PSS->certTemplate into helper function Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Proper string output on rev_sig_alg display Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Copy root's SignatureAlgorithm for CRL building Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-03 16:42:24 +00:00
fields["revocation_signature_algorithm"] = &framework.FieldSchema{
Type: framework.TypeString,
Description: `Which x509.SignatureAlgorithm name to use for
signing CRLs. This parameter allows differentiation between PKCS#1v1.5
and PSS keys and choice of signature hash algorithm. The default (empty
string) value is for Go to select the signature algorithm. This can fail
if the underlying key does not support the requested signature algorithm,
which may not be known at modification time (such as with PKCS#11 managed
RSA keys).`,
Default: "",
}
Add per-issuer AIA URI information to PKI secrets engine (#16563) * Add per-issuer AIA URI information Per discussion on GitHub with @maxb, this allows issuers to have their own copy of AIA URIs. Because each issuer has its own URLs (for CA and CRL access), its necessary to mint their issued certs pointing to the correct issuer and not to the global default issuer. For anyone using multiple issuers within a mount, this change allows the issuer to point back to itself via leaf's AIA info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on per-issuer AIA info Also add it to the considerations page as something to watch out for. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for per-issuer AIA information Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor AIA setting on the issuer This introduces a common helper per Steve's suggestion. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages w.r.t. AIA naming Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages regarding AIA URLs This clarifies which request parameter the invalid URL is contained in, disambiguating the sometimes ambiguous usage of AIA, per suggestion by Max. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename getURLs -> getGlobalAIAURLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correct AIA acronym expansion word orders Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix bad comment suggesting re-generating roots Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add two entries to URL tests Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-19 15:43:44 +00:00
fields["issuing_certificates"] = &framework.FieldSchema{
Type: framework.TypeCommaStringSlice,
Description: `Comma-separated list of URLs to be used
for the issuing certificate attribute. See also RFC 5280 Section 4.2.2.1.`,
}
fields["crl_distribution_points"] = &framework.FieldSchema{
Type: framework.TypeCommaStringSlice,
Description: `Comma-separated list of URLs to be used
for the CRL distribution points attribute. See also RFC 5280 Section 4.2.1.13.`,
}
fields["ocsp_servers"] = &framework.FieldSchema{
Type: framework.TypeCommaStringSlice,
Description: `Comma-separated list of URLs to be used
for the OCSP servers attribute. See also RFC 5280 Section 4.2.2.1.`,
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
return &framework.Path{
// Returns a JSON entry.
Pattern: pattern,
Fields: fields,
Operations: map[logical.Operation]framework.OperationHandler{
logical.ReadOperation: &framework.PathOperation{
Callback: b.pathGetIssuer,
},
logical.UpdateOperation: &framework.PathOperation{
Callback: b.pathUpdateIssuer,
// Read more about why these flags are set in backend.go.
ForwardPerformanceStandby: true,
ForwardPerformanceSecondary: true,
},
logical.DeleteOperation: &framework.PathOperation{
Callback: b.pathDeleteIssuer,
// Read more about why these flags are set in backend.go.
ForwardPerformanceStandby: true,
ForwardPerformanceSecondary: true,
},
logical.PatchOperation: &framework.PathOperation{
Callback: b.pathPatchIssuer,
// Read more about why these flags are set in backend.go.
ForwardPerformanceStandby: true,
ForwardPerformanceSecondary: true,
},
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
},
HelpSynopsis: pathGetIssuerHelpSyn,
HelpDescription: pathGetIssuerHelpDesc,
}
}
func (b *backend) pathGetIssuer(ctx context.Context, req *logical.Request, data *framework.FieldData) (*logical.Response, error) {
// Handle raw issuers first.
if strings.HasSuffix(req.Path, "/der") || strings.HasSuffix(req.Path, "/pem") || strings.HasSuffix(req.Path, "/json") {
return b.pathGetRawIssuer(ctx, req, data)
}
if b.useLegacyBundleCaStorage() {
return logical.ErrorResponse("Can not get issuer until migration has completed"), nil
}
issuerName := getIssuerRef(data)
if len(issuerName) == 0 {
return logical.ErrorResponse("missing issuer reference"), nil
}
sc := b.makeStorageContext(ctx, req.Storage)
ref, err := sc.resolveIssuerReference(issuerName)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
if ref == "" {
return logical.ErrorResponse("unable to resolve issuer id for reference: " + issuerName), nil
}
issuer, err := sc.fetchIssuerById(ref)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
return respondReadIssuer(issuer)
}
func respondReadIssuer(issuer *issuerEntry) (*logical.Response, error) {
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
var respManualChain []string
for _, entity := range issuer.ManualChain {
respManualChain = append(respManualChain, string(entity))
}
Add PSS support to PKI Secrets Engine (#16519) * Add PSS signature support to Vault PKI engine Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use issuer's RevocationSigAlg for CRL signing We introduce a new parameter on issuers, revocation_signature_algorithm to control the signature algorithm used during CRL signing. This is because the SignatureAlgorithm value from the certificate itself is incorrect for this purpose: a RSA root could sign an ECDSA intermediate with say, SHA256WithRSA, but when the intermediate goes to sign a CRL, it must use ECDSAWithSHA256 or equivalent instead of SHA256WithRSA. When coupled with support for PSS-only keys, allowing the user to set the signature algorithm value as desired seems like the best approach. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add use_pss, revocation_signature_algorithm docs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add PSS to signature role issuance test matrix Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow roots to self-identify revocation alg When using PSS support with a managed key, sometimes the underlying device will not support PKCS#1v1.5 signatures. This results in CRL building failing, unless we update the entry's signature algorithm prior to building the CRL for the new root. With a RSA-type key and use_pss=true, we use the signature bits value to decide which hash function to use for PSS support. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add clearer error message on failed import When CRL building fails during cert/key import, due to PSS failures, give a better indication to the user that import succeeded its just CRL building that failed. This tells them the parameter to adjust on the issuer and warns that CRL building will fail until this is fixed. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add case insensitive SigAlgo matching Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Convert UsePSS back to regular bool Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor PSS->certTemplate into helper function Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Proper string output on rev_sig_alg display Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Copy root's SignatureAlgorithm for CRL building Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-03 16:42:24 +00:00
revSigAlgStr := issuer.RevocationSigAlg.String()
if issuer.RevocationSigAlg == x509.UnknownSignatureAlgorithm {
revSigAlgStr = ""
}
Allow marking issuers as revoked (#16621) * Allow marking issuers as revoked This allows PKI's issuers to be considered revoked and appear on each others' CRLs. We disable issuance (via removing the usage) and prohibit modifying the usage via the regular issuer management interface. A separate endpoint is necessary because issuers (especially if signed by a third-party CA using incremental serial numbers) might share a serial number (e.g., an intermediate under cross-signing might share the same number as an external root or an unrelated intermediate). When the next CRL rebuild happens, this issuer will then appear on others issuers CRLs, if they validate this issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on revoking issuers Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for issuer revocation semantics Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Notate that CRLs will be rebuilt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix timestamp field from _utc -> to _rfc3339 Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Ensure serial-based accesses shows as revoked Thanks Kit! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning when revoking default issuer Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-18 22:08:31 +00:00
data := map[string]interface{}{
"issuer_id": issuer.ID,
"issuer_name": issuer.Name,
"key_id": issuer.KeyID,
"certificate": issuer.Certificate,
"manual_chain": respManualChain,
"ca_chain": issuer.CAChain,
"leaf_not_after_behavior": issuer.LeafNotAfterBehavior.String(),
"usage": issuer.Usage.Names(),
"revocation_signature_algorithm": revSigAlgStr,
"revoked": issuer.Revoked,
Add per-issuer AIA URI information to PKI secrets engine (#16563) * Add per-issuer AIA URI information Per discussion on GitHub with @maxb, this allows issuers to have their own copy of AIA URIs. Because each issuer has its own URLs (for CA and CRL access), its necessary to mint their issued certs pointing to the correct issuer and not to the global default issuer. For anyone using multiple issuers within a mount, this change allows the issuer to point back to itself via leaf's AIA info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on per-issuer AIA info Also add it to the considerations page as something to watch out for. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for per-issuer AIA information Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor AIA setting on the issuer This introduces a common helper per Steve's suggestion. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages w.r.t. AIA naming Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages regarding AIA URLs This clarifies which request parameter the invalid URL is contained in, disambiguating the sometimes ambiguous usage of AIA, per suggestion by Max. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename getURLs -> getGlobalAIAURLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correct AIA acronym expansion word orders Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix bad comment suggesting re-generating roots Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add two entries to URL tests Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-19 15:43:44 +00:00
"issuing_certificates": []string{},
"crl_distribution_points": []string{},
"ocsp_servers": []string{},
Allow marking issuers as revoked (#16621) * Allow marking issuers as revoked This allows PKI's issuers to be considered revoked and appear on each others' CRLs. We disable issuance (via removing the usage) and prohibit modifying the usage via the regular issuer management interface. A separate endpoint is necessary because issuers (especially if signed by a third-party CA using incremental serial numbers) might share a serial number (e.g., an intermediate under cross-signing might share the same number as an external root or an unrelated intermediate). When the next CRL rebuild happens, this issuer will then appear on others issuers CRLs, if they validate this issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on revoking issuers Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for issuer revocation semantics Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Notate that CRLs will be rebuilt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix timestamp field from _utc -> to _rfc3339 Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Ensure serial-based accesses shows as revoked Thanks Kit! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning when revoking default issuer Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-18 22:08:31 +00:00
}
if issuer.Revoked {
data["revocation_time"] = issuer.RevocationTime
data["revocation_time_rfc3339"] = issuer.RevocationTimeUTC.Format(time.RFC3339Nano)
}
Add per-issuer AIA URI information to PKI secrets engine (#16563) * Add per-issuer AIA URI information Per discussion on GitHub with @maxb, this allows issuers to have their own copy of AIA URIs. Because each issuer has its own URLs (for CA and CRL access), its necessary to mint their issued certs pointing to the correct issuer and not to the global default issuer. For anyone using multiple issuers within a mount, this change allows the issuer to point back to itself via leaf's AIA info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on per-issuer AIA info Also add it to the considerations page as something to watch out for. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for per-issuer AIA information Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor AIA setting on the issuer This introduces a common helper per Steve's suggestion. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages w.r.t. AIA naming Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages regarding AIA URLs This clarifies which request parameter the invalid URL is contained in, disambiguating the sometimes ambiguous usage of AIA, per suggestion by Max. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename getURLs -> getGlobalAIAURLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correct AIA acronym expansion word orders Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix bad comment suggesting re-generating roots Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add two entries to URL tests Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-19 15:43:44 +00:00
if issuer.AIAURIs != nil {
data["issuing_certificates"] = issuer.AIAURIs.IssuingCertificates
data["crl_distribution_points"] = issuer.AIAURIs.CRLDistributionPoints
data["ocsp_servers"] = issuer.AIAURIs.OCSPServers
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
return &logical.Response{
Allow marking issuers as revoked (#16621) * Allow marking issuers as revoked This allows PKI's issuers to be considered revoked and appear on each others' CRLs. We disable issuance (via removing the usage) and prohibit modifying the usage via the regular issuer management interface. A separate endpoint is necessary because issuers (especially if signed by a third-party CA using incremental serial numbers) might share a serial number (e.g., an intermediate under cross-signing might share the same number as an external root or an unrelated intermediate). When the next CRL rebuild happens, this issuer will then appear on others issuers CRLs, if they validate this issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on revoking issuers Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for issuer revocation semantics Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Notate that CRLs will be rebuilt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix timestamp field from _utc -> to _rfc3339 Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Ensure serial-based accesses shows as revoked Thanks Kit! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning when revoking default issuer Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-18 22:08:31 +00:00
Data: data,
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
}, nil
}
func (b *backend) pathUpdateIssuer(ctx context.Context, req *logical.Request, data *framework.FieldData) (*logical.Response, error) {
// Since we're planning on updating issuers here, grab the lock so we've
// got a consistent view.
b.issuersLock.Lock()
defer b.issuersLock.Unlock()
if b.useLegacyBundleCaStorage() {
return logical.ErrorResponse("Can not update issuer until migration has completed"), nil
}
issuerName := getIssuerRef(data)
if len(issuerName) == 0 {
return logical.ErrorResponse("missing issuer reference"), nil
}
sc := b.makeStorageContext(ctx, req.Storage)
ref, err := sc.resolveIssuerReference(issuerName)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
if ref == "" {
return logical.ErrorResponse("unable to resolve issuer id for reference: " + issuerName), nil
}
issuer, err := sc.fetchIssuerById(ref)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
newName, err := getIssuerName(sc, data)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil && err != errIssuerNameInUse {
// If the error is name already in use, and the new name is the
// old name for this issuer, we're not actually updating the
// issuer name (or causing a conflict) -- so don't err out. Other
// errs should still be surfaced, however.
return logical.ErrorResponse(err.Error()), nil
}
if err == errIssuerNameInUse && issuer.Name != newName {
// When the new name is in use but isn't this name, throw an error.
return logical.ErrorResponse(err.Error()), nil
}
if len(newName) > 0 && !nameMatcher.MatchString(newName) {
return logical.ErrorResponse("new key name outside of valid character limits"), nil
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
newPath := data.Get("manual_chain").([]string)
rawLeafBehavior := data.Get("leaf_not_after_behavior").(string)
var newLeafBehavior certutil.NotAfterBehavior
switch rawLeafBehavior {
case "err":
newLeafBehavior = certutil.ErrNotAfterBehavior
case "truncate":
newLeafBehavior = certutil.TruncateNotAfterBehavior
case "permit":
newLeafBehavior = certutil.PermitNotAfterBehavior
default:
return logical.ErrorResponse("Unknown value for field `leaf_not_after_behavior`. Possible values are `err`, `truncate`, and `permit`."), nil
}
rawUsage := data.Get("usage").([]string)
newUsage, err := NewIssuerUsageFromNames(rawUsage)
if err != nil {
return logical.ErrorResponse(fmt.Sprintf("Unable to parse specified usages: %v - valid values are %v", rawUsage, AllIssuerUsages.Names())), nil
}
Add PSS support to PKI Secrets Engine (#16519) * Add PSS signature support to Vault PKI engine Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use issuer's RevocationSigAlg for CRL signing We introduce a new parameter on issuers, revocation_signature_algorithm to control the signature algorithm used during CRL signing. This is because the SignatureAlgorithm value from the certificate itself is incorrect for this purpose: a RSA root could sign an ECDSA intermediate with say, SHA256WithRSA, but when the intermediate goes to sign a CRL, it must use ECDSAWithSHA256 or equivalent instead of SHA256WithRSA. When coupled with support for PSS-only keys, allowing the user to set the signature algorithm value as desired seems like the best approach. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add use_pss, revocation_signature_algorithm docs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add PSS to signature role issuance test matrix Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow roots to self-identify revocation alg When using PSS support with a managed key, sometimes the underlying device will not support PKCS#1v1.5 signatures. This results in CRL building failing, unless we update the entry's signature algorithm prior to building the CRL for the new root. With a RSA-type key and use_pss=true, we use the signature bits value to decide which hash function to use for PSS support. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add clearer error message on failed import When CRL building fails during cert/key import, due to PSS failures, give a better indication to the user that import succeeded its just CRL building that failed. This tells them the parameter to adjust on the issuer and warns that CRL building will fail until this is fixed. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add case insensitive SigAlgo matching Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Convert UsePSS back to regular bool Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor PSS->certTemplate into helper function Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Proper string output on rev_sig_alg display Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Copy root's SignatureAlgorithm for CRL building Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-03 16:42:24 +00:00
// Revocation signature algorithm changes
revSigAlgStr := data.Get("revocation_signature_algorithm").(string)
revSigAlg, present := certutil.SignatureAlgorithmNames[strings.ToLower(revSigAlgStr)]
if !present && revSigAlgStr != "" {
var knownAlgos []string
for algoName := range certutil.SignatureAlgorithmNames {
knownAlgos = append(knownAlgos, algoName)
}
return logical.ErrorResponse(fmt.Sprintf("Unknown signature algorithm value: %v - valid values are %v", revSigAlg, strings.Join(knownAlgos, ", "))), nil
} else if revSigAlgStr == "" {
revSigAlg = x509.UnknownSignatureAlgorithm
}
if err := issuer.CanMaybeSignWithAlgo(revSigAlg); err != nil {
return nil, err
}
Add per-issuer AIA URI information to PKI secrets engine (#16563) * Add per-issuer AIA URI information Per discussion on GitHub with @maxb, this allows issuers to have their own copy of AIA URIs. Because each issuer has its own URLs (for CA and CRL access), its necessary to mint their issued certs pointing to the correct issuer and not to the global default issuer. For anyone using multiple issuers within a mount, this change allows the issuer to point back to itself via leaf's AIA info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on per-issuer AIA info Also add it to the considerations page as something to watch out for. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for per-issuer AIA information Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor AIA setting on the issuer This introduces a common helper per Steve's suggestion. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages w.r.t. AIA naming Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages regarding AIA URLs This clarifies which request parameter the invalid URL is contained in, disambiguating the sometimes ambiguous usage of AIA, per suggestion by Max. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename getURLs -> getGlobalAIAURLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correct AIA acronym expansion word orders Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix bad comment suggesting re-generating roots Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add two entries to URL tests Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-19 15:43:44 +00:00
// AIA access changes
issuerCertificates := data.Get("issuing_certificates").([]string)
if badURL := validateURLs(issuerCertificates); badURL != "" {
return logical.ErrorResponse(fmt.Sprintf("invalid URL found in Authority Information Access (AIA) parameter issuing_certificates: %s", badURL)), nil
}
crlDistributionPoints := data.Get("crl_distribution_points").([]string)
if badURL := validateURLs(crlDistributionPoints); badURL != "" {
return logical.ErrorResponse(fmt.Sprintf("invalid URL found in Authority Information Access (AIA) parameter crl_distribution_points: %s", badURL)), nil
}
ocspServers := data.Get("ocsp_servers").([]string)
if badURL := validateURLs(ocspServers); badURL != "" {
return logical.ErrorResponse(fmt.Sprintf("invalid URL found in Authority Information Access (AIA) parameter ocsp_servers: %s", badURL)), nil
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
modified := false
var oldName string
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if newName != issuer.Name {
oldName = issuer.Name
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
issuer.Name = newName
PKI - Honor header If-Modified-Since if present (#16249) * honor header if-modified-since if present * pathGetIssuerCRL first version * check if modified since for CA endpoints * fix date comparison for CA endpoints * suggested changes and refactoring * add writeIssuer to updateDefaultIssuerId and fix error * Move methods out of storage.go into util.go For the most part, these take a SC as param, but aren't directly storage relevant operations. Move them out of storage.go as a result. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use UTC timezone for storage Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rework path_fetch for better if-modified-since handling Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Invalidate all issuers, CRLs on default write When the default is updated, access under earlier timestamps will not work as we're unclear if the timestamp is for this issuer or a previous issuer. Thus, we need to invalidate the CRL and both issuers involved (previous, next) by updating their LastModifiedTimes. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for If-Modified-Since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invalidate default issuer changes When the default issuer changes, we'll have to mark the invalidation on PR secondary clusters, so they know to update their CRL mapping as well. The swapped issuers will have an updated modification time (which will eventually replicate down and thus be correct), but the CRL modification time is cluster-local information and thus won't be replicated. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * make fmt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor sendNotModifiedResponseIfNecessary Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on if-modified-since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 19:28:47 +00:00
issuer.LastModified = time.Now().UTC()
// See note in updateDefaultIssuerId about why this is necessary.
b.crlBuilder.invalidateCRLBuildTime()
b.crlBuilder.flushCRLBuildTimeInvalidation(sc)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
modified = true
}
if newLeafBehavior != issuer.LeafNotAfterBehavior {
issuer.LeafNotAfterBehavior = newLeafBehavior
modified = true
}
if newUsage != issuer.Usage {
Allow marking issuers as revoked (#16621) * Allow marking issuers as revoked This allows PKI's issuers to be considered revoked and appear on each others' CRLs. We disable issuance (via removing the usage) and prohibit modifying the usage via the regular issuer management interface. A separate endpoint is necessary because issuers (especially if signed by a third-party CA using incremental serial numbers) might share a serial number (e.g., an intermediate under cross-signing might share the same number as an external root or an unrelated intermediate). When the next CRL rebuild happens, this issuer will then appear on others issuers CRLs, if they validate this issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on revoking issuers Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for issuer revocation semantics Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Notate that CRLs will be rebuilt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix timestamp field from _utc -> to _rfc3339 Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Ensure serial-based accesses shows as revoked Thanks Kit! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning when revoking default issuer Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-18 22:08:31 +00:00
if issuer.Revoked && newUsage.HasUsage(IssuanceUsage) {
// Forbid allowing cert signing on its usage.
return logical.ErrorResponse("This issuer was revoked; unable to modify its usage to include certificate signing again. Reissue this certificate (preferably with a new key) and modify that entry instead."), nil
Allow marking issuers as revoked (#16621) * Allow marking issuers as revoked This allows PKI's issuers to be considered revoked and appear on each others' CRLs. We disable issuance (via removing the usage) and prohibit modifying the usage via the regular issuer management interface. A separate endpoint is necessary because issuers (especially if signed by a third-party CA using incremental serial numbers) might share a serial number (e.g., an intermediate under cross-signing might share the same number as an external root or an unrelated intermediate). When the next CRL rebuild happens, this issuer will then appear on others issuers CRLs, if they validate this issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on revoking issuers Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for issuer revocation semantics Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Notate that CRLs will be rebuilt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix timestamp field from _utc -> to _rfc3339 Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Ensure serial-based accesses shows as revoked Thanks Kit! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning when revoking default issuer Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-18 22:08:31 +00:00
}
// Ensure we deny adding CRL usage if the bits are missing from the
// cert itself.
cert, err := issuer.GetCertificate()
if err != nil {
return nil, fmt.Errorf("unable to parse issuer's certificate: %v", err)
}
if (cert.KeyUsage&x509.KeyUsageCRLSign) == 0 && newUsage.HasUsage(CRLSigningUsage) {
return logical.ErrorResponse("This issuer's underlying certificate lacks the CRLSign KeyUsage value; unable to set CRLSigningUsage on this issuer as a result."), nil
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
issuer.Usage = newUsage
modified = true
}
Add PSS support to PKI Secrets Engine (#16519) * Add PSS signature support to Vault PKI engine Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use issuer's RevocationSigAlg for CRL signing We introduce a new parameter on issuers, revocation_signature_algorithm to control the signature algorithm used during CRL signing. This is because the SignatureAlgorithm value from the certificate itself is incorrect for this purpose: a RSA root could sign an ECDSA intermediate with say, SHA256WithRSA, but when the intermediate goes to sign a CRL, it must use ECDSAWithSHA256 or equivalent instead of SHA256WithRSA. When coupled with support for PSS-only keys, allowing the user to set the signature algorithm value as desired seems like the best approach. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add use_pss, revocation_signature_algorithm docs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add PSS to signature role issuance test matrix Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow roots to self-identify revocation alg When using PSS support with a managed key, sometimes the underlying device will not support PKCS#1v1.5 signatures. This results in CRL building failing, unless we update the entry's signature algorithm prior to building the CRL for the new root. With a RSA-type key and use_pss=true, we use the signature bits value to decide which hash function to use for PSS support. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add clearer error message on failed import When CRL building fails during cert/key import, due to PSS failures, give a better indication to the user that import succeeded its just CRL building that failed. This tells them the parameter to adjust on the issuer and warns that CRL building will fail until this is fixed. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add case insensitive SigAlgo matching Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Convert UsePSS back to regular bool Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor PSS->certTemplate into helper function Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Proper string output on rev_sig_alg display Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Copy root's SignatureAlgorithm for CRL building Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-03 16:42:24 +00:00
if revSigAlg != issuer.RevocationSigAlg {
issuer.RevocationSigAlg = revSigAlg
modified = true
}
Add per-issuer AIA URI information to PKI secrets engine (#16563) * Add per-issuer AIA URI information Per discussion on GitHub with @maxb, this allows issuers to have their own copy of AIA URIs. Because each issuer has its own URLs (for CA and CRL access), its necessary to mint their issued certs pointing to the correct issuer and not to the global default issuer. For anyone using multiple issuers within a mount, this change allows the issuer to point back to itself via leaf's AIA info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on per-issuer AIA info Also add it to the considerations page as something to watch out for. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for per-issuer AIA information Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor AIA setting on the issuer This introduces a common helper per Steve's suggestion. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages w.r.t. AIA naming Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages regarding AIA URLs This clarifies which request parameter the invalid URL is contained in, disambiguating the sometimes ambiguous usage of AIA, per suggestion by Max. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename getURLs -> getGlobalAIAURLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correct AIA acronym expansion word orders Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix bad comment suggesting re-generating roots Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add two entries to URL tests Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-19 15:43:44 +00:00
if issuer.AIAURIs == nil && (len(issuerCertificates) > 0 || len(crlDistributionPoints) > 0 || len(ocspServers) > 0) {
issuer.AIAURIs = &certutil.URLEntries{}
}
if issuer.AIAURIs != nil {
// Associative mapping from data source to destination on the
// backing issuer object.
type aiaPair struct {
Source *[]string
Dest *[]string
}
pairs := []aiaPair{
{
Source: &issuerCertificates,
Dest: &issuer.AIAURIs.IssuingCertificates,
},
{
Source: &crlDistributionPoints,
Dest: &issuer.AIAURIs.CRLDistributionPoints,
},
{
Source: &ocspServers,
Dest: &issuer.AIAURIs.OCSPServers,
},
}
// For each pair, if it is different on the object, update it.
for _, pair := range pairs {
if isStringArrayDifferent(*pair.Source, *pair.Dest) {
*pair.Dest = *pair.Source
modified = true
}
}
// If no AIA URLs exist on the issuer, set the AIA URLs entry to nil
// to ease usage later.
if len(issuer.AIAURIs.IssuingCertificates) == 0 && len(issuer.AIAURIs.CRLDistributionPoints) == 0 && len(issuer.AIAURIs.OCSPServers) == 0 {
issuer.AIAURIs = nil
}
}
// Updating the chain should be the last modification as there's a chance
// it'll write it out to disk for us. We'd hate to then modify the issuer
// again and write it a second time.
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
var updateChain bool
var constructedChain []issuerID
for index, newPathRef := range newPath {
// Allow self for the first entry.
if index == 0 && newPathRef == "self" {
newPathRef = string(ref)
}
resolvedId, err := sc.resolveIssuerReference(newPathRef)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
if index == 0 && resolvedId != ref {
return logical.ErrorResponse(fmt.Sprintf("expected first cert in chain to be a self-reference, but was: %v/%v", newPathRef, resolvedId)), nil
}
constructedChain = append(constructedChain, resolvedId)
if len(issuer.ManualChain) < len(constructedChain) || constructedChain[index] != issuer.ManualChain[index] {
updateChain = true
}
}
if len(issuer.ManualChain) != len(constructedChain) {
updateChain = true
}
if updateChain {
issuer.ManualChain = constructedChain
// Building the chain will write the issuer to disk; no need to do it
// twice.
modified = false
err := sc.rebuildIssuersChains(issuer)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
}
if modified {
err := sc.writeIssuer(issuer)
if err != nil {
return nil, err
}
}
response, err := respondReadIssuer(issuer)
if newName != oldName {
addWarningOnDereferencing(sc, oldName, response)
}
return response, err
}
func (b *backend) pathPatchIssuer(ctx context.Context, req *logical.Request, data *framework.FieldData) (*logical.Response, error) {
// Since we're planning on updating issuers here, grab the lock so we've
// got a consistent view.
b.issuersLock.Lock()
defer b.issuersLock.Unlock()
if b.useLegacyBundleCaStorage() {
return logical.ErrorResponse("Can not patch issuer until migration has completed"), nil
}
// First we fetch the issuer
issuerName := getIssuerRef(data)
if len(issuerName) == 0 {
return logical.ErrorResponse("missing issuer reference"), nil
}
sc := b.makeStorageContext(ctx, req.Storage)
ref, err := sc.resolveIssuerReference(issuerName)
if err != nil {
return nil, err
}
if ref == "" {
return logical.ErrorResponse("unable to resolve issuer id for reference: " + issuerName), nil
}
issuer, err := sc.fetchIssuerById(ref)
if err != nil {
return nil, err
}
// Now We are Looking at What (Might) Have Changed
modified := false
// Name Changes First
_, ok := data.GetOk("issuer_name") // Don't check for conflicts if we aren't updating the name
var oldName string
var newName string
if ok {
newName, err = getIssuerName(sc, data)
if err != nil && err != errIssuerNameInUse {
// If the error is name already in use, and the new name is the
// old name for this issuer, we're not actually updating the
// issuer name (or causing a conflict) -- so don't err out. Other
// errs should still be surfaced, however.
return logical.ErrorResponse(err.Error()), nil
}
if err == errIssuerNameInUse && issuer.Name != newName {
// When the new name is in use but isn't this name, throw an error.
return logical.ErrorResponse(err.Error()), nil
}
if len(newName) > 0 && !nameMatcher.MatchString(newName) {
return logical.ErrorResponse("new key name outside of valid character limits"), nil
}
if newName != issuer.Name {
oldName = issuer.Name
issuer.Name = newName
PKI - Honor header If-Modified-Since if present (#16249) * honor header if-modified-since if present * pathGetIssuerCRL first version * check if modified since for CA endpoints * fix date comparison for CA endpoints * suggested changes and refactoring * add writeIssuer to updateDefaultIssuerId and fix error * Move methods out of storage.go into util.go For the most part, these take a SC as param, but aren't directly storage relevant operations. Move them out of storage.go as a result. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use UTC timezone for storage Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rework path_fetch for better if-modified-since handling Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Invalidate all issuers, CRLs on default write When the default is updated, access under earlier timestamps will not work as we're unclear if the timestamp is for this issuer or a previous issuer. Thus, we need to invalidate the CRL and both issuers involved (previous, next) by updating their LastModifiedTimes. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for If-Modified-Since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invalidate default issuer changes When the default issuer changes, we'll have to mark the invalidation on PR secondary clusters, so they know to update their CRL mapping as well. The swapped issuers will have an updated modification time (which will eventually replicate down and thus be correct), but the CRL modification time is cluster-local information and thus won't be replicated. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * make fmt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor sendNotModifiedResponseIfNecessary Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on if-modified-since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 19:28:47 +00:00
issuer.LastModified = time.Now().UTC()
// See note in updateDefaultIssuerId about why this is necessary.
b.crlBuilder.invalidateCRLBuildTime()
b.crlBuilder.flushCRLBuildTimeInvalidation(sc)
modified = true
}
}
// Leaf Not After Changes
rawLeafBehaviorData, ok := data.GetOk("leaf_not_after_behaivor")
if ok {
rawLeafBehavior := rawLeafBehaviorData.(string)
var newLeafBehavior certutil.NotAfterBehavior
switch rawLeafBehavior {
case "err":
newLeafBehavior = certutil.ErrNotAfterBehavior
case "truncate":
newLeafBehavior = certutil.TruncateNotAfterBehavior
case "permit":
newLeafBehavior = certutil.PermitNotAfterBehavior
default:
return logical.ErrorResponse("Unknown value for field `leaf_not_after_behavior`. Possible values are `err`, `truncate`, and `permit`."), nil
}
if newLeafBehavior != issuer.LeafNotAfterBehavior {
issuer.LeafNotAfterBehavior = newLeafBehavior
modified = true
}
}
// Usage Changes
rawUsageData, ok := data.GetOk("usage")
if ok {
rawUsage := rawUsageData.([]string)
newUsage, err := NewIssuerUsageFromNames(rawUsage)
if err != nil {
return logical.ErrorResponse(fmt.Sprintf("Unable to parse specified usages: %v - valid values are %v", rawUsage, AllIssuerUsages.Names())), nil
}
if newUsage != issuer.Usage {
Allow marking issuers as revoked (#16621) * Allow marking issuers as revoked This allows PKI's issuers to be considered revoked and appear on each others' CRLs. We disable issuance (via removing the usage) and prohibit modifying the usage via the regular issuer management interface. A separate endpoint is necessary because issuers (especially if signed by a third-party CA using incremental serial numbers) might share a serial number (e.g., an intermediate under cross-signing might share the same number as an external root or an unrelated intermediate). When the next CRL rebuild happens, this issuer will then appear on others issuers CRLs, if they validate this issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on revoking issuers Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for issuer revocation semantics Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Notate that CRLs will be rebuilt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix timestamp field from _utc -> to _rfc3339 Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Ensure serial-based accesses shows as revoked Thanks Kit! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning when revoking default issuer Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-18 22:08:31 +00:00
if issuer.Revoked && newUsage.HasUsage(IssuanceUsage) {
// Forbid allowing cert signing on its usage.
return logical.ErrorResponse("This issuer was revoked; unable to modify its usage to include certificate signing again. Reissue this certificate (preferably with a new key) and modify that entry instead."), nil
Allow marking issuers as revoked (#16621) * Allow marking issuers as revoked This allows PKI's issuers to be considered revoked and appear on each others' CRLs. We disable issuance (via removing the usage) and prohibit modifying the usage via the regular issuer management interface. A separate endpoint is necessary because issuers (especially if signed by a third-party CA using incremental serial numbers) might share a serial number (e.g., an intermediate under cross-signing might share the same number as an external root or an unrelated intermediate). When the next CRL rebuild happens, this issuer will then appear on others issuers CRLs, if they validate this issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on revoking issuers Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for issuer revocation semantics Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Notate that CRLs will be rebuilt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix timestamp field from _utc -> to _rfc3339 Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Ensure serial-based accesses shows as revoked Thanks Kit! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning when revoking default issuer Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-18 22:08:31 +00:00
}
cert, err := issuer.GetCertificate()
if err != nil {
return nil, fmt.Errorf("unable to parse issuer's certificate: %v", err)
}
if (cert.KeyUsage&x509.KeyUsageCRLSign) == 0 && newUsage.HasUsage(CRLSigningUsage) {
return logical.ErrorResponse("This issuer's underlying certificate lacks the CRLSign KeyUsage value; unable to set CRLSigningUsage on this issuer as a result."), nil
}
issuer.Usage = newUsage
modified = true
}
}
Add PSS support to PKI Secrets Engine (#16519) * Add PSS signature support to Vault PKI engine Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use issuer's RevocationSigAlg for CRL signing We introduce a new parameter on issuers, revocation_signature_algorithm to control the signature algorithm used during CRL signing. This is because the SignatureAlgorithm value from the certificate itself is incorrect for this purpose: a RSA root could sign an ECDSA intermediate with say, SHA256WithRSA, but when the intermediate goes to sign a CRL, it must use ECDSAWithSHA256 or equivalent instead of SHA256WithRSA. When coupled with support for PSS-only keys, allowing the user to set the signature algorithm value as desired seems like the best approach. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add use_pss, revocation_signature_algorithm docs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add PSS to signature role issuance test matrix Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow roots to self-identify revocation alg When using PSS support with a managed key, sometimes the underlying device will not support PKCS#1v1.5 signatures. This results in CRL building failing, unless we update the entry's signature algorithm prior to building the CRL for the new root. With a RSA-type key and use_pss=true, we use the signature bits value to decide which hash function to use for PSS support. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add clearer error message on failed import When CRL building fails during cert/key import, due to PSS failures, give a better indication to the user that import succeeded its just CRL building that failed. This tells them the parameter to adjust on the issuer and warns that CRL building will fail until this is fixed. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add case insensitive SigAlgo matching Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Convert UsePSS back to regular bool Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor PSS->certTemplate into helper function Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Proper string output on rev_sig_alg display Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Copy root's SignatureAlgorithm for CRL building Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-03 16:42:24 +00:00
// Revocation signature algorithm changes
rawRevSigAlg, ok := data.GetOk("revocation_signature_algorithm")
if ok {
revSigAlgStr := rawRevSigAlg.(string)
revSigAlg, present := certutil.SignatureAlgorithmNames[strings.ToLower(revSigAlgStr)]
if !present && revSigAlgStr != "" {
var knownAlgos []string
for algoName := range certutil.SignatureAlgorithmNames {
knownAlgos = append(knownAlgos, algoName)
}
return logical.ErrorResponse(fmt.Sprintf("Unknown signature algorithm value: %v - valid values are %v", revSigAlg, strings.Join(knownAlgos, ", "))), nil
} else if revSigAlgStr == "" {
revSigAlg = x509.UnknownSignatureAlgorithm
}
if err := issuer.CanMaybeSignWithAlgo(revSigAlg); err != nil {
return nil, err
}
if revSigAlg != issuer.RevocationSigAlg {
issuer.RevocationSigAlg = revSigAlg
modified = true
}
}
Add per-issuer AIA URI information to PKI secrets engine (#16563) * Add per-issuer AIA URI information Per discussion on GitHub with @maxb, this allows issuers to have their own copy of AIA URIs. Because each issuer has its own URLs (for CA and CRL access), its necessary to mint their issued certs pointing to the correct issuer and not to the global default issuer. For anyone using multiple issuers within a mount, this change allows the issuer to point back to itself via leaf's AIA info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on per-issuer AIA info Also add it to the considerations page as something to watch out for. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for per-issuer AIA information Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor AIA setting on the issuer This introduces a common helper per Steve's suggestion. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages w.r.t. AIA naming Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error messages regarding AIA URLs This clarifies which request parameter the invalid URL is contained in, disambiguating the sometimes ambiguous usage of AIA, per suggestion by Max. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename getURLs -> getGlobalAIAURLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correct AIA acronym expansion word orders Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix bad comment suggesting re-generating roots Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add two entries to URL tests Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-19 15:43:44 +00:00
// AIA access changes.
if issuer.AIAURIs == nil {
issuer.AIAURIs = &certutil.URLEntries{}
}
// Associative mapping from data source to destination on the
// backing issuer object. For PATCH requests, we use the source
// data parameter as we still need to validate them and process
// it into a string list.
type aiaPair struct {
Source string
Dest *[]string
}
pairs := []aiaPair{
{
Source: "issuing_certificates",
Dest: &issuer.AIAURIs.IssuingCertificates,
},
{
Source: "crl_distribution_points",
Dest: &issuer.AIAURIs.CRLDistributionPoints,
},
{
Source: "ocsp_servers",
Dest: &issuer.AIAURIs.OCSPServers,
},
}
// For each pair, if it is different on the object, update it.
for _, pair := range pairs {
rawURLsValue, ok := data.GetOk(pair.Source)
if ok {
urlsValue := rawURLsValue.([]string)
if badURL := validateURLs(urlsValue); badURL != "" {
return logical.ErrorResponse(fmt.Sprintf("invalid URL found in Authority Information Access (AIA) parameter %v: %s", pair.Source, badURL)), nil
}
if isStringArrayDifferent(urlsValue, *pair.Dest) {
modified = true
*pair.Dest = urlsValue
}
}
}
// If no AIA URLs exist on the issuer, set the AIA URLs entry to nil to
// ease usage later.
if len(issuer.AIAURIs.IssuingCertificates) == 0 && len(issuer.AIAURIs.CRLDistributionPoints) == 0 && len(issuer.AIAURIs.OCSPServers) == 0 {
issuer.AIAURIs = nil
}
// Manual Chain Changes
newPathData, ok := data.GetOk("manual_chain")
if ok {
newPath := newPathData.([]string)
var updateChain bool
var constructedChain []issuerID
for index, newPathRef := range newPath {
// Allow self for the first entry.
if index == 0 && newPathRef == "self" {
newPathRef = string(ref)
}
resolvedId, err := sc.resolveIssuerReference(newPathRef)
if err != nil {
return nil, err
}
if index == 0 && resolvedId != ref {
return logical.ErrorResponse(fmt.Sprintf("expected first cert in chain to be a self-reference, but was: %v/%v", newPathRef, resolvedId)), nil
}
constructedChain = append(constructedChain, resolvedId)
if len(issuer.ManualChain) < len(constructedChain) || constructedChain[index] != issuer.ManualChain[index] {
updateChain = true
}
}
if len(issuer.ManualChain) != len(constructedChain) {
updateChain = true
}
if updateChain {
issuer.ManualChain = constructedChain
// Building the chain will write the issuer to disk; no need to do it
// twice.
modified = false
err := sc.rebuildIssuersChains(issuer)
if err != nil {
return nil, err
}
}
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if modified {
err := sc.writeIssuer(issuer)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
}
response, err := respondReadIssuer(issuer)
if newName != oldName {
addWarningOnDereferencing(sc, oldName, response)
}
return response, err
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
}
func (b *backend) pathGetRawIssuer(ctx context.Context, req *logical.Request, data *framework.FieldData) (*logical.Response, error) {
if b.useLegacyBundleCaStorage() {
return logical.ErrorResponse("Can not get issuer until migration has completed"), nil
}
issuerName := getIssuerRef(data)
if len(issuerName) == 0 {
return logical.ErrorResponse("missing issuer reference"), nil
}
sc := b.makeStorageContext(ctx, req.Storage)
ref, err := sc.resolveIssuerReference(issuerName)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
if ref == "" {
return logical.ErrorResponse("unable to resolve issuer id for reference: " + issuerName), nil
}
issuer, err := sc.fetchIssuerById(ref)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
var contentType string
PKI - Honor header If-Modified-Since if present (#16249) * honor header if-modified-since if present * pathGetIssuerCRL first version * check if modified since for CA endpoints * fix date comparison for CA endpoints * suggested changes and refactoring * add writeIssuer to updateDefaultIssuerId and fix error * Move methods out of storage.go into util.go For the most part, these take a SC as param, but aren't directly storage relevant operations. Move them out of storage.go as a result. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use UTC timezone for storage Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rework path_fetch for better if-modified-since handling Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Invalidate all issuers, CRLs on default write When the default is updated, access under earlier timestamps will not work as we're unclear if the timestamp is for this issuer or a previous issuer. Thus, we need to invalidate the CRL and both issuers involved (previous, next) by updating their LastModifiedTimes. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for If-Modified-Since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invalidate default issuer changes When the default issuer changes, we'll have to mark the invalidation on PR secondary clusters, so they know to update their CRL mapping as well. The swapped issuers will have an updated modification time (which will eventually replicate down and thus be correct), but the CRL modification time is cluster-local information and thus won't be replicated. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * make fmt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor sendNotModifiedResponseIfNecessary Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on if-modified-since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 19:28:47 +00:00
var certificate []byte
response := &logical.Response{}
ret, err := sendNotModifiedResponseIfNecessary(&IfModifiedSinceHelper{req: req, reqType: ifModifiedCA, issuerRef: ref}, sc, response)
PKI - Honor header If-Modified-Since if present (#16249) * honor header if-modified-since if present * pathGetIssuerCRL first version * check if modified since for CA endpoints * fix date comparison for CA endpoints * suggested changes and refactoring * add writeIssuer to updateDefaultIssuerId and fix error * Move methods out of storage.go into util.go For the most part, these take a SC as param, but aren't directly storage relevant operations. Move them out of storage.go as a result. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use UTC timezone for storage Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rework path_fetch for better if-modified-since handling Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Invalidate all issuers, CRLs on default write When the default is updated, access under earlier timestamps will not work as we're unclear if the timestamp is for this issuer or a previous issuer. Thus, we need to invalidate the CRL and both issuers involved (previous, next) by updating their LastModifiedTimes. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for If-Modified-Since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invalidate default issuer changes When the default issuer changes, we'll have to mark the invalidation on PR secondary clusters, so they know to update their CRL mapping as well. The swapped issuers will have an updated modification time (which will eventually replicate down and thus be correct), but the CRL modification time is cluster-local information and thus won't be replicated. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * make fmt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor sendNotModifiedResponseIfNecessary Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on if-modified-since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 19:28:47 +00:00
if err != nil {
return nil, err
}
if ret {
return response, nil
}
certificate = []byte(issuer.Certificate)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if strings.HasSuffix(req.Path, "/pem") {
contentType = "application/pem-certificate-chain"
} else if strings.HasSuffix(req.Path, "/der") {
contentType = "application/pkix-cert"
}
if strings.HasSuffix(req.Path, "/der") {
pemBlock, _ := pem.Decode(certificate)
if pemBlock == nil {
return nil, err
}
certificate = pemBlock.Bytes
}
statusCode := 200
if len(certificate) == 0 {
statusCode = 204
}
if strings.HasSuffix(req.Path, "/pem") || strings.HasSuffix(req.Path, "/der") {
return &logical.Response{
Data: map[string]interface{}{
logical.HTTPContentType: contentType,
logical.HTTPRawBody: certificate,
logical.HTTPStatusCode: statusCode,
},
}, nil
} else {
return &logical.Response{
Data: map[string]interface{}{
"certificate": string(certificate),
"ca_chain": issuer.CAChain,
},
}, nil
}
}
func (b *backend) pathDeleteIssuer(ctx context.Context, req *logical.Request, data *framework.FieldData) (*logical.Response, error) {
// Since we're planning on updating issuers here, grab the lock so we've
// got a consistent view.
b.issuersLock.Lock()
defer b.issuersLock.Unlock()
if b.useLegacyBundleCaStorage() {
return logical.ErrorResponse("Can not delete issuer until migration has completed"), nil
}
issuerName := getIssuerRef(data)
if len(issuerName) == 0 {
return logical.ErrorResponse("missing issuer reference"), nil
}
sc := b.makeStorageContext(ctx, req.Storage)
ref, err := sc.resolveIssuerReference(issuerName)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
// Return as if we deleted it if we fail to lookup the issuer.
if ref == IssuerRefNotFound {
return &logical.Response{}, nil
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
return nil, err
}
response := &logical.Response{}
issuer, err := sc.fetchIssuerById(ref)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
if issuer.Name != "" {
addWarningOnDereferencing(sc, issuer.Name, response)
}
addWarningOnDereferencing(sc, string(issuer.ID), response)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
wasDefault, err := sc.deleteIssuer(ref)
if err != nil {
return nil, err
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if wasDefault {
response.AddWarning(fmt.Sprintf("Deleted issuer %v (via issuer_ref %v); this was configured as the default issuer. Operations without an explicit issuer will not work until a new default is configured.", ref, issuerName))
addWarningOnDereferencing(sc, defaultRef, response)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
}
// Since we've deleted an issuer, the chains might've changed. Call the
// rebuild code. We shouldn't technically err (as the issuer was deleted
// successfully), but log a warning (and to the response) if this fails.
if err := sc.rebuildIssuersChains(nil); err != nil {
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
msg := fmt.Sprintf("Failed to rebuild remaining issuers' chains: %v", err)
b.Logger().Error(msg)
response.AddWarning(msg)
}
return response, nil
}
func addWarningOnDereferencing(sc *storageContext, name string, resp *logical.Response) {
timeout, inUseBy, err := sc.checkForRolesReferencing(name)
if err != nil || timeout {
if inUseBy == 0 {
resp.AddWarning(fmt.Sprint("Unable to check if any roles referenced this issuer by ", name))
} else {
resp.AddWarning(fmt.Sprint("The name ", name, " was in use by at least ", inUseBy, " roles"))
}
} else {
if inUseBy > 0 {
resp.AddWarning(fmt.Sprint(inUseBy, " roles reference ", name))
}
}
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
const (
pathGetIssuerHelpSyn = `Fetch a single issuer certificate.`
pathGetIssuerHelpDesc = `
This allows fetching information associated with the underlying issuer
certificate.
:ref can be either the literal value "default", in which case /config/issuers
will be consulted for the present default issuer, an identifier of an issuer,
or its assigned name value.
Use /issuer/:ref/der or /issuer/:ref/pem to return just the certificate in
raw DER or PEM form, without the JSON structure of /issuer/:ref.
Writing to /issuer/:ref allows updating of the name field associated with
the certificate.
`
)
func pathGetIssuerCRL(b *backend) *framework.Path {
Support for generating Delta CRLs (#16773) * Allow generation of up-to-date delta CRLs While switching to periodic rebuilds of CRLs alleviates the constant rebuild pressure on Vault during times of high revocation, the CRL proper becomes stale. One response to this is to switch to OCSP, but not every system has support for this. Additionally, OCSP usually requires connectivity and isn't used to augment a pre-distributed CRL (and is instead used independently). By generating delta CRLs containing only new revocations, an existing CRL can be supplemented with newer revocations without requiring Vault to rebuild all complete CRLs. Admins can periodically fetch the delta CRL and add it to the existing CRL and applications should be able to support using serials from both. Because delta CRLs are emptied when the next complete CRL is rebuilt, it is important that applications fetch the delta CRL and correlate it to their complete CRL; if their complete CRL is older than the delta CRL's extension number, applications MUST fetch the newer complete CRL to ensure they have a correct combination. This modifies the revocation process and adds several new configuration options, controlling whether Delta CRLs are enabled and when we'll rebuild it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for delta CRLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on delta CRLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address review feedback: fix several bugs Thanks Steve! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invoke periodic func on active nodes We need to ensure we read the updated config (in case of OCSP request handling on standby nodes), but otherwise want to avoid CRL/DeltaCRL re-building. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 15:37:09 +00:00
pattern := "issuer/" + framework.GenericNameRegex(issuerRefParam) + "/crl(/pem|/der|/delta(/pem|/der)?)?"
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
return buildPathGetIssuerCRL(b, pattern)
}
func buildPathGetIssuerCRL(b *backend, pattern string) *framework.Path {
fields := map[string]*framework.FieldSchema{}
fields = addIssuerRefNameFields(fields)
return &framework.Path{
// Returns raw values.
Pattern: pattern,
Fields: fields,
Operations: map[logical.Operation]framework.OperationHandler{
logical.ReadOperation: &framework.PathOperation{
Callback: b.pathGetIssuerCRL,
},
},
HelpSynopsis: pathGetIssuerCRLHelpSyn,
HelpDescription: pathGetIssuerCRLHelpDesc,
}
}
func (b *backend) pathGetIssuerCRL(ctx context.Context, req *logical.Request, data *framework.FieldData) (*logical.Response, error) {
if b.useLegacyBundleCaStorage() {
return logical.ErrorResponse("Can not get issuer's CRL until migration has completed"), nil
}
issuerName := getIssuerRef(data)
if len(issuerName) == 0 {
return logical.ErrorResponse("missing issuer reference"), nil
}
if err := b.crlBuilder.rebuildIfForced(ctx, b, req); err != nil {
return nil, err
}
PKI - Honor header If-Modified-Since if present (#16249) * honor header if-modified-since if present * pathGetIssuerCRL first version * check if modified since for CA endpoints * fix date comparison for CA endpoints * suggested changes and refactoring * add writeIssuer to updateDefaultIssuerId and fix error * Move methods out of storage.go into util.go For the most part, these take a SC as param, but aren't directly storage relevant operations. Move them out of storage.go as a result. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use UTC timezone for storage Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rework path_fetch for better if-modified-since handling Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Invalidate all issuers, CRLs on default write When the default is updated, access under earlier timestamps will not work as we're unclear if the timestamp is for this issuer or a previous issuer. Thus, we need to invalidate the CRL and both issuers involved (previous, next) by updating their LastModifiedTimes. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for If-Modified-Since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invalidate default issuer changes When the default issuer changes, we'll have to mark the invalidation on PR secondary clusters, so they know to update their CRL mapping as well. The swapped issuers will have an updated modification time (which will eventually replicate down and thus be correct), but the CRL modification time is cluster-local information and thus won't be replicated. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * make fmt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor sendNotModifiedResponseIfNecessary Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on if-modified-since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 19:28:47 +00:00
var certificate []byte
var contentType string
sc := b.makeStorageContext(ctx, req.Storage)
PKI - Honor header If-Modified-Since if present (#16249) * honor header if-modified-since if present * pathGetIssuerCRL first version * check if modified since for CA endpoints * fix date comparison for CA endpoints * suggested changes and refactoring * add writeIssuer to updateDefaultIssuerId and fix error * Move methods out of storage.go into util.go For the most part, these take a SC as param, but aren't directly storage relevant operations. Move them out of storage.go as a result. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use UTC timezone for storage Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rework path_fetch for better if-modified-since handling Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Invalidate all issuers, CRLs on default write When the default is updated, access under earlier timestamps will not work as we're unclear if the timestamp is for this issuer or a previous issuer. Thus, we need to invalidate the CRL and both issuers involved (previous, next) by updating their LastModifiedTimes. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for If-Modified-Since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invalidate default issuer changes When the default issuer changes, we'll have to mark the invalidation on PR secondary clusters, so they know to update their CRL mapping as well. The swapped issuers will have an updated modification time (which will eventually replicate down and thus be correct), but the CRL modification time is cluster-local information and thus won't be replicated. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * make fmt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor sendNotModifiedResponseIfNecessary Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on if-modified-since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 19:28:47 +00:00
response := &logical.Response{}
var crlType ifModifiedReqType = ifModifiedCRL
if strings.Contains(req.Path, "delta") {
crlType = ifModifiedDeltaCRL
}
ret, err := sendNotModifiedResponseIfNecessary(&IfModifiedSinceHelper{req: req, reqType: crlType}, sc, response)
PKI - Honor header If-Modified-Since if present (#16249) * honor header if-modified-since if present * pathGetIssuerCRL first version * check if modified since for CA endpoints * fix date comparison for CA endpoints * suggested changes and refactoring * add writeIssuer to updateDefaultIssuerId and fix error * Move methods out of storage.go into util.go For the most part, these take a SC as param, but aren't directly storage relevant operations. Move them out of storage.go as a result. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Use UTC timezone for storage Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rework path_fetch for better if-modified-since handling Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Invalidate all issuers, CRLs on default write When the default is updated, access under earlier timestamps will not work as we're unclear if the timestamp is for this issuer or a previous issuer. Thus, we need to invalidate the CRL and both issuers involved (previous, next) by updating their LastModifiedTimes. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for If-Modified-Since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invalidate default issuer changes When the default issuer changes, we'll have to mark the invalidation on PR secondary clusters, so they know to update their CRL mapping as well. The swapped issuers will have an updated modification time (which will eventually replicate down and thus be correct), but the CRL modification time is cluster-local information and thus won't be replicated. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * make fmt Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor sendNotModifiedResponseIfNecessary Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on if-modified-since Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 19:28:47 +00:00
if err != nil {
return nil, err
}
if ret {
return response, nil
}
crlPath, err := sc.resolveIssuerCRLPath(issuerName)
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
if err != nil {
return nil, err
}
Support for generating Delta CRLs (#16773) * Allow generation of up-to-date delta CRLs While switching to periodic rebuilds of CRLs alleviates the constant rebuild pressure on Vault during times of high revocation, the CRL proper becomes stale. One response to this is to switch to OCSP, but not every system has support for this. Additionally, OCSP usually requires connectivity and isn't used to augment a pre-distributed CRL (and is instead used independently). By generating delta CRLs containing only new revocations, an existing CRL can be supplemented with newer revocations without requiring Vault to rebuild all complete CRLs. Admins can periodically fetch the delta CRL and add it to the existing CRL and applications should be able to support using serials from both. Because delta CRLs are emptied when the next complete CRL is rebuilt, it is important that applications fetch the delta CRL and correlate it to their complete CRL; if their complete CRL is older than the delta CRL's extension number, applications MUST fetch the newer complete CRL to ensure they have a correct combination. This modifies the revocation process and adds several new configuration options, controlling whether Delta CRLs are enabled and when we'll rebuild it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for delta CRLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add changelog entry Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add documentation on delta CRLs Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address review feedback: fix several bugs Thanks Steve! Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Correctly invoke periodic func on active nodes We need to ensure we read the updated config (in case of OCSP request handling on standby nodes), but otherwise want to avoid CRL/DeltaCRL re-building. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-08-29 15:37:09 +00:00
if strings.Contains(req.Path, "delta") {
crlPath += deltaCRLPathSuffix
}
Allow Multiple Issuers in PKI Secret Engine Mounts - PKI Pod (#15277) * Starter PKI CA Storage API (#14796) * Simple starting PKI storage api for CA rotation * Add key and issuer storage apis * Add listKeys and listIssuers storage implementations * Add simple keys and issuers configuration storage api methods * Handle resolving key, issuer references The API context will usually have a user-specified reference to the key. This is either the literal string "default" to select the default key, an identifier of the key, or a slug name for the key. Here, we wish to resolve this reference to an actual identifier that can be understood by storage. Also adds the missing Name field to keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add method to fetch an issuer's cert bundle This adds a method to construct a certutil.CertBundle from the specified issuer identifier, optionally loading its corresponding key for signing. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Refactor certutil PrivateKey PEM handling This refactors the parsing of PrivateKeys from PEM blobs into shared methods (ParsePEMKey, ParseDERKey) that can be reused by the existing Bundle parsing logic (ParsePEMBundle) or independently in the new issuers/key-based PKI storage code. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add importKey, importCert to PKI storage importKey is generally preferable to the low-level writeKey for adding new entries. This takes only the contents of the private key (as a string -- so a PEM bundle or a managed key handle) and checks if it already exists in the storage. If it does, it returns the existing key instance. Otherwise, we create a new one. In the process, we detect any issuers using this key and link them back to the new key entry. The same holds for importCert over importKey, with the note that keys are not modified when importing certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for importing issuers, keys This adds tests for importing keys and issuers into the new storage layout, ensuring that identifiers are correctly inferred and linked. Note that directly writing entries to storage (writeKey/writeissuer) will take KeyID links from the parent entry and should not be used for import; only existing entries should be updated with this info. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Implement PKI storage migration. - Hook into the backend::initialize function, calling the migration on a primary only. - Migrate an existing certificate bundle to the new issuers and key layout * Make fetchCAInfo aware of new storage layout This allows fetchCAInfo to fetch a specified issuer, via a reference parameter provided by the user. We pass that into the storage layer and have it return a cert bundle for us. Finally, we need to validate that it truly has the key desired. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Begin /issuers API endpoints This implements the fetch operations around issuers in the PKI Secrets Engine. We implement the following operations: - LIST /issuers - returns a list of known issuers' IDs and names. - GET /issuer/:ref - returns a JSON blob with information about this issuer. - POST /issuer/:ref - allows configuring information about issuers, presently just its name. - DELETE /issuer/:ref - allows deleting the specified issuer. - GET /issuer/:ref/{der,pem} - returns a raw API response with just the DER (or PEM) of the issuer's certificate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add import to PKI Issuers API This adds the two core import code paths to the API: /issuers/import/cert and /issuers/import/bundle. The former differs from the latter in that the latter allows the import of keys. This allows operators to restrict importing of keys to privileged roles, while allowing more operators permission to import additional certificates (not used for signing, but instead for path/chain building). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-intermediate endpoint This endpoint allows existing issuers to be used to sign intermediate CA certificates. In the process, we've updated the existing /root/sign-intermediate endpoint to be equivalent to a call to /issuer/default/sign-intermediate. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-self-issued endpoint This endpoint allows existing issuers to be used to sign self-signed certificates. In the process, we've updated the existing /root/sign-self-issued endpoint to be equivalent to a call to /issuer/default/sign-self-issued. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/sign-verbatim endpoint This endpoint allows existing issuers to be used to directly sign CSRs. In the process, we've updated the existing /sign-verbatim endpoint to be equivalent to a call to /issuer/:ref/sign-verbatim. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow configuration of default issuers Using the new updateDefaultIssuerId(...) from the storage migration PR allows for easy implementation of configuring the default issuer. We restrict callers from setting blank defaults and setting default to default. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix fetching default issuers After setting a default issuer, one should be able to use the old /ca, /ca_chain, and /cert/{ca,ca_chain} endpoints to fetch the default issuer (and its chain). Update the fetchCertBySerial helper to no longer support fetching the ca and prefer fetchCAInfo for that instead (as we've already updated that to support fetching the new issuer location). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add /issuer/:ref/{sign,issue}/:role This updates the /sign and /issue endpoints, allowing them to take the default issuer (if none is provided by a role) and adding issuer-specific versions of them. Note that at this point in time, the behavior isn't yet ideal (as /sign/:role allows adding the ref=... parameter to override the default issuer); a later change adding role-based issuer specification will fix this incorrect behavior. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support root issuer generation * Add support for issuer generate intermediate end-point * Update issuer and key arguments to consistent values - Update all new API endpoints to use the new agreed upon argument names. - issuer_ref & key_ref to refer to existing - issuer_name & key_name for new definitions - Update returned values to always user issuer_id and key_id * Add utility methods to fetch common ref and name arguments - Add utility methods to fetch the issuer_name, issuer_ref, key_name and key_ref arguments from data fields. - Centralize the logic to clean up these inputs and apply various validations to all of them. * Rename common PKI backend handlers - Use the buildPath convention for the function name instead of common... * Move setting PKI defaults from writeCaBundle to proper import{keys,issuer} methods - PR feedback, move setting up the default configuration references within the import methods instead of within the writeCaBundle method. This should now cover all use cases of us setting up the defaults properly. * Introduce constants for issuer_ref, rename isKeyDefaultSet... * Fix legacy PKI sign-verbatim api path - Addresses some test failures due to an incorrect refactoring of a legacy api path /sign-verbatim within PKI * Use import code to handle intermediate, config/ca The existing bundle import code will satisfy the intermediate import; use it instead of the old ca_bundle import logic. Additionally, update /config/ca to use the new import code as well. While testing, a panic was discovered: > reflect.Value.SetMapIndex: value of type string is not assignable to type pki.keyId This was caused by returning a map with type issuerId->keyId; instead switch to returning string->string maps so the audit log can properly HMAC them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clarify error message on missing defaults When the default issuer and key are missing (and haven't yet been specified), we should clarify that error message. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update test semantics for new changes This makes two minor changes to the existing test suite: 1. Importing partial bundles should now succeed, where they'd previously error. 2. fetchCertBySerial no longer handles CA certificates. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add support for deleting all keys, issuers The old DELETE /root code must now delete all keys and issuers for backwards compatibility. We strongly suggest calling individual delete methods (DELETE /key/:key_ref or DELETE /issuer/:issuer_ref) instead, for finer control. In the process, we detect whether the deleted key/issuers was set as the default. This will allow us to warn (from the single key/deletion issuer code) whether or not the default was deleted (while allowing the operation to succeed). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Introduce defaultRef constant within PKI - Replace hardcoded "default" references with a constant to easily identify various usages. - Use the addIssuerRefField function instead of redefining the field in various locations. * Rework PKI test TestBackend_Root_Idempotency - Validate that generate/root calls are no longer idempotent, but the bundle importing does not generate new keys/issuers - As before make sure that the delete root api resets everything - Address a bug within the storage that we bombed when we had multiple different key types within storage. * Assign Name=current to migrated key and issuer - Detail I missed from the RFC was to assign the Name field as "current" for migrated key and issuer. * Build CRL upon PKI intermediary set-signed api called - Add a call to buildCRL if we created an issuer within pathImportIssuers - Augment existing FullCAChain to verify we have a proper CRL post set-signed api call - Remove a code block writing out "ca" storage entry that is no longer used. * Identify which certificate or key failed When importing complex chains, we should identify in which certificate or key the failure occurred. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI migration writes out empty migration log entry - Since the elements of the struct were not exported we serialized an empty migration log to disk and would re-run the migration * Add chain-building logic to PKI issuers path With the one-entry-per-issuer approach, CA Chains become implicitly constructed from the pool of issuers. This roughly matches the existing expectations from /config/ca (wherein a chain could be provided) and /intemediate/set-signed (where a chain may be provided). However, in both of those cases, we simply accepted a chain. Here, we need to be able to reconstruct the chain from parts on disk. However, with potential rotation of roots, we need to be aware of disparate chains. Simply concating together all issuers isn't sufficient. Thus we need to be able to parse a certificate's Issuer and Subject field and reconstruct valid (and potentially parallel) parent<->child mappings. This attempts to handle roots, intermediates, cross-signed intermediates, cross-signed roots, and rotated keys (wherein one might not have a valid signature due to changed key material with the same subject). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Return CA Chain when fetching issuers This returns the CA Chain attribute of an issuer, showing its computed chain based on other issuers in the database, when fetching a specific issuer. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add testing for chain building Using the issuance infrastructure, we generate new certificates (either roots or intermediates), positing that this is roughly equivalent to importing an external bundle (minus error handling during partial imports). This allows us to incrementally construct complex chains, creating reissuance cliques and cross-signing cycles. By using ECDSA certificates, we avoid high signature verification and key generation times. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow manual construction of issuer chain Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix handling of duplicate names With the new issuer field (manual_chain), we can no longer err when a name already exists: we might be updating the existing issuer (with the same name), but changing its manual_chain field. Detect this error and correctly handle it. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for manual chain building We break the clique, instead building these chains manually, ensuring that the remaining chains do not change and only the modified certs change. We then reset them (back to implicit chain building) and ensure we get the same results as earlier. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter verification of issuers PEM format This ensures each issuer is only a single certificate entry (as validated by count and parsing) without any trailing data. We further ensure that each certificate PEM has leading and trailing spaces removed with only a single trailing new line remaining. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Fix full chain building Don't set the legacy IssuingCA field on the certificate bundle, as we prefer the CAChain field over it. Additionally, building the full chain could result in duplicate certificates when the CAChain included the leaf certificate itself. When building the full chain, ensure we don't include the bundle's certificate twice. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add stricter tests for full chain construction We wish to ensure that each desired certificate in the chain is only present once. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Rename PKI types to avoid constant variable name collisions keyId -> keyID issuerId -> issuerID key -> keyEntry issuer -> issuerEntry keyConfig -> keyConfigEntry issuerConfig -> issuerConfigEntry * Update CRL handling for multiple issuers When building CRLs, we've gotta make sure certs issued by that issuer land up on that issuer's CRL and not some other CRL. If no CRL is found (matching a cert), we'll place it on the default CRL. However, in the event of equivalent issuers (those with the same subject AND the same key material) -- perhaps due to reissuance -- we'll only create a single (unified) CRL for them. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching updated CRL locations This updates fetchCertBySerial to support querying the default issuer's CRL. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL storage location test case Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update to CRLv2 Format to copy RawIssuer When using the older Certificate.CreateCRL(...) call, Go's x509 library copies the parsed pkix.Name version of the CRL Issuer's Subject field. For certain constructed CAs, this fails since pkix.Name is not suitable for round-tripping. This also builds a CRLv1 (per RFC 5280) CRL. In updating to the newer x509.CreateRevocationList(...) call, we can construct the CRL in the CRLv2 format and correctly copy the issuer's name. However, this requires holding an additional field per-CRL, the CRLNumber field, which is required in Go's implementation of CRLv2 (though OPTIONAL in the spec). We store this on the new LocalCRLConfigEntry object, per-CRL. Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com> Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add comment regarding CRL non-assignment in GOTO In previous versions of Vault, it was possible to sign an empty CRL (when the CRL was disabled and a force-rebuild was requested). Add a comment about this case. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow fetching the specified issuer's CRL We add a new API endpoint to fetch the specified issuer's CRL directly (rather than the default issuer's CRL at /crl and /certs/crl). We also add a new test to validate the CRL in a multi-root scenario and ensure it is signed with the correct keys. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add new PKI key prefix to seal wrapped storage (#15126) * Refactor common backend initialization within backend_test - Leverage an existing helper method within the PKI backend tests to setup a PKI backend with storage. * Add ability to read legacy cert bundle if the migration has not occurred on secondaries. - Track the migration state forbidding an issuer/key writing api call if we have not migrated - For operations that just need to read the CA bundle, use the same tracking variable to switch between reading the legacy bundle or use the new key/issuer storage. - Add an invalidation function that will listen for updates to our log path to refresh the state on secondary clusters. * Always write migration entry to trigger secondary clusters to wake up - Some PR feedback and handle a case in which the primary cluster does not have a CA bundle within storage but somehow a secondary does. * Update CA Chain to report entire chain This merges the ca_chain JSON field (of the /certs/ca_chain path) with the regular certificate field, returning the root of trust always. This also affects the non-JSON (raw) endpoints as well. We return the default issuer's chain here, rather than all known issuers (as that may not form a strict chain). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Allow explicit issuer override on roles When a role is used to generate a certificate (such as with the sign/ and issue/ legacy paths or the legacy sign-verbatim/ paths), we prefer that issuer to the one on the request. This allows operators to set an issuer (other than default) for requests to be issued against, effectively making the change no different from the users' perspective as it is "just" a different role name. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for role-based issuer selection Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Expand NotAfter limit enforcement behavior Vault previously strictly enforced NotAfter/ttl values on certificate requests, erring if the requested TTL extended past the NotAfter date of the issuer. In the event of issuing an intermediate, this behavior was ignored, instead permitting the issuance. Users generally do not think to check their issuer's NotAfter date when requesting a certificate; thus this behavior was generally surprising. Per RFC 5280 however, issuers need to maintain status information throughout the life cycle of the issued cert. If this leaf cert were to be issued for a longer duration than the parent issuer, the CA must still maintain revocation information past its expiration. Thus, we add an option to the issuer to change the desired behavior: - err, to err out, - permit, to permit the longer NotAfter date, or - truncate, to silently truncate the expiration to the issuer's NotAfter date. Since expiration of certificates in the system's trust store are not generally validated (when validating an arbitrary leaf, e.g., during TLS validation), permit should generally only be used in that case. However, browsers usually validate intermediate's validity periods, and thus truncate should likely be used (as with permit, the leaf's chain will not validate towards the end of the issuance period). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add tests for expanded issuance behaviors Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add warning on keyless default issuer (#15178) Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Update PKI to new Operations framework (#15180) The backend Framework has updated Callbacks (used extensively in PKI) to become deprecated; Operations takes their place and clarifies forwarding of requests. We switch to the new format everywhere, updating some bad assumptions about forwarding along the way. Anywhere writes are handled (that should be propagated to all nodes in all clusters), we choose to forward the request all the way up to the performance primary cluster's primary node. This holds for issuers/keys, roles, and configs (such as CRL config, which is globally set for all clusters despite all clusters having their own separate CRL). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Kitography/vault 5474 rebase (#15150) * These parts work (put in signature so that backend wouldn't break, but missing fields, desc, etc.) * Import and Generate API calls w/ needed additions to SDK. * make fmt * Add Help/Sync Text, fix some of internal/exported/kms code. * Fix PEM/DER Encoding issue. * make fmt * Standardize keyIdParam, keyNameParam, keyTypeParam * Add error response if key to be deleted is in use. * replaces all instances of "default" in code with defaultRef * Updates from Callbacks to Operations Function with explicit forwarding. * Fixes a panic with names not being updated everywhere. * add a logged error in addition to warning on deleting default key. * Normalize whitespace upon importing keys. Authored-by: Alexander Scheel <alexander.m.scheel@gmail.com> * Fix isKeyInUse functionality. * Fixes tests associated with newline at end of key pem. * Add alternative proposal PKI aliased paths (#15211) * Add aliased path for root/rotate/:exported This adds a user-friendly path name for generating a rotated root. We automatically choose the name "next" for the newly generated root at this path if it doesn't already exist. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add aliased path for intermediate/cross-sign This allows cross-signatures to work. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add path for replacing the current root This updates default to point to the value of the issuer with name "next" rather than its current value. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove plural issuers/ in signing paths These paths use a single issuer and thus shouldn't include the plural issuers/ as a path prefix, instead using the singular issuer/ path prefix. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Only warn if default issuer was imported When the default issuer was not (re-)imported, we'd fail to find it, causing an extraneous warning about missing keys, even though this issuer indeed had a key. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing issuer sign/issue paths Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Clean up various warnings within the PKI package (#15230) * Rebuild CRLs on secondary performance clusters post migration and on new/updated issuers - Hook into the backend invalidation function so that secondaries are notified of new/updated issuer or migrations occuring on the primary cluster. Upon notification schedule a CRL rebuild to take place upon the next process to read/update the CRL or within the periodic function if no request comes in. * Schedule rebuilding PKI CRLs on active nodes only - Address an issue that we were scheduling the rebuilding of a CRL on standby nodes, which would not be able to write to storage. - Fix an issue with standby nodes not correctly determining that a migration previously occurred. * Return legacy CRL storage path when no migration has occurred. * Handle issuer, keys locking (#15227) * Handle locking of issuers during writes We need a write lock around writes to ensure serialization of modifications. We use a single lock for both issuer and key updates, in part because certain operations (like deletion) will potentially affect both. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add missing b.useLegacyBundleCaStorage guards Several locations needed to guard against early usage of the new issuers endpoint pre-migration. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Address PKI to properly support managed keys (#15256) * Address codebase for managed key fixes * Add proper public key comparison for better managed key support to importKeys * Remove redundant public key fetching within PKI importKeys * Correctly handle rebuilding remaining chains When deleting a specific issuer, we might impact the chains. From a consistency perspective, we need to ensure the remaining chains are correct and don't refer to the since-deleted issuer, so trigger a full rebuild here. We don't need to call this in the delete-the-world (DELETE /root) code path, as there shouldn't be any remaining issuers or chains to build. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove legacy CRL bundle on world deletion When calling DELETE /root, we should remove the legacy CRL bundle, since we're deleting the legacy CA issuer bundle as well. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Remove deleted issuers' CRL entries Since CRLs are no longer resolvable after deletion (due to missing issuer ID, which will cause resolution to fail regardless of if an ID or a name/default reference was used), we should delete these CRLs from storage to avoid leaking them. In the event that this issuer comes back (with key material), we can simply rebuild the CRL at that time (from the remaining revoked storage entries). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthed JSON fetching of CRLs, Issuers (#15253) Default to fetching JSON CRL for consistency This makes the bare issuer-specific CRL fetching endpoint return the JSON-wrapped CRL by default, moving the DER CRL to a specific endpoint. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add JSON-specific endpoint for fetching issuers Unlike the unqualified /issuer/:ref endpoint (which also returns JSON), we have a separate /issuer/:ref/json endpoint to return _only_ the PEM-encoded certificate and the chain, mirroring the existing /cert/ca endpoint but for a specific issuer. This allows us to make the endpoint unauthenticated, whereas the bare endpoint would remain authenticated and usually privileged. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> Add tests for raw JSON endpoints Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add unauthenticated issuers endpoints to PKI table This adds the unauthenticated issuers endpoints? - LIST /issuers, - Fetching _just_ the issuer certificates (in JSON/DER/PEM form), and - Fetching the CRL of this issuer (in JSON/DER/PEM form). Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * Add issuer usage restrictions bitset This allows issuers to have usage restrictions, limiting whether they can be used to issue certificates or if they can generate CRLs. This allows certain issuers to not generate a CRL (if the global config is with the CRL enabled) or allows the issuer to not issue new certificates (but potentially letting the CRL generation continue). Setting both fields to false effectively forms a soft delete capability. Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com> * PKI Pod rotation Add Base Changelog (#15283) * PKI Pod rotation changelog. * Use feature release-note formatting of changelog. Co-authored-by: Steven Clark <steven.clark@hashicorp.com> Co-authored-by: Kit Haines <kit.haines@hashicorp.com> Co-authored-by: kitography <khaines@mit.edu>
2022-05-11 16:42:28 +00:00
crlEntry, err := req.Storage.Get(ctx, crlPath)
if err != nil {
return nil, err
}
if crlEntry != nil && len(crlEntry.Value) > 0 {
certificate = []byte(crlEntry.Value)
}
if strings.HasSuffix(req.Path, "/der") {
contentType = "application/pkix-crl"
} else if strings.HasSuffix(req.Path, "/pem") {
contentType = "application/x-pem-file"
}
if !strings.HasSuffix(req.Path, "/der") {
// Rather return an empty response rather than an empty PEM blob.
// We build this PEM block for both the JSON and PEM endpoints.
if len(certificate) > 0 {
pemBlock := pem.Block{
Type: "X509 CRL",
Bytes: certificate,
}
certificate = pem.EncodeToMemory(&pemBlock)
}
}
statusCode := 200
if len(certificate) == 0 {
statusCode = 204
}
if strings.HasSuffix(req.Path, "/der") || strings.HasSuffix(req.Path, "/pem") {
return &logical.Response{
Data: map[string]interface{}{
logical.HTTPContentType: contentType,
logical.HTTPRawBody: certificate,
logical.HTTPStatusCode: statusCode,
},
}, nil
}
return &logical.Response{
Data: map[string]interface{}{
"crl": string(certificate),
},
}, nil
}
const (
pathGetIssuerCRLHelpSyn = `Fetch an issuer's Certificate Revocation Log (CRL).`
pathGetIssuerCRLHelpDesc = `
This allows fetching the specified issuer's CRL. Note that this is different
than the legacy path (/crl and /certs/crl) in that this is per-issuer and not
just the default issuer's CRL.
Two issuers will have the same CRL if they have the same key material and if
they have the same Subject value.
:ref can be either the literal value "default", in which case /config/issuers
will be consulted for the present default issuer, an identifier of an issuer,
or its assigned name value.
- /issuer/:ref/crl is JSON encoded and contains a PEM CRL,
- /issuer/:ref/crl/pem contains the PEM-encoded CRL,
- /issuer/:ref/crl/DER contains the raw DER-encoded (binary) CRL.
`
)