open-vault/website/content/docs/secrets/pki/considerations.mdx

---
layout: docs
page_title: 'PKI - Secrets Engines: Considerations'
description: The PKI secrets engine for Vault generates TLS certificates.
---

# PKI Secrets Engine - Considerations

To successfully deploy this secrets engine, there are a number of important
considerations to be aware of, as well as some preparatory steps that should be
undertaken. You should read all of these _before_ using this secrets engine or
generating the CA to use with this secrets engine.

## Table of Contents

 - [Be Careful with Root CAs](#be-careful-with-root-cas)
   - [Managed Keys](#managed-keys)
 - [One CA Certificate, One Secrets Engine](#one-ca-certificate-one-secrets-engine)
   - [Always Configure a Default Issuer](#always-configure-a-default-issuer)
   - [Key Types Matter](#key-types-matter)
 - [Use a CA Hierarchy](#use-a-ca-hierarchy)
   - [Cross-Signed Intermediates](#cross-signed-intermediates)
 - [Keep certificate lifetimes short, for CRL's sake](#keep-certificate-lifetimes-short-for-crls-sake)
   - [NotAfter Behavior on Leaf Certificates](#notafter-behavior-on-leaf-certificates)
   - [Cluster Performance and Quantity of Leaf Certificates](#cluster-performance-and-quantity-of-leaf-certificates)
 - [You must configure issuing/CRL/OCSP information _in advance_](#you-must-configure-issuingcrlocsp-information-_in-advance_)
 - [Distribution of CRLs and OCSP](#distribution-of-crls-ocsp)
 - [Automate Leaf Certificate Renewal](#automate-leaf-certificate-renewal)
 - [Automate CRL Building and Tidying](#automate-crl-building-and-tidying)
 - [Safe Minimums](#safe-minimums)
 - [Token Lifetimes and Revocation](#token-lifetimes-and-revocation)
 - [Safe Usage of Roles](#safe-usage-of-roles)
 - [Telemetry](#telemetry)
 - [Auditing](#auditing)
 - [Role-Based Access](#role-based-access)
 - [Replicated DataSets](#replicated-datasets)
 - [Cluster Scalability](#cluster-scalability)
 - [PSS Support](#pss-support)
 - [Issuer Subjects and CRLs](#issuer-subjects-and-crls)
 - [Issuer Storage Migration Issues](#issuer-storage-migration-issues)

## Be Careful with Root CAs

Vault storage is secure, but not as secure as a piece of paper in a bank vault.
It is, after all, networked software. If your root CA is hosted outside of
Vault, don't put it in Vault as well; instead, issue a shorter-lived
intermediate CA certificate and put this into Vault. This aligns with industry
best practices.

Since 0.4, the secrets engine supports generating self-signed root CAs and
creating and signing CSRs for intermediate CAs. In each instance, for security
reasons, the private key can _only_ be exported at generation time, and the
ability to do so is part of the command path (so it can be put into ACL
policies).

If you plan on using intermediate CAs with Vault, it is suggested that you let
Vault create CSRs and do not export the private key, then sign those with your
root CA (which may be a second mount of the `pki` secrets engine).

### Managed Keys

Since 1.10, Vault Enterprise can access private key material in a
[_managed key_](/docs/enterprise/managed-keys).  In this case, Vault never sees the
private key, and the external KMS or HSM performs certificate signing operations.
Managed keys are configured by selecting the `kms` type when generating a root
or intermediate.

## One CA Certificate, One Secrets Engine

Since Vault 1.11.0, the PKI Secrets Engine supports multiple issuers in a single
mount. However, in order to simplify the configuration, it is _strongly_
recommended that operators limit a mount to a single issuer. If you want to issue
certificates from multiple disparate CAs, mount the PKI secrets engine at multiple
mount points with separate CA certificates in each.

The rationale for separating mounts is to simplify permissions management:
very few individuals need access to perform operations with the root, but
many need access to create leaves. The operations on a root should generally
be limited to issuing and revoking intermediate CAs, which is a highly
privileged operation; it becomes much easier to audit these operations when
they're in a separate mount than if they're mixed in with day-to-day leaf
issuance.

A common pattern is to have one mount act as your root CA and to use this CA
only to sign intermediate CA CSRs from other PKI secrets engines.

To keep old CAs active, there's two approaches to achieving rotation:

 1. Use multiple secrets engines. This allows a fresh start, preserving the
    old issuer and CRL. Vault ACL policy can be updated to deny new issuance
    under the old mount point and roles can be re-evaluated before being
    imported into the new mount point.
 2. Use multiple issuers in the same mount point. The usage of the old issuer
    can be restricted to CRL signing, and existing roles and ACL policy can be
    kept as-is. This allows cross-signing within the same mount, and consumers
    of the mount won't have to update their configuration. Once the transitional
    period for this rotation has completed and all past issued certificate have
    expired, it is encouraged to fully remove the old issuer and any unnecessary
    cross-signed issuers from the mount point.

Another suggested use case for multiple issuers in the same mount is splitting
issuance by TTL lifetime. For short-lived certificates, an intermediate
stored in Vault will often out-perform a HSM-backed intermediate. For
longer-lived certificates, however, it is often important to have the
intermediate key material secured throughout the lifetime of the end-entity
certificate. This means that two intermediates in the same mount -- one backed
by the HSM and one backed by Vault -- can satisfy both use cases. Operators
can make roles setting maximum TTLs for each issuer and consumers of the
mount can decide which to use.

### Always Configure a Default Issuer

For backwards compatibility, [the default issuer](/api-docs/secret/pki#read-issuers-configuration)
is used to service PKI endpoints without an explicit issuer (either via path
selection or role-based selection). When certificates are revoked and their
issuer is no longer part of this PKI mount, Vault places them on the default
issuer's CRL. This means maintaining a default issuer is important for both
backwards compatibility for issuing certificates and for ensuring revoked
certificates land on a CRL.

### Key Types Matter

Certain key types have impacts on performance. Signing certificates from a RSA
key will be slower than issuing from an ECDSA or Ed25519 key. Key generation
(using `/issue/:role` endpoints) using RSA keys will also be slow: RSA key
generation involves finding suitable random primes, whereas Ed25519 keys can
be random data. As the number of bits goes up (RSA 2048 -> 4096 or ECDSA
P-256 -> P-521), signature times also increases.

This matters in both directions: not only is issuance more expensive,
but validation of the corresponding signature (in say, TLS handshakes) will
also be more expensive. Careful consideration of both issuer and issued key
types can have meaningful impacts on performance of not only Vault, but
systems using these certificates.

## Use a CA Hierarchy

It is generally recommended to use a hierarchical CA setup, with a root
certificate which issues one or more intermediates (based on usage), which
in turn issue the leaf certificates.

This allows stronger storage or policy guarantees around [protection of the
root CA](#be-careful-with-root-cas), while letting Vault manage the
intermediate CAs and issuance of leaves. Different intermediates might be
issued for different usage, such as VPN signing, Email signing, or testing
versus production TLS services. This helps to keep CRLs limited to specific
purposes: for example, VPN services don't care about the revoked set of
email signing certificates if they're using separate certificates and
different intermediates, and thus don't need both CRL contents. Additionally,
this allows higher risk intermediates (such as those issuing longer-lived
email signing certificates) to have HSM-backing without impacting the
performance of easier-to-rotate intermediates and certificates (such as
TLS intermediates).

Vault supports the use of both the [`allowed_domains` parameter on
Roles](/api-docs/secret/pki#allowed_domains) and the [`permitted_dns_domains`
parameter to set the Name Constraints extension](/api-docs/secret/pki#permitted_dns_domains)
on root and intermediate generation. This allows for several layers of
separation of concerns between TLS-based services.

### Cross-Signed Intermediates

When cross-signing intermediates from two separate roots, two separate
intermediate issuers will exist within the Vault PKI mount. In order to
correctly serve the cross-signed chain on issuance requests, the
`manual_chain` override is required on either or both intermediates. This
can be constructed in the following order:

 - this issuer (`self`)
 - this root
 - the other copy of this intermediate
 - the other root

All requests to this issuer for signing will now present the full cross-signed
chain.

## Keep certificate lifetimes short, for CRL's sake

This secrets engine aligns with Vault's philosophy of short-lived secrets. As
such it is not expected that CRLs will grow large; the only place a private key
is ever returned is to the requesting client (this secrets engine does _not_
store generated private keys, except for CA certificates). In most cases, if the
key is lost, the certificate can simply be ignored, as it will expire shortly.

If a certificate must truly be revoked, the normal Vault revocation function can
be used, and any revocation action will cause the CRL to be regenerated. When
the CRL is regenerated, any expired certificates are removed from the CRL (and
any revoked, expired certificate are removed from secrets engine storage). This
is an expensive operation! Due to the structure of the CRL standard, Vault must
read **all** revoked certificates into memory in order to rebuild the CRL and
clients must fetch the regenerated CRL.

This secrets engine does not support multiple CRL endpoints with sliding date
windows; often such mechanisms will have the transition point a few days apart,
but this gets into the expected realm of the actual certificate validity periods
issued from this secrets engine. A good rule of thumb for this secrets engine
would be to simply not issue certificates with a validity period greater than
your maximum comfortable CRL lifetime. Alternately, you can control CRL caching
behavior on the client to ensure that checks happen more often.

Often multiple endpoints are used in case a single CRL endpoint is down so that
clients don't have to figure out what to do with a lack of response. Run Vault
in HA mode, and the CRL endpoint should be available even if a particular node
is down.

~> Note: Since Vault 1.11.0, with multiple issuers in the same mount point,
   different issuers may have different CRLs (depending on subject and key
   material). This means that Vault may need to regenerate multiple CRLs.
   This is again a rationale for keeping TTLs short and avoiding revocation
   if possible.

~> Note: Since Vault 1.12.0, we support two complementary revocation
   mechanisms: Delta CRLs, which allow for rebuilds of smaller, incremental
   additions to the last complete CRL, and OCSP, which allows responding to
   revocation status requests for individual certificates. When coupled with
   the new CRL auto-rebuild functionality, this means that the revoking step
   isn't as costly (as the CRL isn't always rebuilt on each revocation),
   outside of storage considerations. However, while the rebuild operation
   still can be expensive with lots of certificates, it will be done on a
   schedule rather than on demand.

### NotAfter Behavior on Leaf Certificates

In Vault 1.11.0, the PKI Secrets Engine has introduced a new
`leaf_not_after_behavior` [parameter on
issuers](/api-docs/secret/pki#leaf_not_after_behavior).
This allows modification of the issuance behavior: should Vault `err`,
preventing issuance of a longer-lived leaf cert than issuer, silently
`truncate` to that of the issuer's `NotAfter` value, or `permit` longer
expirations.

It is strongly suggested to use `err` or `truncate` for intermediates;
`permit` is only useful for root certificates, as intermediate's NotAfter
expiration are checked when validating presented chains.

In combination with a cascading expiration with longer lived roots (perhaps
on the range of 2-10 years), shorter lived intermediates (perhaps on the
range of 6 months to 2 years), and short-lived leaf certificates (on the
range of 30 to 90 days), and the [rotation strategies discussed in other
sections](/docs/secrets/pki/rotation-primitives), this should keep the
CRLs adequately small.

### Cluster Performance and Quantity of Leaf Certificates

As mentioned above, keeping TTLs short (or using `no_store=true`) and avoiding
leases is important for a healthy cluster. However it is important to note
this is a scale problem: 10-1000 long-lived, stored certificates are probably
fine, but 50k-100k become a problem and 500k+ stored, unexpired certificates
can negatively impact even large Vault clusters--even with short TTLs!

However, once these certificates are expired, a [tidy operation](/api-docs/secret/pki#tidy)
will clean up CRLs and Vault cluster storage.

Note that organizational risk assessments for certificate compromise might
mean certain certificate types should always be issued with `no_store=false`;
even short-lived broad wildcard certificates (say, `*.example.com`) might be
important enough to have precise control over revocation. However, an internal
service with a well-scoped certificate (say, `service.example.com`) might be
of low enough risk to issue a 90-day TTL with `no_store=true`, preventing
the need for revocation in the unlikely case of compromise.

Having a shorter TTL decreases the likelihood of needing to revoke a cert
(but cannot prevent it entirely) and decrease the impact of any such
compromise.

~> Note: As of Vault 1.12, the PKI Secret Engine's [Bring-Your-Own-Cert
    (BYOC)](/api-docs/secret/pki#revoke-certificate)
   functionality allows revocation of certificates not previously stored
   (e.g., issued via a role with `no_store=true`). This means that setting
   `no_store=true` _is now_ safe to be used globally, regardless of importance
   of issued certificates (and their likelihood for revocation).

## You must configure issuing/CRL/OCSP information _in advance_

This secrets engine serves CRLs from a predictable location, but it is not
possible for the secrets engine to know where it is running. Therefore, you must
configure desired URLs for the issuing certificate, CRL distribution points, and
OCSP servers manually using the `config/urls` endpoint. It is supported to have
more than one of each of these by passing in the multiple URLs as a
comma-separated string parameter.

~> Note: when using Vault Enterprise's Performance Replication features with a
   PKI Secrets Engine mount, each cluster will have its own CRL; this means
   each cluster's unique CRL address should be included in the [AIA
   information](https://datatracker.ietf.org/doc/html/rfc5280#section-5.2.7)
   field separately, or the CRLs should be consolidated and served outside of
   Vault.

~> Note: When using multiple issuers in the same mount, it is suggested to use
   the per-issuer AIA fields rather than the global (`/config/urls`) variant.
   This is for correctness: these fields are used for chain building and
   automatic CRL detection in certain applications. If they point to the wrong
   issuer's information, these applications may break.

## Distribution of CRLs and OCSP

Both CRLs and OCSP allow interrogating revocation status of certificates. Both
of these methods include internal security and authenticity (both CRLs and
OCSP responses are signed by the issuing CA within Vault). This means both are
fine to distribute over non-secure and non-authenticated channels, such as
HTTP.

## Automate Leaf Certificate Renewal

As much as possible, for managing certificates for services at scale, it is
best to automate renewal of certificates. Vault agent [has support for
automatically renewing requested certificates](/docs/agent/template#certificates)
based on the `validTo` field. Other solutions might involve using
[cert-manager](https://cert-manager.io/) in Kubernetes or OpenShift, backed
by the Vault CA.

## Automate CRL Building and Tidying

Since Vault 1.12, the PKI Secrets Engine supports automated CRL rebuilding
(including optional Delta CRLs which can be built more frequently than
complete CRLs) via the `/config/crl` endpoint. Additionally, tidying of
revoked and expired certificates can be configured automatically via the
`/config/auto-tidy` endpoint. Both of these should be enabled to ensure
compatibility with the wider PKIX ecosystem and performance of the cluster.

## Safe Minimums

Since its inception, this secrets engine has enforced SHA256 for signature
hashes rather than SHA1. As of 0.5.1, a minimum of 2048 bits for RSA keys is
also enforced. Software that can handle SHA256 signatures should also be able to
handle 2048-bit keys, and 1024-bit keys are considered unsafe and are disallowed
in the Internet PKI.

## Token Lifetimes and Revocation

When a token expires, it revokes all leases associated with it. This means that
long-lived CA certs need correspondingly long-lived tokens, something that is
easy to forget. Starting with 0.6, root and intermediate CA certs no longer have
associated leases, to prevent unintended revocation when not using a token with
a long enough lifetime. To revoke these certificates, use the `pki/revoke`
endpoint.

## Safe Usage of Roles

The Vault PKI Secrets Engine supports many options to limit issuance via
[Roles](/api-docs/secret/pki#create-update-role).
Careful consideration of construction is necessary to ensure that more
permissions are not given than necessary. Additionally, roles should generally
do _one_ thing; multiple roles should be preferable over having too permissive
roles that allow arbitrary issuance (e.g., `allow_any_name` should generally
be used sparingly, if at all).

 - `allow_any_name` should generally be set to `false`; this is the default.
 - `allow_localhost` should generally be set to `false` for production
   services, unless listening on `localhost` is expected.
 - Unless necessary, `allow_wildcard_certificates` should generally be set to
   `false`. This is **not** the default due to backwards compatibility
   concerns.
   - This is especially necessary when `allow_subdomains` or `allow_glob_domains`
     are enabled.
 - `enforce_hostnames` should generally be enabled for TLS services; this is
   the default.
 - `allow_ip_sans` should generally be set to `false` (but defaults to `true`),
   unless IP address certificates are explicitly required.
 - When using short TTLs (< 30 days) or with high issuance volume, it is
   generally recommend to set `no_store` to `true` (defaults to `false`).
   This prevents revocation but allows higher throughput as Vault no longer
   needs to store every issued certificate. This is discussed more in the
   [Replicated Datasets](#replicated-datasets) section below.
 - Do not use roles with root certificates (`issuer_ref`). Root certificates
   should generally only issue intermediates (see the section on [CA hierarchy
   above](#use-a-ca-hierarchy)), which doesn't rely on roles.
 - Limit `key_usage` and `ext_key_usage`; don't attempt to allow all usages
   for all purposes. Generally the default values are useful for client and
   server TLS authentication.

## Telemetry

Beyond Vault's default telemetry around request processing, PKI exposes count and
duration metrics for the issue, sign, sign-verbatim, and revoke calls. The
metrics keys take the form `mount-path,operation,[failure]` with labels for
namespace and role name.

Note that these metrics are per-node and thus would need to be aggregated across
nodes and clusters.

## Auditing

Because Vault HMACs audit string keys by default, it is necessary to tune
PKI secrets mounts to get an accurate view of issuance that is occurring under
this mount.

~> Note: Depending on usage of Vault, CRLs (and rarely, CA chains) can grow to
   be rather large. We don't recommend un-HMACing the `crl` field for this
   reason, but note that the recommendations below suggest to un-HMAC the
   `certificate` response parameter, which the CRL can be served in via
   the `/pki/cert/crl` API endpoint. Additionally, the `http_raw_body` can
   be used to return CRL both in PEM and raw binary DER form, so it is
   suggested not to un-HMAC that field to not corrupt the log format.<br /><br />
   If this is done with only a [syslog](/docs/audit/syslog) audit device,
   Vault can deny requests (with an opaque `500 Internal Error` message)
   after the action has been performed on the server, because it was
   unable to log the message.<br /><br />
   The suggested workaround is to either leave the `certificate` and `crl`
   response fields HMACed and/or to also enable the [`file`](/docs/audit/file)
   audit log type.

Some suggested keys to un-HMAC for requests are as follows:

 - `csr` - the requested CSR to sign,
 - `certificate` - the requested self-signed certificate to re-sign or
   when importing issuers,
 - Various issuance-related overriding parameters, such as:
   - `issuer_ref` - the issuer requested to sign this certificate,
   - `common_name` - the requested common name,
   - `alt_names` - alternative requested DNS-type SANs for this certificate,
   - `other_sans` - other (non-DNS, non-Email, non-IP, non-URI) requested SANs for this certificate,
   - `ip_sans` - requested IP-type SANs for this certificate,
   - `uri_sans` - requested URI-type SANs for this certificate,
   - `ttl` - requested expiration date of this certificate,
   - `not_after` - requested expiration date of this certificate,
   - `serial_number` - the subject's requested serial number,
   - `key_type` - the requested key type,
   - `private_key_format` - the requested key format which is also
     used for the public certificate format as well,
 - Various role- or issuer-related generation parameters, such as:
   - `managed_key_name` - when creating an issuer, the requested managed
     key name,
   - `managed_key_id` -  when creating an issuer, the requested managed
     key identifier,
   - `ou` - the subject's organizational unit,
   - `organization` - the subject's organization,
   - `country` - the subject's country code,
   - `locality` - the subject's locality,
   - `province` - the subject's province,
   - `street_address` - the subject's street address,
   - `postal_code` - the subject's postal code,
   - `permitted_dns_domains` - permitted DNS domains,
   - `policy_identifiers` - the requested policy identifiers when creating a role, and
   - `ext_key_usage_oids` - the extended key usage OIDs for the requested certificate.

Some suggested keys to un-HMAC for responses are as follows:

 - `certificate` - the certificate that was issued,
 - `issuing_ca` - the certificate of the CA which issued the requested
   certificate,
 - `serial_number` - the serial number of the certificate that was issued,
 - `error` - to show errors associated with the request, and
 - `ca_chain` - optional due to noise; the full CA chain of the issuer of
   the requested certificate.

~> Note: These list of parameters to un-HMAC are provided as a suggestion and
   may not be exhaustive.

The following keys are suggested **NOT** to un-HMAC, due to their sensitive
nature:

 - `private_key` - this response parameter contains the private keys
   generated by Vault during issuance, and
 - `pem_bundle` this request parameter is only used on the issuer-import
   paths and may contain sensitive private key material.

## Role-Based Access

Vault supports [path-based ACL Policies](https://learn.hashicorp.com/tutorials/vault/getting-started-policies)
for limiting access to various paths within Vault.

The following is a condensed example reference of ACLing the PKI Secrets
Engine. These are just a suggestion; other personas and policy approaches
may also be valid.

We suggest the following personas:

 - *Operator*;  a privileged user who manages the health of the PKI
   subsystem; manages issuers and key material.
 - *Agent*; a semi-privileged user that manages roles and handles
   revocation on behalf of an operator; may also handle delegated
   issuance. This may also be called an *administrator* or *role
   manager*.
 - *Advanced*; potentially a power-user or service that has access to
   additional issuance APIs.
 - *Requester*; a low-level user or service that simply requests certificates.
 - *Unauthed*; any arbitrary user or service that lacks a Vault token.

For these personas, we suggest the following ACLs, in condensed, tabular form:

| Path | Operations | Operator | Agent | Advanced | Requester | Unauthed |
| :--- | :--------- | :------- | :---- | :------- | :-------- | :------- |
| `/ca(/pem)?` | Read | Yes | Yes | Yes | Yes | Yes |
| `/ca_chain` | Read | Yes | Yes | Yes | Yes | Yes |
| `/crl(/pem)?` | Read | Yes | Yes | Yes | Yes | Yes |
| `/crl/delta(/pem)?` | Read | Yes | Yes | Yes | Yes | Yes |
| `/cert/:serial(/raw(/pem)?)?` | Read | Yes | Yes | Yes | Yes | Yes |
| `/issuers` | List | Yes | Yes | Yes | Yes | Yes |
| `/issuer/:issuer_ref/(json¦der¦pem)` | Read | Yes | Yes | Yes | Yes | Yes |
| `/issuer/:issuer_ref/crl(/der¦/pem)?` | Read | Yes | Yes | Yes | Yes | Yes |
| `/issuer/:issuer_ref/crl/delta(/der¦/pem)?` | Read | Yes | Yes | Yes | Yes | Yes |
| `/ocsp/<request>` | Read | Yes | Yes | Yes | Yes | Yes |
| `/ocsp` | Write | Yes | Yes | Yes | Yes | Yes |
| `/certs` | List | Yes | Yes | Yes | Yes | |
| `/revoke-with-key` | Write | Yes | Yes | Yes | Yes | |
| `/roles` | List | Yes | Yes | Yes | Yes | |
| `/roles/:role` | Read | Yes | Yes | Yes | Yes | |
| `/(issue¦sign)/:role` | Write | Yes | Yes | Yes | Yes | |
| `/issuer/:issuer_ref/(issue¦sign)/:role` | Write | Yes | Yes | Yes | | |
| `/config/auto-tidy` | Read | Yes | Yes |  |  |  |
| `/config/ca` | Read | Yes | Yes |  |  |  |
| `/config/crl` | Read | Yes | Yes |  |  |  |
| `/config/issuers` | Read | Yes | Yes |  |  |  |
| `/crl/rotate` | Read | Yes | Yes | | | |
| `/crl/rotate-delta` | Read | Yes | Yes | | | |
| `/roles/:role` | Write | Yes | Yes | | | |
| `/issuer/:issuer_ref` | Read | Yes | Yes | | | |
| `/sign-verbatim(/:role)?` | Write | Yes | Yes | | | |
| `/issuer/:issuer_ref/sign-verbatim(/:role)?` | Write | Yes | Yes | | | |
| `/revoke` | Write | Yes | Yes | | | |
| `/tidy` | Write | Yes | Yes | | | |
| `/tidy-cancel` | Write | Yes | Yes | | | |
| `/tidy-status` | Read | Yes | Yes | | | |
| `/config/auto-tidy` | Write | Yes | |  |  |  |
| `/config/ca` | Write | Yes | |  |  |  |
| `/config/crl` | Write | Yes | |  |  |  |
| `/config/issuers` | Write | Yes | |  |  |  |
| `/config/keys` | Read, Write | Yes | |  |  |  |
| `/config/urls` | Read, Write | Yes | |  |  |  |
| `/issuer/:issuer_ref` | Write | Yes | | | | |
| `/issuer/:issuer_ref/revoke` | Write | Yes | | | | |
| `/issuer/:issuer_ref/sign-intermediate` | Write | Yes | | | | |
| `/issuer/issuer_ref/sign-self-issued` | Write | Yes | | | | |
| `/issuers/generate/+/+` | Write | Yes | | | | |
| `/issuers/import/+` | Write | Yes | | | | |
| `/intermediate/generate/+` | Write | Yes | | | | |
| `/intermediate/cross-sign` | Write | Yes | | | | |
| `/intermediate/set-signed` | Write | Yes | | | | |
| `/keys` | List | Yes | | | | |
| `/key/:key_ref` | Read, Write | Yes | | | | |
| `/keys/generate/+` | Write | Yes | | | | |
| `/keys/import` | Write | Yes | | | | |
| `/root/generate/+` | Write | Yes | | | | |
| `/root/sign-intermediate` | Write | Yes | | | | |
| `/root/sign-self-issued` | Write | Yes | | | | |
| `/root/rotate/+` | Write | Yes | | | | |
| `/root/replace` | Write | Yes | | | | |

~> Note: With managed keys, operators might need access to [read the mount
   point's tunable data](/api-docs/system/mounts) (Read on `/sys/mounts`) and
   may need access [to use or manage managed keys](/api-docs/system/managed-keys).

## Replicated DataSets

When operating with [Performance Secondary](/docs/enterprise/replication#architecture)
clusters, certain data-sets are maintained across all clusters, while others for performance
and scalability reasons are kept within a given cluster.

The following table breaks down by data type what data sets will cross the cluster boundaries.
For data-types that do not cross a cluster boundary, read requests for that data will need to be
sent to the appropriate cluster that the data was generated on.

| Data Set                 | Replicated Across Clusters |
|--------------------------|----------------------------|
| Issuers & Keys           | Yes                        |
| Roles                    | Yes                        |
| CRL Config               | Yes                        |
| URL Config               | Yes                        |
| Issuer Config            | Yes                        |
| Key Config               | Yes                        |
| CRL                      | No                         |
| Revoked Certificates     | No                         |
| Leaf/Issued Certificates | No                         |

The main effect is that within the PKI secrets engine leaf certificates
issued with `no_store` set to `false` are stored local to the cluster that issued them.
This allows for both primary and [Performance Secondary](/docs/enterprise/replication#architecture)
clusters' active node to issue certificates for greater scalability. As a
result, these certificates and any revocations are visible only on the issuing
cluster. This additionally means each cluster has its own set of CRLs, distinct
from other clusters. These CRLs should either be unified into a single CRL for
distribution from a single URI, or server operators should know to fetch all
CRLs from all clusters.

## Cluster Scalability

Most non-introspection operations in the PKI secrets engine require a write to
storage, and so are forwarded to the cluster's active node for execution.
This table outlines which operations can be executed on performance standby nodes
and thus scale horizontally across all nodes within a cluster.

| Path                          | Operations           |
|-------------------------------|----------------------|
| ca[/pem]                      | Read                 |
| cert/<em>serial-number</em>   | Read                 |
| cert/ca_chain                 | Read                 |
| config/crl                    | Read                 |
| certs                         | List                 |
| ca_chain                      | Read                 |
| crl[/pem]                     | Read                 |
| issue                         | Update <sup>\*</sup> |
| revoke/<em>serial-number</em> | Read                 |
| sign                          | Update <sup>\*</sup> |
| sign-verbatim                 | Update <sup>\*</sup> |

\* Only if the corresponding role has `no_store` set to true and `generate_lease`
set to false. If `generate_lease` is true the lease creation will be forwarded to
the active node; if `no_store` is false the entire request will be forwarded to
the active node.

## PSS Support

Go lacks support for PSS certificates, keys, and CSRs using the `rsaPSS` OID
(`1.2.840.113549.1.1.10`). It requires all RSA certificates, keys, and CSRs
to use the alternative `rsaEncryption` OID (`1.2.840.113549.1.1.1`).

When using OpenSSL to generate CAs or CSRs from PKCS8-encoded PSS keys, the
resulting CAs and CSRs will have the `rsaPSS` OID. Go and Vault will reject
them. Instead, use OpenSSL to generate or convert to a PKCS#1v1.5 private
key file and use this to generate the CSR. Vault will, depending on the role
and the signing mechanism, still use a PSS signature despite the
`rsaEncryption` OID on the request as the SubjectPublicKeyInfo and
SignatureAlgorithm fields are orthogonal. When creating an external CA and
importing it into Vault, ensure that the `rsaEncryption` OID is present on
the SubjectPublicKeyInfo field even if the SignatureAlgorithm is PSS-based.

These certificates generated by Go (with `rsaEncryption` OID but PSS-based
signatures) are otherwise compatible with the fully PSS-based certificates.
OpenSSL and NSS support parsing and verifying chains using this type of
certificate. Note that some TLS implementations may not support these types
of certificates if they do not support `rsa_pss_rsae_*` signature schemes.
Additionally, some implementations allow rsaPSS OID certificates to contain
restrictions on signature parameters allowed by this certificate, but Go and
Vault do not support adding such restrictions.

At this time Go lacks support for signing CSRs with the PSS signature
algorithm. If using a managed key that requires a RSA PSS algorithm (such as GCP or
a PKCS#11 HSM) as a backing for an intermediate CA key, attempting to generate
a CSR (via `pki/intermediate/generate/kms`) will fail signature verification.
In this case, the CSR will need to be generated outside of Vault and the
signed final certificate can be imported into the mount.

Go additionally lacks support for creating OCSP responses with the PSS
signature algorithm. Vault will automatically downgrade issuers with
PSS-based revocation signature algorithms to PKCS#1v1.5, but note that
certain KMS devices (like HSMs and GCP) may not support this with the
same key. As a result, the OCSP responder may fail to sign responses,
returning an internal error.

## Issuer Subjects and CRLs

As noted on several [GitHub issues](https://github.com/hashicorp/vault/issues/10176),
Go's x509 library has an opinionated parsing and structuring mechanism for
certificate's Subjects. Issuers created within Vault are fine, but when using
externally created CA certificates, note that these may not be parsed
correctly throughout all parts of the PKI. In particular, CRLs embed a
(modified) copy of the issuer name. This can be avoided by using OCSP to
track revocation, but note that performance characteristics are different
between OCSP and CRLs.

## Issuer Storage Migration Issues

When Vault migrates to the new multi-issuer storage layout on releases prior
to 1.11.6, 1.12.2, and 1.13, and storage write errors occur during the mount
initialization and storage migration process, the default issuer _may_ not
have the correct `ca_chain` value and may only have the self-reference. These
write errors most commonly manifest in logs as a message like
`failed to persist issuer ... chain to disk: <cause>` and indicate that Vault
was not stable at the time of migration. Note that this only occurs when more
than one issuer exists within the mount (such as an intermediate with root).

To fix this manually (until a new version of Vault automatically rebuilds the
issuer chain), a rebuild of the chains can be performed:

```
curl -X PATCH -H "Content-Type: application/merge-patch+json" -H "X-Vault-Request: true" -H "X-Vault-Token: $(vault print token)" -d '{"manual_chain":"self"}' https://.../issuer/default
curl -X PATCH -H "Content-Type: application/merge-patch+json" -H "X-Vault-Request: true" -H "X-Vault-Token: $(vault print token)" -d '{"manual_chain":""}' https://.../issuer/default
```

This temporarily sets the manual chain on the default issuer to a self-chain
only, before reverting it back to automatic chain building. This triggers a
refresh of the `ca_chain` field on the issuer, and can be verified with:

```
vault read pki/issuer/default
```

## Tutorial

Refer to the [Build Your Own Certificate Authority (CA)](https://learn.hashicorp.com/vault/secrets-management/sm-pki-engine)
guide for a step-by-step tutorial.

Have a look at the [PKI Secrets Engine with Managed Keys](https://learn.hashicorp.com/tutorials/vault/managed-key-pki?in=vault/enterprise)
for more about how to use externally managed keys with PKI.

## API

The PKI secrets engine has a full HTTP API. Please see the
[PKI secrets engine API](/api-docs/secret/pki) for more
details.