open-vault/website/content/docs/enterprise/pkcs11-provider/aws-xks.mdx

---
layout: docs
page_title: AWS KMS External Key Store (XKS) - PKCS#11 Provider - Vault Enterprise
description: |-
  AWS KMS External Key Store can use Vault as a key store via the Vault PKCS#11 Provider.
---

# Vault with AWS KMS External Key Store (XKS) via PKCS#11 and XKS Proxy

~> **Note**: AWS [`xks-proxy`](https://github.com/aws-samples/aws-kms-xks-proxy) is used in this document as a sample implementation.

Vault's KMIP Secrets Engine can be used as an external key store for the AWS KMS [External Key Store (XKS)](https://aws.amazon.com/blogs/aws/announcing-aws-kms-external-key-store-xks/) protocol using the AWS [`xks-proxy`](https://github.com/aws-samples/aws-kms-xks-proxy) along
with the [Vault PKCS#11 Provider](https://developer.hashicorp.com/vault/docs/enterprise/pkcs11-provider).

## Overview

This is tested as working with Vault 1.11.0 Enterprise (and later) with Advanced Data Protection (KMIP support).

Prerequisites:

* A server capable of running XKS Proxy on port 443, which is exposed to the Internet or a VPC endpoint. This can be the same as the Vault server.
* A valid DNS entry with a valid TLS certificate for XKS Proxy.
* `libvault-pkcs11.so` downloaded from [releases.hashicorp.com](https://releases.hashicorp.com/vault-pkcs11-provider) for your platform and available on the XKS Proxy server.
* Vault Enterprise with the KMIP Secrets Engine available and with TCP port 5696 accessible to where XKS Proxy will be running.

There are 3 parts to this setup:

1. Vault KMIP Secrets Engine standard setup. (There is nothing specific to XKS in this setup.)
1. Vault PKCS#11 setup to tell the PKCS#11 provider (`libvault-pkcs11.so`) how to talk to the Vault KMIP Secrets Engine. (There is nothing specific to XKS in this setup.)
1. XKS Proxy setup.

~> **Important**: XKS has a strict 250 ms latency requirement.
In order to serve requests with this latency, we recommend hosting Vault and the XKS proxy as close as possible
to the desired KMS region.

## Vault Setup

On the Vault server, we need to [setup the KMIP Secrets Engine](/docs/secrets/kmip):

1. Start the [KMIP Secrets Engine](/docs/secrets/kmip) and listener:

    ```sh
    vault secrets enable kmip
    vault write kmip/config listen_addrs=0.0.0.0:5696
    ```

1. Create a KMIP scope to contain the AES keys that will be accessible.
   The KMIP scope is essentially an isolated namespace.
   Here is an example creating one called `my-service` (which will be used throughout this document).

    ```sh
    vault write -f kmip/scope/my-service
    ```

1. Create a KMIP role that has access to the scope:

    ```sh
    vault write kmip/scope/my-service/role/admin operation_all=true
    ```

1. Create TLS credentials (a certificate, key, and CA bundle) for the KMIP role:

    ~> **Note**: This command will output the credentials in plaintext.

    ```sh
    vault write -f -format=json kmip/scope/my-service/role/admin/credential/generate | tee kmip.json
    ```

    The response from the `credential/generate` endpoint is JSON.
    The `.data.certificate` entry contains a bundle of the TLS client key and certificate we will use to connect to KMIP with from `xks-proxy`.
    The `.data.ca_chain[]` entries contain the CA bundle to verify the KMIP server's certificate.
    Save these to, e.g., `cert.pem` and `ca.pem`:

    ```sh
    jq --raw-output --exit-status '.data.ca_chain[]' kmip.json > ca.pem
    jq --raw-output --exit-status '.data.certificate' kmip.json > cert.pem
    ```

## XKS Proxy Setup

The rest of the steps take place on the XKS Proxy server.

For this example, We will use an HTTPS proxy service like [ngrok](https://ngrok.com/) to forward connections
to the XKS proxy. This helps to quickly setup a valid domain and TLS endpoint for testing.

1. Start `ngrok`:

   ```shell-session
   $ ngrok http 8000
   ```

   This will output a domain that can be used to configure KMS later, such as `https://example.ngrok.io`.

1. Copy the `libvault-pkcs11.so` binary to the server, such as `/usr/local/lib` (should be same as in the TOML config file below), and `chmod` it so that it is executable.

1. Copy the TLS certificate bundle (e.g., `/etc/kmip/cert.pem`) and CA bundle (e.g., `/etc/kmip/ca.pem`) to the `xks-proxy` server (doesn't matter where, as long as the `xks-proxy` process has access to it) from the Vault setup.

1. Create a `configuration/settings_vault.toml` file for the XKS to Vault PKCS#11 configuration,
   and set the `XKS_PROXY_SETTINGS_TOML` environment variable to point to the file location.

   The important settings to change:

   * `[tls]`: change key and certificate location to point to KMIP certs
   * `[[external_key_stores]]`:
     * change URI path prefix to anything you like
     * choose random access ID
     * choose random secret key
     * set which key labels are accessible to XKS (`xks_key_id_set`)
   * `[pkcs11]`: set the `PKCS11_HSM_MODULE` to the location of the `libvault-pkcs11.so` (or `.dylib`) file downloaded from [releases.hashicorp.com](https://releases.hashicorp.com/vault-pkcs11-provider).

   ```toml
   [server]
   ip = "0.0.0.0"
   port = 8000
   region = "us-east-2"
   service = "kms-xks-proxy"

   [server.tcp_keepalive]
   tcp_keepalive_secs = 60
   tcp_keepalive_retries = 3
   tcp_keepalive_interval_secs = 1

   [tracing]
   is_stdout_writer_enabled = true
   is_file_writer_enabled = true
   level = "DEBUG"
   directory = "/var/local/xks-proxy/logs"
   file_prefix = "xks-proxy.log"
   rotation_kind = "HOURLY"

   [security]
   is_sigv4_auth_enabled = true
   is_tls_enabled = true
   is_mtls_enabled = false

   [tls]
   tls_cert_pem = "tls/server_cert.pem"
   tls_key_pem = "tls/server_key.pem"
   mtls_client_ca_pem = "tls/client_ca.pem"
   mtls_client_dns_name = "us-east-2.alpha.cks.kms.aws.internal.amazonaws.com"

   [[external_key_stores]]
   uri_path_prefix = "/xyz"
   sigv4_access_key_id = "AKIA4GBY3I6JCE5M2HPM"
   sigv4_secret_access_key = "1234567890123456789012345678901234567890123="
   xks_key_id_set = ["abc123"]

   [pkcs11]
   session_pool_max_size = 30
   session_pool_timeout_milli = 0
   session_eager_close = false
   user_pin = ""
   PKCS11_HSM_MODULE = "/usr/local/lib/libvault-pkcs11.so"
   context_read_timeout_milli = 100

   [limits]
   max_plaintext_in_base64 = 8192
   max_aad_in_base64 = 16384

   [hsm_capabilities]
   can_generate_iv = true
   is_zero_iv_required = true
   ```

~> **Note**: This uses an IV of zero until the next release of `vault-pkcs11-provider`.

1. Create a file, `/etc/vault-pkcs11.hcl` with the following contents:

    ```hcl
    slot {
      server = "VAULT_ADDRESS:5696"
      tls_cert_path = "/etc/kmip/cert.pem"
      ca_path = "/etc/kmip/ca.pem"
      scope = "my-service"
    }
    ```

    This file is used by `libvault-pkcs11.so` to know how to find and communicate with the KMIP server.
    See [the Vault docs](https://developer.hashicorp.com/vault/docs/enterprise/pkcs11-provider) for all available parameters and their usage.

1. If you want to view the Vault logs (helpful when trying to find error messages), you can specify the `VAULT_LOG_FILE` (default is stdout) and `VAULT_LOG_LEVEL` (default is `INFO`). We'd recommend setting `VAULT_LOG_FILE` to something like `/tmp/vault.log` or `/var/log/vault.log`. Other useful log levels are `WARN` (quieter) and `TRACE` (very verbose, could possibly contain sensitive information, like raw network packets).

1. Create an AES-256 key in KMIP, for example, using `pkcs11-tool` (usually installed with the OpenSC package). See the [Vault docs](https://developer.hashicorp.com/vault/docs/enterprise/pkcs11-provider) for the full setup.
   ```sh
   VAULT_LOG_FILE=/dev/null pkcs11-tool --module ./libvault-pkcs11.so --keygen -a abc123 --key-type AES:32 \
       --extractable --allow-sw
   Key generated:
   Secret Key Object; AES length 32
   VALUE:
   label:      abc123
   Usage:      encrypt, decrypt, wrap, unwrap
   Access:     none
   ```

## Enable XKS in the AWS CLI


1. Create the KMS custom key store with the appropriate parameters to point to your XKS proxy (in this example, through `ngrok`).

   ```shell-session
   $ aws kms create-custom-key-store \
       --custom-key-store-name myVaultKeyStore \
       --custom-key-store-type EXTERNAL_KEY_STORE \
       --xks-proxy-uri-endpoint https://example.ngrok.io \
       --xks-proxy-uri-path /xyz/kms/xks/v1 \
       --xks-proxy-authentication-credential AccessKeyId=AKIA4GBY3I6JCE5M2HPM,RawSecretAccessKey=1234567890123456789012345678901234567890123= \
       --xks-proxy-connectivity PUBLIC_ENDPOINT

   {
       "CustomKeyStoreId": "cks-d7a55fe93d63191d6"
   }
   ```

1. Tell KMS to connect to the key store.

   ```shell-session
   $ aws kms connect-custom-key-store --custom-key-store-id cks-d7a55fe93d63191d6
   ```

1. Wait for the `ConnectionState` of your custom key store to be `CONNECTED`. This can take a few minutes.

   ```shell-session
   $ aws kms describe-custom-key-stores --custom-key-store-id cks-d7a55fe93d63191d6
   ```

1. Create a KMS key associated with the XKS key ID (`abc123` in this example):

   ```shell-session
   $ aws kms create-key --custom-key-store-id cks-d7a55fe93d63191d6 \
       --xks-key-id abc123 --origin EXTERNAL_KEY_STORE
   {
       "KeyMetadata": {
           "AWSAccountId": "111111111111",
           "KeyId": "a93f205a-2a37-4338-aa64-92b4a4b0b67d",
           "Arn": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
           "CreationDate": "2022-12-22T11:03:23.695000-08:00",
           "Enabled": true,
           "Description": "",
           "KeyUsage": "ENCRYPT_DECRYPT",
           "KeyState": "Enabled",
           "Origin": "EXTERNAL_KEY_STORE",
           "CustomKeyStoreId": "cks-16460f66b34705025",
           "KeyManager": "CUSTOMER",
           "CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
           "KeySpec": "SYMMETRIC_DEFAULT",
           "EncryptionAlgorithms": [
               "SYMMETRIC_DEFAULT"
           ],
           "MultiRegion": false,
           "XksKeyConfiguration": {
               "Id": "abc123"
           }
       }
   }
   ```

1. Encrypt some data with this key:

   ```shell-session
   $ aws kms encrypt --key-id a93f205a-2a37-4338-aa64-92b4a4b0b67d --plaintext YWJjMTIzCg==
   {
       "CiphertextBlob": "somerandomciphertextblob=",
       "KeyId": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
       "EncryptionAlgorithm": "SYMMETRIC_DEFAULT"
   }

1. Decypt the resulting ciphertext:

   ```shell-session
   $ aws kms decrypt --ciphertext-blob somerandomciphertextblob=
   {
       "KeyId": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
       "Plaintext": "YWJjMTIzCg==",
       "EncryptionAlgorithm": "SYMMETRIC_DEFAULT"
   }
   ```

1. Optionally, clean up your key and key store with:

   ```shell-session
   $ aws kms disable-key --key-id a93f205a-2a37-4338-aa64-92b4a4b0b67d
   $ aws kms disconnect-custom-key-store --custom-key-store-id cks-16460f66b34705025
   $ aws kms delete-custom-key-store --custom-key-store-id cks-16460f66b34705025
   ```

   (The `aws kms delete-custom-key-store` command will not succeed until all keys in the key store have been disabled and deleted.)