dfdeca7b5d
The TLS settings should not need to be modified as xks-proxy should generate the certificate and key itself for listening.
285 lines
11 KiB
Plaintext
285 lines
11 KiB
Plaintext
---
|
||
layout: docs
|
||
page_title: AWS KMS External Key Store (XKS) - PKCS#11 Provider - Vault Enterprise
|
||
description: |-
|
||
AWS KMS External Key Store can use Vault as a key store via the Vault PKCS#11 Provider.
|
||
---
|
||
|
||
# Vault with AWS KMS External Key Store (XKS) via PKCS#11 and XKS Proxy
|
||
|
||
~> **Note**: AWS [`xks-proxy`](https://github.com/aws-samples/aws-kms-xks-proxy) is used in this document as a sample implementation.
|
||
|
||
Vault's KMIP Secrets Engine can be used as an external key store for the AWS KMS [External Key Store (XKS)](https://aws.amazon.com/blogs/aws/announcing-aws-kms-external-key-store-xks/) protocol using the AWS [`xks-proxy`](https://github.com/aws-samples/aws-kms-xks-proxy) along
|
||
with the [Vault PKCS#11 Provider](/vault/docs/enterprise/pkcs11-provider).
|
||
|
||
## Overview
|
||
|
||
This is tested as working with Vault 1.11.0 Enterprise (and later) with Advanced Data Protection (KMIP support).
|
||
|
||
Prerequisites:
|
||
|
||
* A server capable of running XKS Proxy on port 443, which is exposed to the Internet or a VPC endpoint. This can be the same as the Vault server.
|
||
* A valid DNS entry with a valid TLS certificate for XKS Proxy.
|
||
* `libvault-pkcs11.so` downloaded from [releases.hashicorp.com](https://releases.hashicorp.com/vault-pkcs11-provider) for your platform and available on the XKS Proxy server.
|
||
* Vault Enterprise with the KMIP Secrets Engine available and with TCP port 5696 accessible to where XKS Proxy will be running.
|
||
|
||
There are 3 parts to this setup:
|
||
|
||
1. Vault KMIP Secrets Engine standard setup. (There is nothing specific to XKS in this setup.)
|
||
1. Vault PKCS#11 setup to tell the PKCS#11 provider (`libvault-pkcs11.so`) how to talk to the Vault KMIP Secrets Engine. (There is nothing specific to XKS in this setup.)
|
||
1. XKS Proxy setup.
|
||
|
||
~> **Important**: XKS has a strict 250 ms latency requirement.
|
||
In order to serve requests with this latency, we recommend hosting Vault and the XKS proxy as close as possible
|
||
to the desired KMS region.
|
||
|
||
## Vault Setup
|
||
|
||
On the Vault server, we need to [setup the KMIP Secrets Engine](/vault/docs/secrets/kmip):
|
||
|
||
1. Start the [KMIP Secrets Engine](/vault/docs/secrets/kmip) and listener:
|
||
|
||
```sh
|
||
vault secrets enable kmip
|
||
vault write kmip/config listen_addrs=0.0.0.0:5696
|
||
```
|
||
|
||
1. Create a KMIP scope to contain the AES keys that will be accessible.
|
||
The KMIP scope is essentially an isolated namespace.
|
||
Here is an example creating one called `my-service` (which will be used throughout this document).
|
||
|
||
```sh
|
||
vault write -f kmip/scope/my-service
|
||
```
|
||
|
||
1. Create a KMIP role that has access to the scope:
|
||
|
||
```sh
|
||
vault write kmip/scope/my-service/role/admin operation_all=true
|
||
```
|
||
|
||
1. Create TLS credentials (a certificate, key, and CA bundle) for the KMIP role:
|
||
|
||
~> **Note**: This command will output the credentials in plaintext.
|
||
|
||
```sh
|
||
vault write -f -format=json kmip/scope/my-service/role/admin/credential/generate | tee kmip.json
|
||
```
|
||
|
||
The response from the `credential/generate` endpoint is JSON.
|
||
The `.data.certificate` entry contains a bundle of the TLS client key and certificate we will use to connect to KMIP with from `xks-proxy`.
|
||
The `.data.ca_chain[]` entries contain the CA bundle to verify the KMIP server's certificate.
|
||
Save these to, e.g., `cert.pem` and `ca.pem`:
|
||
|
||
```sh
|
||
jq --raw-output --exit-status '.data.ca_chain[]' kmip.json > ca.pem
|
||
jq --raw-output --exit-status '.data.certificate' kmip.json > cert.pem
|
||
```
|
||
|
||
## XKS Proxy Setup
|
||
|
||
The rest of the steps take place on the XKS Proxy server.
|
||
|
||
For this example, We will use an HTTPS proxy service like [ngrok](https://ngrok.com/) to forward connections
|
||
to the XKS proxy. This helps to quickly setup a valid domain and TLS endpoint for testing.
|
||
|
||
1. Start `ngrok`:
|
||
|
||
```shell-session
|
||
$ ngrok http 8000
|
||
```
|
||
|
||
This will output a domain that can be used to configure KMS later, such as `https://example.ngrok.io`.
|
||
|
||
1. Copy the `libvault-pkcs11.so` binary to the server, such as `/usr/local/lib` (should be same as in the TOML config file below), and `chmod` it so that it is executable.
|
||
|
||
1. Copy the TLS certificate bundle (e.g., `/etc/kmip/cert.pem`) and CA bundle (e.g., `/etc/kmip/ca.pem`) to the `xks-proxy` server (doesn't matter where, as long as the `xks-proxy` process has access to it) from the Vault setup.
|
||
|
||
1. Create a `configuration/settings_vault.toml` file for the XKS to Vault PKCS#11 configuration,
|
||
and set the `XKS_PROXY_SETTINGS_TOML` environment variable to point to the file location.
|
||
|
||
The important settings to change:
|
||
|
||
* `[[external_key_stores]]`:
|
||
* change URI path prefix to anything you like
|
||
* choose random access ID
|
||
* choose random secret key
|
||
* set which key labels are accessible to XKS (`xks_key_id_set`)
|
||
* `[pkcs11]`: set the `PKCS11_HSM_MODULE` to the location of the `libvault-pkcs11.so` (or `.dylib`) file downloaded from [releases.hashicorp.com](https://releases.hashicorp.com/vault-pkcs11-provider).
|
||
|
||
```toml
|
||
[server]
|
||
ip = "0.0.0.0"
|
||
port = 8000
|
||
region = "us-east-2"
|
||
service = "kms-xks-proxy"
|
||
|
||
[server.tcp_keepalive]
|
||
tcp_keepalive_secs = 60
|
||
tcp_keepalive_retries = 3
|
||
tcp_keepalive_interval_secs = 1
|
||
|
||
[tracing]
|
||
is_stdout_writer_enabled = true
|
||
is_file_writer_enabled = true
|
||
level = "DEBUG"
|
||
directory = "/var/local/xks-proxy/logs"
|
||
file_prefix = "xks-proxy.log"
|
||
rotation_kind = "HOURLY"
|
||
|
||
[security]
|
||
is_sigv4_auth_enabled = true
|
||
is_tls_enabled = true
|
||
is_mtls_enabled = false
|
||
|
||
[tls]
|
||
tls_cert_pem = "tls/server_cert.pem"
|
||
tls_key_pem = "tls/server_key.pem"
|
||
mtls_client_ca_pem = "tls/client_ca.pem"
|
||
mtls_client_dns_name = "us-east-2.alpha.cks.kms.aws.internal.amazonaws.com"
|
||
|
||
[[external_key_stores]]
|
||
uri_path_prefix = "/xyz"
|
||
sigv4_access_key_id = "AKIA4GBY3I6JCE5M2HPM"
|
||
sigv4_secret_access_key = "1234567890123456789012345678901234567890123="
|
||
xks_key_id_set = ["abc123"]
|
||
|
||
[pkcs11]
|
||
session_pool_max_size = 30
|
||
session_pool_timeout_milli = 0
|
||
session_eager_close = false
|
||
user_pin = ""
|
||
PKCS11_HSM_MODULE = "/usr/local/lib/libvault-pkcs11.so"
|
||
context_read_timeout_milli = 100
|
||
|
||
[limits]
|
||
max_plaintext_in_base64 = 8192
|
||
max_aad_in_base64 = 16384
|
||
|
||
[hsm_capabilities]
|
||
can_generate_iv = false
|
||
is_zero_iv_required = false
|
||
```
|
||
|
||
~> **Note**: `vault-pkcs11-provider` versions of 0.1.0–0.1.2 require the last two lines to be changed to `can_generate_iv = true` and `is_zero_iv_required = true`.
|
||
|
||
1. Create a file, `/etc/vault-pkcs11.hcl` with the following contents:
|
||
|
||
```hcl
|
||
slot {
|
||
server = "VAULT_ADDRESS:5696"
|
||
tls_cert_path = "/etc/kmip/cert.pem"
|
||
ca_path = "/etc/kmip/ca.pem"
|
||
scope = "my-service"
|
||
}
|
||
```
|
||
|
||
This file is used by `libvault-pkcs11.so` to know how to find and communicate with the KMIP server.
|
||
See [the Vault docs](/vault/docs/enterprise/pkcs11-provider) for all available parameters and their usage.
|
||
|
||
1. If you want to view the Vault logs (helpful when trying to find error messages), you can specify the `VAULT_LOG_FILE` (default is stdout) and `VAULT_LOG_LEVEL` (default is `INFO`). We'd recommend setting `VAULT_LOG_FILE` to something like `/tmp/vault.log` or `/var/log/vault.log`. Other useful log levels are `WARN` (quieter) and `TRACE` (very verbose, could possibly contain sensitive information, like raw network packets).
|
||
|
||
1. Create an AES-256 key in KMIP, for example, using `pkcs11-tool` (usually installed with the OpenSC package). See the [Vault docs](/vault/docs/enterprise/pkcs11-provider) for the full setup.
|
||
```sh
|
||
VAULT_LOG_FILE=/dev/null pkcs11-tool --module ./libvault-pkcs11.so --keygen -a abc123 --key-type AES:32 \
|
||
--extractable --allow-sw
|
||
Key generated:
|
||
Secret Key Object; AES length 32
|
||
VALUE:
|
||
label: abc123
|
||
Usage: encrypt, decrypt, wrap, unwrap
|
||
Access: none
|
||
```
|
||
|
||
## Enable XKS in the AWS CLI
|
||
|
||
|
||
1. Create the KMS custom key store with the appropriate parameters to point to your XKS proxy (in this example, through `ngrok`).
|
||
|
||
```shell-session
|
||
$ aws kms create-custom-key-store \
|
||
--custom-key-store-name myVaultKeyStore \
|
||
--custom-key-store-type EXTERNAL_KEY_STORE \
|
||
--xks-proxy-uri-endpoint https://example.ngrok.io \
|
||
--xks-proxy-uri-path /xyz/kms/xks/v1 \
|
||
--xks-proxy-authentication-credential AccessKeyId=AKIA4GBY3I6JCE5M2HPM,RawSecretAccessKey=1234567890123456789012345678901234567890123= \
|
||
--xks-proxy-connectivity PUBLIC_ENDPOINT
|
||
|
||
{
|
||
"CustomKeyStoreId": "cks-d7a55fe93d63191d6"
|
||
}
|
||
```
|
||
|
||
1. Tell KMS to connect to the key store.
|
||
|
||
```shell-session
|
||
$ aws kms connect-custom-key-store --custom-key-store-id cks-d7a55fe93d63191d6
|
||
```
|
||
|
||
1. Wait for the `ConnectionState` of your custom key store to be `CONNECTED`. This can take a few minutes.
|
||
|
||
```shell-session
|
||
$ aws kms describe-custom-key-stores --custom-key-store-id cks-d7a55fe93d63191d6
|
||
```
|
||
|
||
1. Create a KMS key associated with the XKS key ID (`abc123` in this example):
|
||
|
||
```shell-session
|
||
$ aws kms create-key --custom-key-store-id cks-d7a55fe93d63191d6 \
|
||
--xks-key-id abc123 --origin EXTERNAL_KEY_STORE
|
||
{
|
||
"KeyMetadata": {
|
||
"AWSAccountId": "111111111111",
|
||
"KeyId": "a93f205a-2a37-4338-aa64-92b4a4b0b67d",
|
||
"Arn": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
|
||
"CreationDate": "2022-12-22T11:03:23.695000-08:00",
|
||
"Enabled": true,
|
||
"Description": "",
|
||
"KeyUsage": "ENCRYPT_DECRYPT",
|
||
"KeyState": "Enabled",
|
||
"Origin": "EXTERNAL_KEY_STORE",
|
||
"CustomKeyStoreId": "cks-16460f66b34705025",
|
||
"KeyManager": "CUSTOMER",
|
||
"CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
|
||
"KeySpec": "SYMMETRIC_DEFAULT",
|
||
"EncryptionAlgorithms": [
|
||
"SYMMETRIC_DEFAULT"
|
||
],
|
||
"MultiRegion": false,
|
||
"XksKeyConfiguration": {
|
||
"Id": "abc123"
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
1. Encrypt some data with this key:
|
||
|
||
```shell-session
|
||
$ aws kms encrypt --key-id a93f205a-2a37-4338-aa64-92b4a4b0b67d --plaintext YWJjMTIzCg==
|
||
{
|
||
"CiphertextBlob": "somerandomciphertextblob=",
|
||
"KeyId": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
|
||
"EncryptionAlgorithm": "SYMMETRIC_DEFAULT"
|
||
}
|
||
|
||
1. Decypt the resulting ciphertext:
|
||
|
||
```shell-session
|
||
$ aws kms decrypt --ciphertext-blob somerandomciphertextblob=
|
||
{
|
||
"KeyId": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
|
||
"Plaintext": "YWJjMTIzCg==",
|
||
"EncryptionAlgorithm": "SYMMETRIC_DEFAULT"
|
||
}
|
||
```
|
||
|
||
1. Optionally, clean up your key and key store with:
|
||
|
||
```shell-session
|
||
$ aws kms disable-key --key-id a93f205a-2a37-4338-aa64-92b4a4b0b67d
|
||
$ aws kms disconnect-custom-key-store --custom-key-store-id cks-16460f66b34705025
|
||
$ aws kms delete-custom-key-store --custom-key-store-id cks-16460f66b34705025
|
||
```
|
||
|
||
(The `aws kms delete-custom-key-store` command will not succeed until all keys in the key store have been disabled and deleted.) |