286 lines
11 KiB
Plaintext
286 lines
11 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: AWS KMS External Key Store (XKS) - PKCS#11 Provider - Vault Enterprise
|
|
description: |-
|
|
AWS KMS External Key Store can use Vault as a key store via the Vault PKCS#11 Provider.
|
|
---
|
|
|
|
# Vault with AWS KMS External Key Store (XKS) via PKCS#11 and XKS Proxy
|
|
|
|
~> **Note**: AWS [`xks-proxy`](https://github.com/aws-samples/aws-kms-xks-proxy) is used in this document as a sample implementation.
|
|
|
|
Vault's KMIP Secrets Engine can be used as an external key store for the AWS KMS [External Key Store (XKS)](https://aws.amazon.com/blogs/aws/announcing-aws-kms-external-key-store-xks/) protocol using the AWS [`xks-proxy`](https://github.com/aws-samples/aws-kms-xks-proxy) along
|
|
with the [Vault PKCS#11 Provider](https://developer.hashicorp.com/vault/docs/enterprise/pkcs11-provider).
|
|
|
|
## Overview
|
|
|
|
This is tested as working with Vault 1.11.0 Enterprise (and later) with Advanced Data Protection (KMIP support).
|
|
|
|
Prerequisites:
|
|
|
|
* A server capable of running XKS Proxy on port 443, which is exposed to the Internet or a VPC endpoint. This can be the same as the Vault server.
|
|
* A valid DNS entry with a valid TLS certificate for XKS Proxy.
|
|
* `libvault-pkcs11.so` downloaded from [releases.hashicorp.com](https://releases.hashicorp.com/vault-pkcs11-provider) for your platform and available on the XKS Proxy server.
|
|
* Vault Enterprise with the KMIP Secrets Engine available and with TCP port 5696 accessible to where XKS Proxy will be running.
|
|
|
|
There are 3 parts to this setup:
|
|
|
|
1. Vault KMIP Secrets Engine standard setup. (There is nothing specific to XKS in this setup.)
|
|
1. Vault PKCS#11 setup to tell the PKCS#11 provider (`libvault-pkcs11.so`) how to talk to the Vault KMIP Secrets Engine. (There is nothing specific to XKS in this setup.)
|
|
1. XKS Proxy setup.
|
|
|
|
~> **Important**: XKS has a strict 250 ms latency requirement.
|
|
In order to serve requests with this latency, we recommend hosting Vault and the XKS proxy as close as possible
|
|
to the desired KMS region.
|
|
|
|
## Vault Setup
|
|
|
|
On the Vault server, we need to [setup the KMIP Secrets Engine](/docs/secrets/kmip):
|
|
|
|
1. Start the [KMIP Secrets Engine](/docs/secrets/kmip) and listener:
|
|
|
|
```sh
|
|
vault secrets enable kmip
|
|
vault write kmip/config listen_addrs=0.0.0.0:5696
|
|
```
|
|
|
|
1. Create a KMIP scope to contain the AES keys that will be accessible.
|
|
The KMIP scope is essentially an isolated namespace.
|
|
Here is an example creating one called `my-service` (which will be used throughout this document).
|
|
|
|
```sh
|
|
vault write -f kmip/scope/my-service
|
|
```
|
|
|
|
1. Create a KMIP role that has access to the scope:
|
|
|
|
```sh
|
|
vault write kmip/scope/my-service/role/admin operation_all=true
|
|
```
|
|
|
|
1. Create TLS credentials (a certificate, key, and CA bundle) for the KMIP role:
|
|
|
|
~> **Note**: This command will output the credentials in plaintext.
|
|
|
|
```sh
|
|
vault write -f -format=json kmip/scope/my-service/role/admin/credential/generate | tee kmip.json
|
|
```
|
|
|
|
The response from the `credential/generate` endpoint is JSON.
|
|
The `.data.certificate` entry contains a bundle of the TLS client key and certificate we will use to connect to KMIP with from `xks-proxy`.
|
|
The `.data.ca_chain[]` entries contain the CA bundle to verify the KMIP server's certificate.
|
|
Save these to, e.g., `cert.pem` and `ca.pem`:
|
|
|
|
```sh
|
|
jq --raw-output --exit-status '.data.ca_chain[]' kmip.json > ca.pem
|
|
jq --raw-output --exit-status '.data.certificate' kmip.json > cert.pem
|
|
```
|
|
|
|
## XKS Proxy Setup
|
|
|
|
The rest of the steps take place on the XKS Proxy server.
|
|
|
|
For this example, We will use an HTTPS proxy service like [ngrok](https://ngrok.com/) to forward connections
|
|
to the XKS proxy. This helps to quickly setup a valid domain and TLS endpoint for testing.
|
|
|
|
1. Start `ngrok`:
|
|
|
|
```shell-session
|
|
$ ngrok http 8000
|
|
```
|
|
|
|
This will output a domain that can be used to configure KMS later, such as `https://example.ngrok.io`.
|
|
|
|
1. Copy the `libvault-pkcs11.so` binary to the server, such as `/usr/local/lib` (should be same as in the TOML config file below), and `chmod` it so that it is executable.
|
|
|
|
1. Copy the TLS certificate bundle (e.g., `/etc/kmip/cert.pem`) and CA bundle (e.g., `/etc/kmip/ca.pem`) to the `xks-proxy` server (doesn't matter where, as long as the `xks-proxy` process has access to it) from the Vault setup.
|
|
|
|
1. Create a `configuration/settings_vault.toml` file for the XKS to Vault PKCS#11 configuration,
|
|
and set the `XKS_PROXY_SETTINGS_TOML` environment variable to point to the file location.
|
|
|
|
The important settings to change:
|
|
|
|
* `[tls]`: change key and certificate location to point to KMIP certs
|
|
* `[[external_key_stores]]`:
|
|
* change URI path prefix to anything you like
|
|
* choose random access ID
|
|
* choose random secret key
|
|
* set which key labels are accessible to XKS (`xks_key_id_set`)
|
|
* `[pkcs11]`: set the `PKCS11_HSM_MODULE` to the location of the `libvault-pkcs11.so` (or `.dylib`) file downloaded from [releases.hashicorp.com](https://releases.hashicorp.com/vault-pkcs11-provider).
|
|
|
|
```toml
|
|
[server]
|
|
ip = "0.0.0.0"
|
|
port = 8000
|
|
region = "us-east-2"
|
|
service = "kms-xks-proxy"
|
|
|
|
[server.tcp_keepalive]
|
|
tcp_keepalive_secs = 60
|
|
tcp_keepalive_retries = 3
|
|
tcp_keepalive_interval_secs = 1
|
|
|
|
[tracing]
|
|
is_stdout_writer_enabled = true
|
|
is_file_writer_enabled = true
|
|
level = "DEBUG"
|
|
directory = "/var/local/xks-proxy/logs"
|
|
file_prefix = "xks-proxy.log"
|
|
rotation_kind = "HOURLY"
|
|
|
|
[security]
|
|
is_sigv4_auth_enabled = true
|
|
is_tls_enabled = true
|
|
is_mtls_enabled = false
|
|
|
|
[tls]
|
|
tls_cert_pem = "tls/server_cert.pem"
|
|
tls_key_pem = "tls/server_key.pem"
|
|
mtls_client_ca_pem = "tls/client_ca.pem"
|
|
mtls_client_dns_name = "us-east-2.alpha.cks.kms.aws.internal.amazonaws.com"
|
|
|
|
[[external_key_stores]]
|
|
uri_path_prefix = "/xyz"
|
|
sigv4_access_key_id = "AKIA4GBY3I6JCE5M2HPM"
|
|
sigv4_secret_access_key = "1234567890123456789012345678901234567890123="
|
|
xks_key_id_set = ["abc123"]
|
|
|
|
[pkcs11]
|
|
session_pool_max_size = 30
|
|
session_pool_timeout_milli = 0
|
|
session_eager_close = false
|
|
user_pin = ""
|
|
PKCS11_HSM_MODULE = "/usr/local/lib/libvault-pkcs11.so"
|
|
context_read_timeout_milli = 100
|
|
|
|
[limits]
|
|
max_plaintext_in_base64 = 8192
|
|
max_aad_in_base64 = 16384
|
|
|
|
[hsm_capabilities]
|
|
can_generate_iv = true
|
|
is_zero_iv_required = true
|
|
```
|
|
|
|
~> **Note**: This uses an IV of zero until the next release of `vault-pkcs11-provider`.
|
|
|
|
1. Create a file, `/etc/vault-pkcs11.hcl` with the following contents:
|
|
|
|
```hcl
|
|
slot {
|
|
server = "VAULT_ADDRESS:5696"
|
|
tls_cert_path = "/etc/kmip/cert.pem"
|
|
ca_path = "/etc/kmip/ca.pem"
|
|
scope = "my-service"
|
|
}
|
|
```
|
|
|
|
This file is used by `libvault-pkcs11.so` to know how to find and communicate with the KMIP server.
|
|
See [the Vault docs](https://developer.hashicorp.com/vault/docs/enterprise/pkcs11-provider) for all available parameters and their usage.
|
|
|
|
1. If you want to view the Vault logs (helpful when trying to find error messages), you can specify the `VAULT_LOG_FILE` (default is stdout) and `VAULT_LOG_LEVEL` (default is `INFO`). We'd recommend setting `VAULT_LOG_FILE` to something like `/tmp/vault.log` or `/var/log/vault.log`. Other useful log levels are `WARN` (quieter) and `TRACE` (very verbose, could possibly contain sensitive information, like raw network packets).
|
|
|
|
1. Create an AES-256 key in KMIP, for example, using `pkcs11-tool` (usually installed with the OpenSC package). See the [Vault docs](https://developer.hashicorp.com/vault/docs/enterprise/pkcs11-provider) for the full setup.
|
|
```sh
|
|
VAULT_LOG_FILE=/dev/null pkcs11-tool --module ./libvault-pkcs11.so --keygen -a abc123 --key-type AES:32 \
|
|
--extractable --allow-sw
|
|
Key generated:
|
|
Secret Key Object; AES length 32
|
|
VALUE:
|
|
label: abc123
|
|
Usage: encrypt, decrypt, wrap, unwrap
|
|
Access: none
|
|
```
|
|
|
|
## Enable XKS in the AWS CLI
|
|
|
|
|
|
1. Create the KMS custom key store with the appropriate parameters to point to your XKS proxy (in this example, through `ngrok`).
|
|
|
|
```shell-session
|
|
$ aws kms create-custom-key-store \
|
|
--custom-key-store-name myVaultKeyStore \
|
|
--custom-key-store-type EXTERNAL_KEY_STORE \
|
|
--xks-proxy-uri-endpoint https://example.ngrok.io \
|
|
--xks-proxy-uri-path /xyz/kms/xks/v1 \
|
|
--xks-proxy-authentication-credential AccessKeyId=AKIA4GBY3I6JCE5M2HPM,RawSecretAccessKey=1234567890123456789012345678901234567890123= \
|
|
--xks-proxy-connectivity PUBLIC_ENDPOINT
|
|
|
|
{
|
|
"CustomKeyStoreId": "cks-d7a55fe93d63191d6"
|
|
}
|
|
```
|
|
|
|
1. Tell KMS to connect to the key store.
|
|
|
|
```shell-session
|
|
$ aws kms connect-custom-key-store --custom-key-store-id cks-d7a55fe93d63191d6
|
|
```
|
|
|
|
1. Wait for the `ConnectionState` of your custom key store to be `CONNECTED`. This can take a few minutes.
|
|
|
|
```shell-session
|
|
$ aws kms describe-custom-key-stores --custom-key-store-id cks-d7a55fe93d63191d6
|
|
```
|
|
|
|
1. Create a KMS key associated with the XKS key ID (`abc123` in this example):
|
|
|
|
```shell-session
|
|
$ aws kms create-key --custom-key-store-id cks-d7a55fe93d63191d6 \
|
|
--xks-key-id abc123 --origin EXTERNAL_KEY_STORE
|
|
{
|
|
"KeyMetadata": {
|
|
"AWSAccountId": "111111111111",
|
|
"KeyId": "a93f205a-2a37-4338-aa64-92b4a4b0b67d",
|
|
"Arn": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
|
|
"CreationDate": "2022-12-22T11:03:23.695000-08:00",
|
|
"Enabled": true,
|
|
"Description": "",
|
|
"KeyUsage": "ENCRYPT_DECRYPT",
|
|
"KeyState": "Enabled",
|
|
"Origin": "EXTERNAL_KEY_STORE",
|
|
"CustomKeyStoreId": "cks-16460f66b34705025",
|
|
"KeyManager": "CUSTOMER",
|
|
"CustomerMasterKeySpec": "SYMMETRIC_DEFAULT",
|
|
"KeySpec": "SYMMETRIC_DEFAULT",
|
|
"EncryptionAlgorithms": [
|
|
"SYMMETRIC_DEFAULT"
|
|
],
|
|
"MultiRegion": false,
|
|
"XksKeyConfiguration": {
|
|
"Id": "abc123"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
1. Encrypt some data with this key:
|
|
|
|
```shell-session
|
|
$ aws kms encrypt --key-id a93f205a-2a37-4338-aa64-92b4a4b0b67d --plaintext YWJjMTIzCg==
|
|
{
|
|
"CiphertextBlob": "somerandomciphertextblob=",
|
|
"KeyId": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
|
|
"EncryptionAlgorithm": "SYMMETRIC_DEFAULT"
|
|
}
|
|
|
|
1. Decypt the resulting ciphertext:
|
|
|
|
```shell-session
|
|
$ aws kms decrypt --ciphertext-blob somerandomciphertextblob=
|
|
{
|
|
"KeyId": "arn:aws:kms:us-east-2:111111111111:key/a93f205a-2a37-4338-aa64-92b4a4b0b67d",
|
|
"Plaintext": "YWJjMTIzCg==",
|
|
"EncryptionAlgorithm": "SYMMETRIC_DEFAULT"
|
|
}
|
|
```
|
|
|
|
1. Optionally, clean up your key and key store with:
|
|
|
|
```shell-session
|
|
$ aws kms disable-key --key-id a93f205a-2a37-4338-aa64-92b4a4b0b67d
|
|
$ aws kms disconnect-custom-key-store --custom-key-store-id cks-16460f66b34705025
|
|
$ aws kms delete-custom-key-store --custom-key-store-id cks-16460f66b34705025
|
|
```
|
|
|
|
(The `aws kms delete-custom-key-store` command will not succeed until all keys in the key store have been disabled and deleted.) |