[Guide] Transit Secrets Engine beginner guide (#4943 )

* Intro to Transit Secrets Engine guide

* Added the Katacoda scenario link in the Reference Materials section

* Referencig this guide in the existing encryption guides

2018-07-27 16:08:52 -07:00

18 KiB

Raw Blame History

layout	page_title	sidebar_current	description
guides	Encryption as a Service - Guides	guides-encryption-transit	HashiCorp Vault's transit secrets engine handles cryptographic functions on data in-transit. It can also viewed as _encryption as a service_.

Encryption as a Service: Transit Secrets Engine

Vault's transit secrets engine handles cryptographic functions on data-in-transit. Vault doesn't store the data sent to the secrets engine, so it can also be viewed as encryption as a service.

Although the transit secrets engine provides additional features (sign and verify data, generate hashes and HMACs of data, and act as a source of random bytes), its primary use case is to encrypt data. This relieves the burden of proper encryption/decryption from application developers and pushes the burden onto the operators of Vault.

Reference Materials

~> NOTE: An interactive tutorial is also available if you do not have a Vault environment to perform the steps described in this guide.

Estimated Time to Complete

10 minutes

Personas

The end-to-end scenario described in this guide involves two personas:

operator with privileged permissions to manage the encryption keys
app with un-privileged permissions encrypt/decrypt secrets via API

Challenge

Think of the following scenario:

Example Inc. recently made headlines for a massive data breach which exposed millions of their users' payment card accounts online. When they tracked down the problem they found that a new HVAC system with management software had been put into their data centers and had created vulnerabilities in their networks and exposed ports and IPs to the databases publicly.

Solutions

The transit secrets engine enables security teams to fortify data during transit and at rest. So even if an intrusion occurs, your data is encrypted with AES 256-bit CBC encryption (TLS in transit). Even if an attacker were able to access the raw data, they would only have encrypted bits. This means attackers would need to compromise multiple systems before exfiltrating data.

This guide demonstrates the basics of the transit secrets engine.

Prerequisites

To perform the tasks described in this guide, you need to have a Vault environment. Refer to the Getting Started guide to install Vault. Make sure that your Vault server has been initialized and unsealed.

Policy requirements

-> NOTE: For the purpose of this guide, you can use root token to work with Vault. However, it is recommended that root tokens are only used for just enough initial setup or in emergencies. As a best practice, use tokens with appropriate set of policies based on your role in the organization.

To perform all tasks demonstrated in this guide, your policy must include the following permissions:

# Enable transit secrets engine
path "sys/mounts/transit" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# To read enabled secrets engines
path "sys/mounts" {
  capabilities = [ "read" ]
}

# Manage the transit secrets engine
path "transit/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

If you are not familiar with policies, complete the policies guide.

Steps

You will perform the following:

Configure Transit Secret Engine
Encrypt Secrets
Decrypt a cipher-text
Rotate the Encryption Key
Update Key Configuration

Step 1: Configure Transit Secret Engine

(Persona: operator)

The transit secrets engine must be configured before it can perform its operations. This step is usually done by an operator or configuration management tool.

CLI command

Enable the transit secret engine by executing the following command:

$ vault secrets enable transit

By default, the secrets engine will mount at the name of the engine. If you wish to enable it at a different path, use the -path argument.

Example: vault secrets enable -path=encryption transit

Now, create an encryption key ring named, orders by executing the following command:

$ vault write -f transit/keys/orders

API call using cURL

Enable transit secret engine using /sys/mounts endpoint:

$ curl --header "X-Vault-Token: <TOKEN>" \
       --request POST \
       --data <PARAMETERS> \
       <VAULT_ADDRESS>/v1/sys/mounts/<PATH>

Where <TOKEN> is your valid token, and <PARAMETERS> holds configuration parameters of the secret engine.

Example:

The following example enables transit secret engine at sys/mounts/transit path, and passed the secret engine type (transit) in the request payload.

$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"type":"transit"}' \
       https://127.0.0.1:8200/v1/sys/mounts/transit

Now, create an encryption key ring named, orders using the transit/keys endpoint:

$ curl --header "X-Vault-Token: ..." \
       --request POST \
       https://127.0.0.1:8200/v1/transit/keys/orders

Web UI

Open a web browser and launch the Vault UI (e.g. http://127.0.0.1:8200/ui) and then login.

Select Enable new engine and select Transit from Secrets engine type drop-down list.
Click Enable Engine.
Select Create encryption key and enter orders in the Name field.
Click Create encryption key to complete.

~> NOTE: Typically, you want to create an encryption key ring for each application.

Step 2: Encrypt Secrets

(Persona: operator)

Once the transit secrets engine has been configured, any client with a valid token with proper permission can send data to encrypt.

Here, you are going to encrypt a plaintext, "credit-card-number".

-> NOTE: You can pass non-text binary file such as a PDF or image. When you encrypt a plaintext, it must be base64-encoded.

CLI command

To encrypt your secret, use the transit/encrypt endpoint:

$ vault write transit/encrypt/<key_ring_name>

Execute the following command to encrypt a plaintext:

$ vault write transit/encrypt/orders plaintext=$(base64 <<< "credit-card-number")

Key           Value
---           -----
ciphertext    vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZPvy8xKfkytpwKcbC0fF2U=

Vault does NOT store any of this data. The output you received is the ciphertext. You can store this ciphertext at the desired location (e.g. MySQL database) or pass it to another application.

API call using cURL

To encrypt your secret, use the transit/encrypt endpoint.

Example:

# Generate base64-encoded plaintext
$ base64 <<< "credit-card-number"
Y3JlZGl0LWNhcmQtbnVtYmVyCg==

# Pass the base64-encoded plaintext in the request payload
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"plaintext": "Y3JlZGl0LWNhcmQtbnVtYmVyCg=="}' \
       https://127.0.0.1:8200/v1/transit/encrypt/orders | jq
{
  "request_id": "f483d9b6-8132-782e-1665-ad432c2461ab",
  "lease_id": "",
  "renewable": false,
  "lease_duration": 0,
  "data": {
    "ciphertext": "vault:v1:/9hdQutaWpZR72s3+VSCLK1JNhV1wKM49hYVjh7RjmuxIy/OvshtgV4L4uVB+aQ="
  },
  ...
}

Vault does NOT store any of this data. The output you received is the ciphertext. You can store this ciphertext at the desired location (e.g. MySQL database) or pass it to another application.

Web UI

Select the orders encryption key.
Select Key actions.
Make sure that Encrypt is selected under TRANSIT ACTIONS, and then enter "credit-card-number" in the Plaintext field.
Click Encode to base64 to encode the plaintext.
Click Encrypt. Vault does NOT store any of this data. The output you received is the ciphertext. You can click Copy to copy the resulting ciphertext and store it at the desired location (e.g. MySQL database) or pass it to another application.

Step 3: Decrypt a cipher-text

(Persona: operator)

Any client with a valid token with proper permission can decrypt the ciphertext generated by Vault. To decrypt the ciphertext, invoke the transit/decrypt endpoint.

CLI command

Execute the following command to decrypt the ciphertext resulted in Step 2:

$ vault write transit/decrypt/orders \
        ciphertext="vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZPvy8xKfkytpwKcbC0fF2U=" \

Key          Value
---          -----
plaintext    Y3JlZGl0LWNhcmQtbnVtYmVyCg==

The resulting data is base64-encoded. To reveal the original plaintext, run the following command:

$ base64 --decode <<< "Y3JlZGl0LWNhcmQtbnVtYmVyCg=="
credit-card-number

API call using cURL

Use the transit/decrypt endpoint to decrypt the ciphertext resulted in Step 2:

Example:

# Pass the ciphertext in the request payload to decode
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"ciphertext": "Yvault:v1:/9hdQutaWpZR72s3+VSCLK1JNhV1wKM49hYVjh7RjmuxIy/OvshtgV4L4uVB+aQ="}' \
       https://127.0.0.1:8200/v1/transit/decrypt/orders | jq
{
   "request_id": "062d7998-8932-76f2-f96c-5938a55ff005",
   "lease_id": "",
   "renewable": false,
   "lease_duration": 0,
   "data": {
     "plaintext": "Y3JlZGl0LWNhcmQtbnVtYmVyCg=="
   },
   ...
}

# The resulting data is base64-encoded that it must be decoded to reveal the plaintext
$ base64 --decode <<< "Y3JlZGl0LWNhcmQtbnVtYmVyCg=="
credit-card-number

Web UI

Select the orders encryption key.
Select Key actions.
Make sure that Decrypt is selected under TRANSIT ACTIONS, and then enter the ciphertext you wish to decrypt.
Click Decrypt.
The resulting data is base64-encoded. Click Decode from base64 to reveal the plaintext.

Step 4: Rotate the Encryption Key

(Persona: operator)

One of the benefits of using the Vault transit secrets engine is its ability to easily rotate the encryption keys. Keys can be rotated manually by a human, or an automated process which invokes the key rotation API endpoint through cron, a CI pipeline, a periodic Nomad batch job, Kubernetes Job, etc.

Vault maintains the versioned keyring and the operator can decide the minimum version allowed for decryption operations. When data is encrypted using Vault, the resulting ciphertext is prepended with the version of the key used to encrypt it.

CLI command

To rotate the encryption key, invoke the transit/keys/<key_ring_name>/rotate endpoint.

$ vault write -f transit/keys/orders/rotate

Let's encrypt another data:

$ vault write transit/encrypt/orders plaintext=$(base64 <<< "visa-card-number")
Key           Value
---           -----
ciphertext    vault:v2:45f9zW6cglbrzCjI0yCyC6DBYtSBSxnMgUn9B5aHcGEit71xefPEmmjMbrk3

Compare the ciphertexts from Step 2.

ciphertext    vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZPvy8xKfkytpwKcbC0fF2U=

Notice that the first ciphertext starts with "vault:v1:". After rotating the encryption key, the ciphertext starts with "vault:v2:". This indicates that the data gets encrypted using the latest version of the key after the rotation.

Execute the following command to rewrap your cipertext from Step 2 with the latest version of the encryption key:

$ vault write transit/rewrap/orders \
        ciphertext="vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZPvy8xKfkytpwKcbC0fF2U="
Key           Value
---           -----
ciphertext    vault:v2:kChHZ9w4ILRfw+DzO53IZ8m5PyB2yp2/tKbub34uB+iDqtDRB+NLCPrpzTtJHJ4=

Notice that the resulting ciphertext now starts with "vault:v2:".

This operation does not reveal the plaintext data. But Vault will decrypt the value using the appropriate key in the keyring and then encrypted the resulting plaintext with the newest key in the keyring.

API call using cURL

To rotate the encryption key, invoke the transit/keys/<key_ring_name>/rotate endpoint.

$ curl --header "X-Vault-Token: ..." \
       --request POST
       https://127.0.0.1:8200/v1/transit/keys/orders/rotate

Let's encrypt another data:

# Generate base64-encoded plaintext
$ base64 <<< "visa-card-number"
dmlzYS1jYXJkLW51bWJlcgo=

# Pass the base64-encoded plaintext in the request payload
$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"plaintext": "dmlzYS1jYXJkLW51bWJlcgo="}' \
       https://127.0.0.1:8200/v1/transit/encrypt/orders | jq
{
  ...
  "data": {
    "ciphertext": "vault:v2:et873RqkfLlS268LqYspVUnqhqZm0flNwhthe4ZzfcuZQab1TnirQ8/hMNYA"
  },
  ...
}

Compare the ciphertexts from Step 2.

ciphertext    vault:v1:cZNHVx+sxdMErXRSuDa1q/pz49fXTn1PScKfhf+PIZPvy8xKfkytpwKcbC0fF2U=

Execute the transit/rewrap endpoint to rewrap your cipertext from Step 2 with the latest version of the encryption key:

$ curl --header "X-Vault-Token: ..." \
       --request POST \
       --data '{"ciphertext": "vault:v1:/9hdQutaWpZR72s3+VSCLK1JNhV1wKM49hYVjh7RjmuxIy/OvshtgV4L4uVB+aQ="}' \
       https://127.0.0.1:8200/v1/transit/rewrap/orders | jq
{
 ...
 "data": {
   "ciphertext": "vault:v2:ykqXDP65tLVSrqxoNZh51gIobYQSwNGT+SbiD/2nl8rrhF2md+wplBGdXlhDzd4="
 },
 ...

Notice that the resulting ciphertext now starts with "vault:v2:".

Step 5: Update Key Configuration

(Persona: operator)

The operators can update the encryption key configuration to specify the minimum version of ciphertext allowed to be decrypted, the minimum version of the key that can be used to encrypt the plaintext, the key is allowed to be deleted, etc.

This helps further tightening the data encryption rule.

CLI Command

Execute the key rotation command a few times to generate multiple versions of the key:

$ vault write -f transit/keys/orders/rotate

Now, read the orders key information:

$ vault read transit/keys/orders

Key                       Value
---                       -----
...
keys                      map[6:1531439714 1:1531439594 2:1531439667 3:1531439714 4:1531439714 5:1531439714]
latest_version            6
min_decryption_version    1
min_encryption_version    0
...

In the example, the current version of the key is 6. However, there is no restriction about the minimum encryption key version, and any of the key versions can decrypt the data (min_decryption_version).

Run the following command to enforce the use of the encryption key at version 5 or later to decrypt the data.

$ vault write transit/keys/orders/config min_decryption_version=5

Now, verify the orders key configuration:

$ vault read transit/keys/orders

Key                       Value
---                       -----
allow_plaintext_backup    false
deletion_allowed          false
derived                   false
exportable                false
keys                      map[5:1531811719 6:1531811721]
latest_version            6
min_decryption_version    5
min_encryption_version    0
...

API call using cURL

Execute the transit/keys/<key_ring_name>/rotate endpoint a few times key rotation command a few times to generate multiple versions of the key:

$ curl --header "X-Vault-Token: ..." \
       --request POST
       https://127.0.0.1:8200/v1/transit/keys/orders/rotate

Read the transit/keys/orders endpoint to review the orders key detail:

$ curl --header "X-Vault-Token: ..." \
       https://127.0.0.1:8200/v1/transit/keys/orders | jq
{
  ...
   "keys": {
     "1": 1531804669,
      "2": 1531810236,
      "3": 1531811712,
      "4": 1531811715,
      "5": 1531811719,
      "6": 1531811721
   },
   "latest_version": 6,
   "min_decryption_version": 1,
   "min_encryption_version": 0,
   ...

Run the following command to enforce the use of the encryption key at version 5 or later to decrypt the data.

$ curl --header "X-Vault-Token: ..." \
       --request POST
       --data '{"min_decryption_version": 5}'
       https://127.0.0.1:8200/v1/transit/keys/orders/config

Now, verify the orders key configuration:

$ curl --header "X-Vault-Token: ..." \
       https://127.0.0.1:8200/v1/transit/keys/orders | jq
{
  ...
  "keys": {
     "5": 1531811719,
     "6": 1531811721
   },
   "latest_version": 6,
   "min_decryption_version": 5,
   "min_encryption_version": 0,
   ...

-> NOTE: Notice that the output only displays two valid encryption key versions (5 and 6).

Next steps

Transit Secrets Re-wrapping guide introduces a sample application which re-wraps data after rotating an encryption key in the transit engine in Vault.

18 KiB Raw Blame History

Encryption as a Service: Transit Secrets Engine

Reference Materials

Estimated Time to Complete

Personas

Challenge

Solutions

Prerequisites

Policy requirements

Steps

Step 1: Configure Transit Secret Engine

CLI command

API call using cURL

Web UI

Step 2: Encrypt Secrets

CLI command

API call using cURL

Web UI

Step 3: Decrypt a cipher-text

CLI command

API call using cURL

Web UI

Step 4: Rotate the Encryption Key

CLI command

API call using cURL

Step 5: Update Key Configuration

CLI Command

API call using cURL

Next steps

18 KiB

Raw Blame History