261e7c6b17
* first docs pass * filled in read output * transform docs changes * transform docs changes * transform docs changes * transform docs changes * transform docs changes * transform docs changes * transform docs changes
126 lines
5.9 KiB
Plaintext
126 lines
5.9 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Transform - Secrets Engines - Tokenization
|
|
sidebar_title: Tokenization Transform <sup>ENTERPRISE</sup>
|
|
description: >-
|
|
More information on the Tokenization transform.
|
|
---
|
|
|
|
# Tokenization Transform
|
|
|
|
Not to be confused with Vault tokens, [Tokenization](transform/tokenization) exchanges a
|
|
sensitive value for an unrelated value called a _token_. The original sensitive
|
|
value cannot be recovered from a token alone, they are irreversible. Instead, unlike format preserving encryption, tokenization is stateful. To
|
|
decode the original value, the token must be submitted to Vault where it is
|
|
retrieved from a cryptographic mapping in storage.
|
|
|
|
## Operation
|
|
|
|
On encode, Vault generates a random, signed token and stores a mapping of a
|
|
version of that token to encrypted versions of the plaintext and metadata, as
|
|
well as a fingerprint of the original plaintext which facilitates the `tokenized`
|
|
endpoint that lets one query whether a plaintext exists in the system.
|
|
|
|
Depending on the mapping mode, the plaintext may be decoded only with posession
|
|
of the distributed token, or may be recoverable in the export operation. See
|
|
[Security Considerations](#security-considerations) for more.
|
|
|
|
## Performance Considerations
|
|
|
|
### Builtin (Internal) Store
|
|
|
|
As tokenization is stateful, the encode operation necessarily writes values to
|
|
storage. By default, that storage is the Vault backend store itself. This
|
|
differs from some secret engines in that the encode and decode operations require
|
|
an access of storage per operation. Other engines use storage for configuration
|
|
but can process operations largely without accessing any storage.
|
|
|
|
Since these operations involve writes to storage, and therefore must be performed
|
|
on primary nodes, the scalability of the encode operation is limited by the
|
|
primary's storage performance.
|
|
|
|
Additionally, using internal storage, since writes must be performed on primary
|
|
nodes, the scalability of the encode operation will be limited by the performance
|
|
of the primary and its storage subsystem. All other operations can be performed
|
|
on secondaries.
|
|
|
|
Finally, due to replication, writes to the primary may take some time to reach
|
|
secondaries, so other read operations like decode or metadata may not succeed on
|
|
the secondaries until this happens. In other words, tokenization is eventually
|
|
consistent.
|
|
|
|
### External Storage
|
|
|
|
All nodes (except DRs) can participate in all operations using external storage, but one must take care
|
|
to monitor and scale the external storage for the level of traffic experienced.
|
|
The storage schema is simple however and well known approaches should be effective.
|
|
|
|
## Security Considerations
|
|
|
|
The goal of Tokenization is to let end users' devices store the token rather than
|
|
their sensitive values (such as credit card numbers) and still participate in
|
|
transations where the token is a standin for the sensitive value. For this reason
|
|
the token Vault generates is completely unrelated (e.g. irreversible) to the
|
|
sensitive value.
|
|
|
|
Furthermore, the Tokenization transform is designed to resist a number of attacks
|
|
on the values produced during encode. In particular it is designed so that
|
|
attackers cannot recover plaintext even if they steal the tokenization values
|
|
from Vault itself. In the default mapping mode,
|
|
even stealing the underlying transform key does not allow them to recover
|
|
the plaintext without also posessing the encoded token. An attacker must have
|
|
gotten access to all values in the construct.
|
|
|
|
In the `exportable` mapping mode however, the plaintext values are encrypted
|
|
in a way that can be decrypted within Vault. If the attacker posesses the
|
|
transform key and the tokenization mapping values, the plaintext can be
|
|
recovered. This mode is available for the case where operators prioritize the
|
|
ability to export all of the plaintext values in an emergency, via the export
|
|
operation.
|
|
|
|
### Metadata
|
|
|
|
Since tokenization isn't format preserving and requires storage, one can associate
|
|
arbitrary metadata with a token. Metadata is considered less sensitive than the
|
|
original plaintext value. As it has it's own retrieval endpoint, operators can
|
|
configure policies that may allow access to the metadata of a token but not
|
|
its decoded value to enable workflows that operate just on the metadata.
|
|
|
|
## TTLs and Tidying
|
|
|
|
By default, tokens are long lived, and the storage for them will be maintained
|
|
indefinitely. Where there is a concept of time-to-live, it is strongely encouraged
|
|
that the tokens be generated with a TTL. For example, as credit cards
|
|
have an expiration date, it is recommended that tokenizing a credit card
|
|
primary account number (PAN) be done with a TTL that corresponds to the time
|
|
after which the PAN is invalid.
|
|
|
|
This allows such values to be _tidied_ and removed from storage once expired.
|
|
Tokens themselves encode the expiration time, so decode and other operations
|
|
can immediately reject the operation when presented with an expired token.
|
|
|
|
### Key Management
|
|
|
|
Tokenization supports key rotation. Keys are tied to transforms, so key
|
|
names are the same as the name of the corresponding tokenization transform.
|
|
Keys can be rotated to a new version, with backward compatibility for
|
|
decoding. Encoding is always performed with the newest key version. Keys versions
|
|
can be tidied as well. For more information, see the [transform api docs](../../../api-docs/secret/transform).
|
|
|
|
## External Storage
|
|
|
|
### SQL Stores
|
|
|
|
Currently only PostgreSQL is supported as an external storage backend for tokenization.
|
|
The [Schema Endpoint](../../../api-docs/secret/transform#create-update-store-schema) may be used to initialize and upgrade the necessary
|
|
database tables. Vault uses a schema versioning table to determine if it needs
|
|
to create or modify the tables when using that endpoint. If you make changes to
|
|
those tables yourself, the automatic schema management may become out of sync
|
|
and may fail in the future.
|
|
|
|
## Learn
|
|
|
|
Refer to [Tokenize Data with Transform Secrets
|
|
Engine](https://learn.hashicorp.com/tutorials/vault/tokenization) for a
|
|
step-by-step tutorial.
|