2019-02-26 14:57:17 +00:00
|
|
|
---
|
2020-01-18 00:18:09 +00:00
|
|
|
layout: docs
|
|
|
|
page_title: Vault Agent Caching
|
2019-02-26 14:57:17 +00:00
|
|
|
description: |-
|
|
|
|
Vault Agent Caching allows client-side caching of responses containing newly
|
|
|
|
created tokens and responses containing leased secrets generated off of these
|
|
|
|
newly created tokens.
|
|
|
|
---
|
|
|
|
|
|
|
|
# Vault Agent Caching
|
|
|
|
|
|
|
|
Vault Agent Caching allows client-side caching of responses containing newly
|
|
|
|
created tokens and responses containing leased secrets generated off of these
|
2019-03-05 20:19:52 +00:00
|
|
|
newly created tokens. The renewals of the cached tokens and leases are also
|
2019-02-26 14:57:17 +00:00
|
|
|
managed by the agent.
|
|
|
|
|
2019-03-05 22:12:04 +00:00
|
|
|
-> **Note:** Vault Agent Caching works best with servers/clusters that are
|
2019-03-18 23:38:54 +00:00
|
|
|
running on Vault 1.1 and above due to changes that were introduced
|
2019-03-05 22:12:04 +00:00
|
|
|
alongside this feature, such as the exposure of the `orphan` field in token
|
2019-03-18 23:38:54 +00:00
|
|
|
creation responses.
|
2019-03-05 22:12:04 +00:00
|
|
|
|
2019-02-26 14:57:17 +00:00
|
|
|
## Caching and Renewals
|
|
|
|
|
2019-03-05 20:19:52 +00:00
|
|
|
Response caching and renewals are managed by the agent only under these
|
|
|
|
specific scenarios.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
2019-03-05 20:19:52 +00:00
|
|
|
1. Token creation requests are made through the agent. This means that any
|
|
|
|
login operations performed using various auth methods and invoking the token
|
|
|
|
creation endpoints of the token auth method via the agent will result in the
|
|
|
|
response getting cached by the agent. Responses containing new tokens will
|
|
|
|
be cached by the agent only if the parent token is already being managed by
|
|
|
|
the agent or if the new token is an orphan token.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
|
|
|
2. Leased secret creation requests are made through the agent using tokens that
|
2019-03-05 20:19:52 +00:00
|
|
|
are already managed by the agent. This means that any dynamic credentials
|
|
|
|
that are issued using the tokens managed by the agent, will be cached and
|
|
|
|
its renewals are taken care of.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
|
|
|
## Using Auto-Auth Token
|
|
|
|
|
2019-03-05 20:19:52 +00:00
|
|
|
Vault Agent allows for easy authentication to Vault in a wide variety of
|
2020-01-22 20:05:41 +00:00
|
|
|
environments using [Auto-Auth](/docs/agent/autoauth). By setting the
|
2019-03-05 20:19:52 +00:00
|
|
|
`use_auto_auth_token` (see below) configuration, clients will not be required
|
|
|
|
to provide a Vault token to the requests made to the agent. When this
|
|
|
|
configuration is set, if the request doesn't already bear a token, then the
|
|
|
|
auto-auth token will be used to forward the request to the Vault server. This
|
|
|
|
configuration will be overridden if the request already has a token attached,
|
|
|
|
in which case, the token present in the request will be used to forward the
|
|
|
|
request to the Vault server.
|
|
|
|
|
2020-02-14 21:48:12 +00:00
|
|
|
## Forcing Auto-Auth Token
|
|
|
|
|
|
|
|
Vault Agent can be configured to force the use of the auto-auth token by using
|
|
|
|
the value `force` for the `use_auto_auth_token` option. This configuration
|
2021-04-22 17:09:31 +00:00
|
|
|
overrides the default behavior described above in [Using Auto-Auth
|
2020-02-14 21:48:12 +00:00
|
|
|
Token](/docs/agent/caching#using-auto-auth-token), and instead ignores any
|
2020-05-21 17:18:17 +00:00
|
|
|
existing Vault token in the request and instead uses the auto-auth token.
|
2020-02-14 21:48:12 +00:00
|
|
|
|
2021-04-08 14:19:17 +00:00
|
|
|
## Persistent Cache
|
|
|
|
|
2022-04-29 19:10:48 +00:00
|
|
|
Vault Agent can restore tokens and leases from a persistent cache file created
|
|
|
|
by a previous Vault Agent process.
|
2021-04-08 14:19:17 +00:00
|
|
|
|
2022-04-29 19:10:48 +00:00
|
|
|
Refer to the [Vault Agent Persistent
|
|
|
|
Caching](/docs/agent/caching/persistent-caches) page for more information on
|
|
|
|
this functionality.
|
2021-04-08 14:19:17 +00:00
|
|
|
|
2019-02-26 14:57:17 +00:00
|
|
|
## Cache Evictions
|
|
|
|
|
2019-03-05 20:19:52 +00:00
|
|
|
The eviction of cache entries pertaining to secrets will occur when the agent
|
|
|
|
can no longer renew them. This can happen when the secrets hit their maximum
|
|
|
|
TTL or if the renewals result in errors.
|
|
|
|
|
|
|
|
Agent does some best-effort cache evictions by observing specific request types
|
|
|
|
and response codes. For example, if a token revocation request is made via the
|
|
|
|
agent and if the forwarded request to the Vault server succeeds, then agent
|
|
|
|
evicts all the cache entries associated with the revoked token. Similarly, any
|
|
|
|
lease revocation operation will also be intercepted by the agent and the
|
|
|
|
respective cache entries will be evicted.
|
|
|
|
|
|
|
|
Note that while agent evicts the cache entries upon secret expirations and upon
|
|
|
|
intercepting revocation requests, it is still possible for the agent to be
|
|
|
|
completely unaware of the revocations that happen through direct client
|
|
|
|
interactions with the Vault server. This could potentially lead to stale cache
|
|
|
|
entries. For managing the stale entries in the cache, an endpoint
|
2019-03-06 19:13:43 +00:00
|
|
|
`/agent/v1/cache-clear`(see below) is made available to manually evict cache
|
2019-03-05 20:19:52 +00:00
|
|
|
entries based on some of the query criteria used for indexing the cache entries.
|
|
|
|
|
|
|
|
## Request Uniqueness
|
|
|
|
|
|
|
|
In order to detect repeat requests and return cached responses, agent will need
|
|
|
|
to have a way to uniquely identify the requests. This computation as it stands
|
|
|
|
today takes a simplistic approach (may change in future) of serializing and
|
|
|
|
hashing the HTTP request along with all the headers and the request body. This
|
|
|
|
hash value is then used as an index into the cache to check if the response is
|
|
|
|
readily available. The consequence of this approach is that the hash value for
|
|
|
|
any request will differ if any data in the request is modified. This has the
|
|
|
|
side-effect of resulting in false negatives if say, the ordering of the request
|
|
|
|
parameters are modified. As long as the requests come in without any change,
|
|
|
|
caching behavior should be consistent. Identical requests with differently
|
|
|
|
ordered request values will result in duplicated cache entries. A heuristic
|
|
|
|
assumption that the clients will use consistent mechanisms to make requests,
|
|
|
|
thereby resulting in consistent hash values per request is the idea upon which
|
|
|
|
the caching functionality is built upon.
|
|
|
|
|
|
|
|
## Renewal Management
|
|
|
|
|
|
|
|
The tokens and leases are renewed by the agent using the secret renewer that is
|
|
|
|
made available via the Vault server's [Go
|
|
|
|
API](https://godoc.org/github.com/hashicorp/vault/api#Renewer). Agent performs
|
|
|
|
all operations in memory and does not persist anything to storage. This means
|
|
|
|
that when the agent is shut down, all the renewal operations are immediately
|
|
|
|
terminated and there is no way for agent to resume renewals after the fact.
|
|
|
|
Note that shutting down the agent does not indicate revocations of the secrets,
|
|
|
|
instead it only means that renewal responsibility for all the valid unrevoked
|
|
|
|
secrets are no longer performed by the Vault agent.
|
|
|
|
|
|
|
|
### Agent CLI
|
|
|
|
|
|
|
|
Agent's listener address will be picked up by the CLI through the
|
|
|
|
`VAULT_AGENT_ADDR` environment variable. This should be a complete URL such as
|
2019-03-14 18:53:14 +00:00
|
|
|
"http://127.0.0.1:8200".
|
2019-02-26 14:57:17 +00:00
|
|
|
|
|
|
|
## API
|
|
|
|
|
|
|
|
### Cache Clear
|
|
|
|
|
2019-03-18 20:07:10 +00:00
|
|
|
This endpoint clears the cache based on given criteria. To use this
|
2019-03-05 20:19:52 +00:00
|
|
|
API, some information on how the agent caches values should be known
|
2019-03-18 20:07:10 +00:00
|
|
|
beforehand. Each response that is cached in the agent will be indexed on some
|
2019-02-26 14:57:17 +00:00
|
|
|
factors depending on the type of request. Those factors can be the `token` that
|
2019-03-05 20:19:52 +00:00
|
|
|
is belonging to the cached response, the `token_accessor` of the token
|
|
|
|
belonging to the cached response, the `request_path` that resulted in the
|
|
|
|
cached response, the `lease` that is attached to the cached response, the
|
|
|
|
`namespace` to which the cached response belongs to, and a few more. This API
|
|
|
|
exposes some factors through which associated cache entries are fetched and
|
|
|
|
evicted.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
2020-01-18 00:18:09 +00:00
|
|
|
| Method | Path | Produces |
|
|
|
|
| :----- | :---------------------- | :--------------------- |
|
|
|
|
| `POST` | `/agent/v1/cache-clear` | `200 application/json` |
|
2019-02-26 14:57:17 +00:00
|
|
|
|
|
|
|
#### Parameters
|
|
|
|
|
|
|
|
- `type` `(strings: required)` - The type of cache entries to evict. Valid
|
2019-03-18 20:07:10 +00:00
|
|
|
values are `request_path`, `lease`, `token`, `token_accessor`, and `all`.
|
|
|
|
If the `type` is set to `all`, the _entire cache_ is cleared.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
|
|
|
- `value` `(string: required)` - An exact value or the prefix of the value for
|
|
|
|
the `type` selected. This parameter is optional when the `type` is set
|
|
|
|
to `all`.
|
|
|
|
|
2019-03-05 20:19:52 +00:00
|
|
|
- `namespace` `(string: optional)` - This is only applicable when the `type` is set to
|
|
|
|
`request_path`. The namespace of which the cache entries to be evicted for
|
|
|
|
the given request path.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
|
|
|
### Sample Payload
|
|
|
|
|
|
|
|
```json
|
|
|
|
{
|
|
|
|
"type": "token",
|
|
|
|
"value": "s.rlNjegSKykWcplOkwsjd8bP9"
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
|
|
|
### Sample Request
|
|
|
|
|
2020-05-21 17:18:17 +00:00
|
|
|
```shell-session
|
2019-02-26 14:57:17 +00:00
|
|
|
$ curl \
|
|
|
|
--request POST \
|
|
|
|
--data @payload.json \
|
2019-03-06 19:13:43 +00:00
|
|
|
http://127.0.0.1:1234/agent/v1/cache-clear
|
2019-02-26 14:57:17 +00:00
|
|
|
```
|
|
|
|
|
2019-03-05 20:19:52 +00:00
|
|
|
## Configuration (`cache`)
|
2019-02-26 14:57:17 +00:00
|
|
|
|
2019-03-05 20:19:52 +00:00
|
|
|
The top level `cache` block has the following configuration entries:
|
2019-02-26 14:57:17 +00:00
|
|
|
|
2020-02-14 21:48:12 +00:00
|
|
|
- `use_auto_auth_token (bool/string: false)` - If set, the requests made to agent
|
2019-03-05 20:19:52 +00:00
|
|
|
without a Vault token will be forwarded to the Vault server with the
|
|
|
|
auto-auth token attached. If the requests already bear a token, this
|
|
|
|
configuration will be overridden and the token in the request will be used to
|
2020-02-14 21:48:12 +00:00
|
|
|
forward the request to the Vault server. If set to `"force"` Agent will use the
|
2020-05-21 17:18:17 +00:00
|
|
|
auto-auth token, overwriting the attached Vault token if set.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
2021-04-08 14:19:17 +00:00
|
|
|
- `persist` `(object: optional)` - Configuration for the persistent cache.
|
|
|
|
|
2021-06-29 19:36:10 +00:00
|
|
|
The following two `cache` options are only useful when talking to a Vault
|
|
|
|
Enterprise cluster, and are documented as part of its
|
|
|
|
[Eventual Consistency](/docs/enterprise/consistency#vault-agent-and-consistency-headers)
|
|
|
|
page.
|
|
|
|
|
|
|
|
- `enforce_consistency` `(string: "never")` - Set to one of `"always"`
|
|
|
|
or `"never"`.
|
|
|
|
|
|
|
|
- `when_inconsistent` `(string: optional)` - Set to one of `"fail"`, `"retry"`,
|
2022-04-04 17:05:34 +00:00
|
|
|
or `"forward"`.
|
2021-06-29 19:36:10 +00:00
|
|
|
|
2021-11-02 18:29:40 +00:00
|
|
|
-> **Note:** When the `cache` block is defined, at least one
|
|
|
|
[template][agent-template] or [listener][agent-listener] must also be defined
|
|
|
|
in the config, otherwise there is no way to utilize the cache.
|
|
|
|
|
|
|
|
[agent-template]: /docs/agent/template
|
|
|
|
[agent-listener]: /docs/agent/caching#configuration-listener
|
|
|
|
|
2021-04-08 14:19:17 +00:00
|
|
|
### Configuration (Persist)
|
|
|
|
|
|
|
|
These are common configuration values that live within the `persist` block:
|
|
|
|
|
2021-11-02 18:29:40 +00:00
|
|
|
- `type` `(string: required)` - The type of the persistent cache to use,
|
2021-04-08 14:19:17 +00:00
|
|
|
e.g. `kubernetes`. _Note_: when using HCL this can be used as the key for
|
|
|
|
the block, e.g. `persist "kubernetes" {...}`.
|
|
|
|
|
2021-06-29 19:36:10 +00:00
|
|
|
- `path` `(string: required)` - The path on disk where the persistent cache file
|
2021-04-08 14:19:17 +00:00
|
|
|
should be created or restored from.
|
|
|
|
|
2021-06-29 19:36:10 +00:00
|
|
|
- `keep_after_import` `(bool: optional)` - When set to true, a restored cache file
|
2021-04-08 14:19:17 +00:00
|
|
|
is not deleted. Defaults to `false`.
|
|
|
|
|
2021-06-29 19:36:10 +00:00
|
|
|
- `exit_on_err` `(bool: optional)` - When set to true, if any errors occur during
|
2021-11-02 18:29:40 +00:00
|
|
|
a persistent cache restore, Vault Agent will exit with an error. Defaults to `true`.
|
2021-04-08 14:19:17 +00:00
|
|
|
|
2019-03-15 18:58:53 +00:00
|
|
|
## Configuration (`listener`)
|
2019-02-26 14:57:17 +00:00
|
|
|
|
2019-03-15 18:58:53 +00:00
|
|
|
- `listener` `(array of objects: required)` - Configuration for the listeners.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
2021-05-27 14:47:45 +00:00
|
|
|
There can be one or more `listener` blocks at the top level. These configuration
|
|
|
|
values are common to both `tcp` and `unix` listener blocks. Blocks of type
|
|
|
|
`tcp` support the standard `tcp` [listener](/docs/configuration/listener/tcp)
|
|
|
|
options.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
|
|
|
- `type` `(string: required)` - The type of the listener to use. Valid values
|
|
|
|
are `tcp` and `unix`.
|
2020-01-18 00:18:09 +00:00
|
|
|
_Note_: when using HCL this can be used as the key for the block, e.g.
|
2019-02-26 14:57:17 +00:00
|
|
|
`listener "tcp" {...}`.
|
|
|
|
|
|
|
|
- `address` `(string: required)` - The address for the listener to listen to.
|
|
|
|
This can either be a URL path when using `tcp` or a file path when using
|
2019-03-14 18:53:14 +00:00
|
|
|
`unix`. For example, `127.0.0.1:8200` or `/path/to/socket`. Defaults to
|
|
|
|
`127.0.0.1:8200`.
|
2019-02-26 14:57:17 +00:00
|
|
|
|
|
|
|
- `tls_disable` `(bool: false)` - Specifies if TLS will be disabled.
|
|
|
|
|
|
|
|
- `tls_key_file` `(string: optional)` - Specifies the path to the private key
|
|
|
|
for the certificate.
|
|
|
|
|
|
|
|
- `tls_cert_file` `(string: optional)` - Specifies the path to the certificate
|
|
|
|
for TLS.
|
|
|
|
|
|
|
|
### Example Configuration
|
|
|
|
|
2022-04-29 19:10:48 +00:00
|
|
|
Here is an example of a cache configuration.
|
2021-04-08 14:19:17 +00:00
|
|
|
|
2022-04-29 19:10:48 +00:00
|
|
|
```hcl
|
|
|
|
# Other Vault Agent configuration blocks
|
|
|
|
# ...
|
2021-04-08 14:19:17 +00:00
|
|
|
|
|
|
|
cache {
|
|
|
|
use_auto_auth_token = true
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
2022-04-04 17:05:34 +00:00
|
|
|
## Tutorial
|
2019-10-14 17:31:03 +00:00
|
|
|
|
|
|
|
Refer to the [Vault Agent
|
|
|
|
Caching](https://learn.hashicorp.com/vault/identity-access-management/agent-caching)
|
2022-04-04 17:05:34 +00:00
|
|
|
tutorial to learn how to use the Vault Agent to increase the availability of tokens and secrets to clients using its Caching function.
|