open-vault/website/content/docs/agent/caching/index.mdx

264 lines
11 KiB
Plaintext
Raw Normal View History

---
layout: docs
page_title: Vault Agent Caching
description: |-
Vault Agent Caching allows client-side caching of responses containing newly
created tokens and responses containing leased secrets generated off of these
newly created tokens.
---
# Vault Agent Caching
Vault Agent Caching allows client-side caching of responses containing newly
created tokens and responses containing leased secrets generated off of these
newly created tokens. The renewals of the cached tokens and leases are also
managed by the agent.
-> **Note:** Vault Agent Caching works best with servers/clusters that are
2019-03-18 23:38:54 +00:00
running on Vault 1.1 and above due to changes that were introduced
alongside this feature, such as the exposure of the `orphan` field in token
2019-03-18 23:38:54 +00:00
creation responses.
## Caching and Renewals
Response caching and renewals are managed by the agent only under these
specific scenarios.
1. Token creation requests are made through the agent. This means that any
login operations performed using various auth methods and invoking the token
creation endpoints of the token auth method via the agent will result in the
response getting cached by the agent. Responses containing new tokens will
be cached by the agent only if the parent token is already being managed by
the agent or if the new token is an orphan token.
2. Leased secret creation requests are made through the agent using tokens that
are already managed by the agent. This means that any dynamic credentials
that are issued using the tokens managed by the agent, will be cached and
its renewals are taken care of.
## Persistent Cache
Vault Agent can restore tokens and leases from a persistent cache file created
by a previous Vault Agent process.
Refer to the [Vault Agent Persistent
Caching](/vault/docs/agent/caching/persistent-caches) page for more information on
this functionality.
## Cache Evictions
The eviction of cache entries pertaining to secrets will occur when the agent
can no longer renew them. This can happen when the secrets hit their maximum
TTL or if the renewals result in errors.
Agent does some best-effort cache evictions by observing specific request types
and response codes. For example, if a token revocation request is made via the
agent and if the forwarded request to the Vault server succeeds, then agent
evicts all the cache entries associated with the revoked token. Similarly, any
lease revocation operation will also be intercepted by the agent and the
respective cache entries will be evicted.
Note that while agent evicts the cache entries upon secret expirations and upon
intercepting revocation requests, it is still possible for the agent to be
completely unaware of the revocations that happen through direct client
interactions with the Vault server. This could potentially lead to stale cache
entries. For managing the stale entries in the cache, an endpoint
`/agent/v1/cache-clear`(see below) is made available to manually evict cache
entries based on some of the query criteria used for indexing the cache entries.
## Request Uniqueness
In order to detect repeat requests and return cached responses, agent will need
to have a way to uniquely identify the requests. This computation as it stands
today takes a simplistic approach (may change in future) of serializing and
hashing the HTTP request along with all the headers and the request body. This
hash value is then used as an index into the cache to check if the response is
readily available. The consequence of this approach is that the hash value for
any request will differ if any data in the request is modified. This has the
side-effect of resulting in false negatives if say, the ordering of the request
parameters are modified. As long as the requests come in without any change,
caching behavior should be consistent. Identical requests with differently
ordered request values will result in duplicated cache entries. A heuristic
assumption that the clients will use consistent mechanisms to make requests,
thereby resulting in consistent hash values per request is the idea upon which
the caching functionality is built upon.
## Renewal Management
The tokens and leases are renewed by the agent using the secret renewer that is
made available via the Vault server's [Go
API](https://godoc.org/github.com/hashicorp/vault/api#Renewer). Agent performs
all operations in memory and does not persist anything to storage. This means
that when the agent is shut down, all the renewal operations are immediately
terminated and there is no way for agent to resume renewals after the fact.
Note that shutting down the agent does not indicate revocations of the secrets,
instead it only means that renewal responsibility for all the valid unrevoked
secrets are no longer performed by the Vault agent.
### Agent CLI
Agent's listener address will be picked up by the CLI through the
`VAULT_AGENT_ADDR` environment variable. This should be a complete URL such as
`"http://127.0.0.1:8200"`.
## API
### Cache Clear
2019-03-18 20:07:10 +00:00
This endpoint clears the cache based on given criteria. To use this
API, some information on how the agent caches values should be known
2019-03-18 20:07:10 +00:00
beforehand. Each response that is cached in the agent will be indexed on some
factors depending on the type of request. Those factors can be the `token` that
is belonging to the cached response, the `token_accessor` of the token
belonging to the cached response, the `request_path` that resulted in the
cached response, the `lease` that is attached to the cached response, the
`namespace` to which the cached response belongs to, and a few more. This API
exposes some factors through which associated cache entries are fetched and
evicted. For listeners without caching enabled, this API will still be available,
but will do nothing (there is no cache to clear) and will return a `200` response.
| Method | Path | Produces |
| :----- | :---------------------- | :--------------------- |
| `POST` | `/agent/v1/cache-clear` | `200 application/json` |
#### Parameters
- `type` `(strings: required)` - The type of cache entries to evict. Valid
2019-03-18 20:07:10 +00:00
values are `request_path`, `lease`, `token`, `token_accessor`, and `all`.
If the `type` is set to `all`, the _entire cache_ is cleared.
- `value` `(string: required)` - An exact value or the prefix of the value for
the `type` selected. This parameter is optional when the `type` is set
to `all`.
- `namespace` `(string: optional)` - This is only applicable when the `type` is set to
`request_path`. The namespace of which the cache entries to be evicted for
the given request path.
### Sample Payload
```json
{
"type": "token",
"value": "hvs.rlNjegSKykWcplOkwsjd8bP9"
}
```
### Sample Request
```shell-session
$ curl \
--request POST \
--data @payload.json \
http://127.0.0.1:1234/agent/v1/cache-clear
```
## Configuration (`cache`)
The presence of the top level `cache` block in any way (including an empty `cache` block) will enable the cache.
The top level `cache` block has the following configuration entry:
- `persist` `(object: optional)` - Configuration for the persistent cache.
The `cache` block also supports the `use_auto_auth_token`, `enforce_consistency`, and
`when_inconsistent` configuration values of the `api_proxy` block
[described in the API Proxy documentation](/vault/docs/agent/apiproxy#configuration-api_proxy) only to
maintain backwards compatibility. This configuration **cannot** be specified alongside `api_proxy` equivalents,
should not be preferred over configuring these values in the `api_proxy` block,
and `api_proxy` should be the preferred place to configure these values.
-> **Note:** When the `cache` block is defined, at least one
[template][agent-template] or [listener][agent-listener] must also be defined
in the config, otherwise there is no way to utilize the cache.
[agent-template]: /vault/docs/agent/template
[agent-listener]: /vault/docs/agent#listener-stanza
### Configuration (Persist)
These are common configuration values that live within the `persist` block:
- `type` `(string: required)` - The type of the persistent cache to use,
e.g. `kubernetes`. _Note_: when using HCL this can be used as the key for
the block, e.g. `persist "kubernetes" {...}`.
- `path` `(string: required)` - The path on disk where the persistent cache file
should be created or restored from.
- `keep_after_import` `(bool: optional)` - When set to true, a restored cache file
is not deleted. Defaults to `false`.
- `exit_on_err` `(bool: optional)` - When set to true, if any errors occur during
a persistent cache restore, Vault Agent will exit with an error. Defaults to `true`.
- `service_account_token_file` `(string: optional)` - When `type` is set to `kubernetes`,
this configures the path on disk where the Kubernetes service account token can be found.
Defaults to `/var/run/secrets/kubernetes.io/serviceaccount/token`.
## Configuration (`listener`)
- `listener` `(array of objects: required)` - Configuration for the listeners.
There can be one or more `listener` blocks at the top level. Adding a listener enables
the [API Proxy](/vault/docs/agent/apiproxy) and enables the API proxy to use the cache, if configured.
These configuration values are common to both `tcp` and `unix` listener blocks. Blocks of type
`tcp` support the standard `tcp` [listener](/vault/docs/configuration/listener/tcp)
options. Additionally, the `role` string option is available as part of the top level
of the `listener` block, which can be configured to `metrics_only` to serve only metrics,
or the default role, `default`, which serves everything (including metrics).
- `type` `(string: required)` - The type of the listener to use. Valid values
are `tcp` and `unix`.
_Note_: when using HCL this can be used as the key for the block, e.g.
`listener "tcp" {...}`.
- `address` `(string: required)` - The address for the listener to listen to.
This can either be a URL path when using `tcp` or a file path when using
`unix`. For example, `127.0.0.1:8200` or `/path/to/socket`. Defaults to
`127.0.0.1:8200`.
- `tls_disable` `(bool: false)` - Specifies if TLS will be disabled.
- `tls_key_file` `(string: optional)` - Specifies the path to the private key
for the certificate.
- `tls_cert_file` `(string: optional)` - Specifies the path to the certificate
for TLS.
### Example Configuration
Here is an example of a cache configuration with the optional `persist` block,
alongside a regular listener, and a listener that only serves metrics.
```hcl
# Other Vault Agent configuration blocks
# ...
cache {
persist = {
type = "kubernetes"
path = "/vault/agent-cache/"
keep_after_import = true
exit_on_err = true
service_account_token_file = "/tmp/serviceaccount/token"
}
}
listener "tcp" {
address = "127.0.0.1:8100"
tls_disable = true
}
listener "tcp" {
address = "127.0.0.1:3000"
tls_disable = true
role = "metrics_only"
}
```
## Tutorial
Refer to the [Vault Agent
Caching](/vault/tutorials/vault-agent/agent-caching)
tutorial to learn how to use the Vault Agent to increase the availability of tokens and secrets to clients using its Caching function.