open-vault/website/content/docs/audit/index.mdx

174 lines
7.8 KiB
Plaintext
Raw Normal View History

2015-04-20 05:59:39 +00:00
---
layout: docs
page_title: Audit Devices
description: Audit devices are mountable devices that log requests and responses in Vault.
2015-04-20 05:59:39 +00:00
---
2017-09-08 02:38:47 +00:00
# Audit Devices
2015-04-20 05:59:39 +00:00
Audit devices are the components in Vault that collectively keep a detailed log of all
requests and response to Vault. Because every operation with Vault is an API
request/response, when using a single audit device, the audit log contains _every authenticated_ interaction with
Vault, including errors.
2015-04-20 05:59:39 +00:00
## Enabling Multiple Devices
When multiple audit devices are enabled, Vault will attempt to send the audit
logs to all of them. This allows you to not only have redundant copies, but also
a way to check for data tampering in the logs themselves.
Vault considers a request to be successful if it can log to *at least* one
configured audit device (see: [Blocked Audit
Devices](/vault/docs/audit#blocked-audit-devices) section below). Therefore in order
to build a complete picture of all audited actions, use the aggregate/union of
the logs from each audit device.
~> Note: It is **highly recommended** that you configure Vault to use multiple audit
devices. Audit failures can prevent Vault from servicing requests, so it is
important to provide at least one other device.
2017-09-08 02:38:47 +00:00
## Format
Each line in the audit log is a JSON object. The `type` field specifies what
type of object it is. Currently, only two types exist: `request` and `response`.
The line contains all of the information for any given request and response. By
default, all the sensitive information is first hashed before logging in the
audit logs.
2015-04-20 05:59:39 +00:00
## Sensitive Information
The audit logs contain the full request and response objects for every
2016-07-25 01:23:12 +00:00
interaction with Vault. The request and response can be matched utilizing a
unique identifier assigned to each request.
Most strings contained within requests and responses are hashed with a salt using HMAC-SHA256. The purpose of the hash is so that secrets aren't in plaintext within your audit logs. However, you're still able to check the value of secrets by generating HMACs yourself; this can be done with the audit device's hash function and salt by using the `/sys/audit-hash` API endpoint (see the documentation for more details).
~> Currently, only strings that come from JSON or returned in JSON are
HMAC'd. Other data types, like integers, booleans, and so on, are passed
through in plaintext. We recommend that all sensitive data be provided as string values
inside all JSON sent to Vault (i.e., that integer values are provided in quotes).
2015-04-20 05:59:39 +00:00
While most strings are hashed, Vault does make some exceptions, such as auth and secrets, and users can enable additional exceptions using the [secrets enable](/vault/docs/commands/secrets/enable) command, and then tune it afterward.
**see also**:
[secrets tune](/vault/docs/commands/secrets/tune)
[auth enable](/vault/docs/commands/auth/enable)
[auth tune](/vault/docs/commands/auth/tune)
2017-09-08 02:38:47 +00:00
## Enabling/Disabling Audit Devices
2015-04-20 05:59:39 +00:00
When a Vault server is first initialized, no auditing is enabled. Audit
devices must be enabled by a root user using `vault audit enable`.
2015-04-20 05:59:39 +00:00
2017-09-08 02:38:47 +00:00
When enabling an audit device, options can be passed to it to configure it.
For example, the command below enables the file audit device:
2015-04-20 05:59:39 +00:00
```shell-session
2017-09-08 02:38:47 +00:00
$ vault audit enable file file_path=/var/log/vault_audit.log
2015-04-20 05:59:39 +00:00
```
In the command above, we passed the "file_path" parameter to specify the path
2017-09-08 02:38:47 +00:00
where the audit log will be written to. Each audit device has its own
2015-04-20 05:59:39 +00:00
set of parameters. See the documentation to the left for more details.
~> Note: Audit device configuration is replicated to all nodes within a
cluster by default, and to performance/DR secondaries for Vault Enterprise clusters.
Before enabling an audit device, ensure that all nodes within the cluster(s)
will be able to successfully log to the audit device to avoid Vault being
blocked from serving requests.
An audit device can be limited to only within the node's cluster with the [`local`](/vault/api-docs/system/audit#local) parameter.
2017-09-08 02:38:47 +00:00
When an audit device is disabled, it will stop receiving logs immediately.
2015-04-20 05:59:39 +00:00
The existing logs that it did store are untouched.
2017-09-08 02:38:47 +00:00
## Blocked Audit Devices
2015-04-20 05:59:39 +00:00
Audit device logs are critically important and ignoring auditing failures opens an avenue for attack. Vault will not respond to requests when no enabled audit devices can record them.
2015-04-20 05:59:39 +00:00
Vault can distinguish between two types of audit device failures.
2015-04-20 05:59:39 +00:00
- A blocking failure is one where an attempt to write to the audit device never completes. This is unlikely with a local disk device, but could occure with a network-based audit device.
2015-04-20 05:59:39 +00:00
- When multiple audit devices are enabled, if any of them fail in a non-blocking fashion, Vault requests can still complete successfully provided at least one audit device successfully writes the audit record. If any of the audit devices fail in a blocking fashion however, Vault requests will hang until the blocking is resolved.
In other words, Vault will not complete any requests until the blocked audit device can write.
2022-06-02 17:15:21 +00:00
## Tutorial
Refer to [Blocked Audit Devices](/vault/tutorials/monitoring/blocked-audit-devices) for a step-by-step tutorial.
2022-06-02 17:15:21 +00:00
## API
2017-09-08 02:38:47 +00:00
Audit devices also have a full HTTP API. Please see the [Audit device API
docs](/vault/api-docs/system/audit) for more details.
Add option 'elide_list_responses' to audit backends (#18128) This PR relates to a feature request logged through HashiCorp commercial support. Vault lacks pagination in its APIs. As a result, certain list operations can return **very** large responses. The user's chosen audit sinks may experience difficulty consuming audit records that swell to tens of megabytes of JSON. In our case, one of the systems consuming audit log data could not cope, and failed. The responses of list operations are typically not very interesting, as they are mostly lists of keys, or, even when they include a "key_info" field, are not returning confidential information. They become even less interesting once HMAC-ed by the audit system. Some example Vault "list" operations that are prone to becoming very large in an active Vault installation are: auth/token/accessors/ identity/entity/id/ identity/entity-alias/id/ pki/certs/ In response, I've coded a new option that can be applied to audit backends, `elide_list_responses`. When enabled, response data is elided from audit logs, only when the operation type is "list". For added safety, the elision only applies to the "keys" and "key_info" fields within the response data - these are conventionally the only fields present in a list response - see logical.ListResponse, and logical.ListResponseWithInfo. However, other fields are technically possible if a plugin author writes unusual code, and these will be preserved in the audit log even with this option enabled. The elision replaces the values of the "keys" and "key_info" fields with an integer count of the number of entries. This allows even the elided audit logs to still be useful for answering questions like "Was any data returned?" or "How many records were listed?".
2023-01-11 21:15:52 +00:00
## Common configuration options
- `elide_list_responses` `(bool: false)` - See [Eliding list response
bodies](/vault/docs/audit#eliding-list-response-bodies) below.
Add option 'elide_list_responses' to audit backends (#18128) This PR relates to a feature request logged through HashiCorp commercial support. Vault lacks pagination in its APIs. As a result, certain list operations can return **very** large responses. The user's chosen audit sinks may experience difficulty consuming audit records that swell to tens of megabytes of JSON. In our case, one of the systems consuming audit log data could not cope, and failed. The responses of list operations are typically not very interesting, as they are mostly lists of keys, or, even when they include a "key_info" field, are not returning confidential information. They become even less interesting once HMAC-ed by the audit system. Some example Vault "list" operations that are prone to becoming very large in an active Vault installation are: auth/token/accessors/ identity/entity/id/ identity/entity-alias/id/ pki/certs/ In response, I've coded a new option that can be applied to audit backends, `elide_list_responses`. When enabled, response data is elided from audit logs, only when the operation type is "list". For added safety, the elision only applies to the "keys" and "key_info" fields within the response data - these are conventionally the only fields present in a list response - see logical.ListResponse, and logical.ListResponseWithInfo. However, other fields are technically possible if a plugin author writes unusual code, and these will be preserved in the audit log even with this option enabled. The elision replaces the values of the "keys" and "key_info" fields with an integer count of the number of entries. This allows even the elided audit logs to still be useful for answering questions like "Was any data returned?" or "How many records were listed?".
2023-01-11 21:15:52 +00:00
- `format` `(string: "json")` - Allows selecting the output format. Valid values
are `"json"` and `"jsonx"`, which formats the normal log entries as XML.
- `hmac_accessor` `(bool: true)` - If enabled, enables the hashing of token
accessor.
- `log_raw` `(bool: false)` - If enabled, logs the security sensitive
information without hashing, in the raw format.
- `prefix` `(string: "")` - A customizable string prefix to write before the
actual log line.
## Eliding list response bodies
Some Vault responses can be very large. Primarily, this affects list operations -
as Vault lacks pagination in its APIs, listing a very large collection can result
in a response that is tens of megabytes long. Some audit backends are unable to
process individual audit records of larger sizes.
The contents of the response for a list operation is often not very interesting;
most contain only a "keys" field, containing a list of IDs. Select API endpoints
additionally return a "key_info" field, a map from ID to some additional
information about the list entry - `identity/entity/id/` is an example of this.
Even in this case, the response to a list operation is usually less-confidential
or public information, for which having the full response in the audit logs is of
lesser importance.
The `elide_list_responses` audit option provides the flexibility to not write the
full list response data from the audit log, to mitigate the creation of very long
individual audit records.
When enabled, it affects only audit records of `type=response` and
`request.operation=list`. The values of `response.data.keys` and
`response.data.key_info` will be replaced with a simple integer, recording how
many entries were contained in the list (`keys`) or map (`key_info`) - therefore
even with this feature enabled, it is still possible to see how many items were
returned by a list operation.
This extra processing only affects the response data fields `keys` and `key_info`,
and only when they have the expected data types - in the event a list response
contains data outside of the usual conventions that apply to Vault list responses,
it will be left as is by this feature.
Here is an example of an audit record that has been processed by this feature
(formatted with extra whitespace, and with fields not relevant to the example
omitted):
```json
{
"type": "response",
"request": {
"operation": "list"
},
"response": {
"data": {
"key_info": 4,
"keys": 4
}
}
}
```