2017-05-02 20:26:32 +00:00
|
|
|
---
|
2020-01-18 00:18:09 +00:00
|
|
|
layout: docs
|
|
|
|
page_title: Database - Secrets Engines
|
|
|
|
sidebar_title: Databases
|
2017-05-02 20:26:32 +00:00
|
|
|
description: |-
|
2017-09-20 20:05:00 +00:00
|
|
|
The database secrets engine generates database credentials dynamically based
|
|
|
|
on configured roles. It works with a number of different databases through a
|
2020-10-19 21:58:09 +00:00
|
|
|
plugin interface. There are a number of built-in database types and an exposed
|
2017-09-20 20:05:00 +00:00
|
|
|
framework for running custom database types for extendability.
|
2017-05-02 20:26:32 +00:00
|
|
|
---
|
|
|
|
|
|
|
|
# Databases
|
|
|
|
|
2017-09-20 20:05:00 +00:00
|
|
|
The database secrets engine generates database credentials dynamically based on
|
|
|
|
configured roles. It works with a number of different databases through a plugin
|
2020-10-19 21:58:09 +00:00
|
|
|
interface. There are a number of built-in database types, and an exposed framework
|
2017-09-20 20:05:00 +00:00
|
|
|
for running custom database types for extendability. This means that services
|
|
|
|
that need to access a database no longer need to hardcode credentials: they can
|
|
|
|
request them from Vault, and use Vault's leasing mechanism to more easily roll
|
2020-05-01 21:05:05 +00:00
|
|
|
keys. These are referred to as "dynamic roles" or "dynamic secrets".
|
2017-05-03 05:24:31 +00:00
|
|
|
|
2019-03-22 03:49:52 +00:00
|
|
|
Since every service is accessing the database with unique credentials, it makes
|
2017-09-20 20:05:00 +00:00
|
|
|
auditing much easier when questionable data access is discovered. You can track
|
|
|
|
it down to the specific instance of a service based on the SQL username.
|
2017-05-03 05:24:31 +00:00
|
|
|
|
|
|
|
Vault makes use of its own internal revocation system to ensure that users
|
|
|
|
become invalid within a reasonable time of the lease expiring.
|
|
|
|
|
Combined Database Backend: Static Accounts (#6834)
* Add priority queue to sdk
* fix issue of storing pointers and now copy
* update to use copy structure
* Remove file, put Item struct def. into other file
* add link
* clean up docs
* refactor internal data structure to hide heap method implementations. Other cleanup after feedback
* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods
* updates after feedback
* refactoring/renaming
* guard against pushing a nil item
* minor updates after feedback
* Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC
* Initial Combined database backend implementation of static accounts and automatic rotation
* vendor updates
* initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation
* add lock and setup of rotation queue
* vendor the queue
* rebase on new method signature of queue
* remove mongo tests for now
* update default role sql
* gofmt after rebase
* cleanup after rebasing to remove checks for ErrNotFound error
* rebase cdcr-priority-queue
* vendor dependencies with 'go mod vendor'
* website database docs for Static Role support
* document the rotate-role API endpoint
* postgres specific static role docs
* use constants for paths
* updates from review
* remove dead code
* combine and clarify error message for older plugins
* Update builtin/logical/database/backend.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* cleanups from feedback
* code and comment cleanups
* move db.RLock higher to protect db.GenerateCredentials call
* Return output with WALID if we failed to delete the WAL
* Update builtin/logical/database/path_creds_create.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* updates after running 'make fmt'
* update after running 'make proto'
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* update comment and remove and rearrange some dead code
* Update website/source/api/secret/databases/index.html.md
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* cleanups after review
* Update sdk/database/dbplugin/grpc_transport.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* code cleanup after feedback
* remove PasswordLastSet; it's not used
* document GenerateCredentials and SetCredentials
* Update builtin/logical/database/path_rotate_credentials.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* wrap pop and popbykey in backend methods to protect against nil cred rotation queue
* use strings.HasPrefix instead of direct equality check for path
* Forgot to commit this
* updates after feedback
* re-purpose an outdated test to now check that static and dynamic roles cannot share a name
* check for unique name across dynamic and static roles
* refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials
* remove commented out code
* refactor to have loadstaticwals filter out wals for roles that no longer exist
* return error if nil input given
* add nil check for input into setStaticAccount
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* add constant for queue tick time in seconds, used for comparrison in updates
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* code cleanup after review
* remove misplaced code comment
* remove commented out code
* create a queue in the Factory method, even if it's never used
* update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type
* document new method
* move rotation things into a specific file
* rename test file and consolidate some static account tests
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* update code comments, method names, and move more methods into rotation.go
* update comments to be capitalized
* remove the item from the queue before we try to destroy it
* findStaticWAL returns an error
* use lowercase keys when encoding WAL entries
* small cleanups
* remove vestigial static account check
* remove redundant DeleteWAL call in populate queue
* if we error on loading role, push back to queue with 10 second backoff
* poll in initqueue to make sure the backend is setup and can write/delete data
* add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false
* add code comments on read-only loop
* code comment updates
* re-push if error returned from find static wal
* add locksutil and acquire locks when pop'ing from the queue
* grab exclusive locks for updating static roles
* Add SetCredentials and GenerateCredentials stubs to mockPlugin
* add a switch in initQueue to listen for cancelation
* remove guard on zero time, it should have no affect
* create a new context in Factory to pass on and use for closing the backend queue
* restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
|
|
|
### Static Roles
|
|
|
|
|
|
|
|
The database secrets engine supports the concept of "static roles", which are
|
|
|
|
a 1-to-1 mapping of Vault Roles to usernames in a database. The current password
|
|
|
|
for the database user is stored and automatically rotated by Vault on a
|
|
|
|
configurable period of time. This is in contrast to dynamic secrets, where a
|
|
|
|
unique username and password pair are generated with each credential request.
|
|
|
|
When credentials are requested for the Role, Vault returns the current
|
|
|
|
password for the configured database user, allowing anyone with the proper
|
|
|
|
Vault policies to have access to the user account in the database.
|
|
|
|
|
2020-05-01 21:05:05 +00:00
|
|
|
~> Not all database types support static roles at this time. Please consult the
|
|
|
|
specific database documentation on the left navigation or the table below under
|
|
|
|
[Database Capabilities](#database-capabilities) to see if a given database
|
|
|
|
backend supports static roles.
|
Combined Database Backend: Static Accounts (#6834)
* Add priority queue to sdk
* fix issue of storing pointers and now copy
* update to use copy structure
* Remove file, put Item struct def. into other file
* add link
* clean up docs
* refactor internal data structure to hide heap method implementations. Other cleanup after feedback
* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods
* updates after feedback
* refactoring/renaming
* guard against pushing a nil item
* minor updates after feedback
* Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC
* Initial Combined database backend implementation of static accounts and automatic rotation
* vendor updates
* initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation
* add lock and setup of rotation queue
* vendor the queue
* rebase on new method signature of queue
* remove mongo tests for now
* update default role sql
* gofmt after rebase
* cleanup after rebasing to remove checks for ErrNotFound error
* rebase cdcr-priority-queue
* vendor dependencies with 'go mod vendor'
* website database docs for Static Role support
* document the rotate-role API endpoint
* postgres specific static role docs
* use constants for paths
* updates from review
* remove dead code
* combine and clarify error message for older plugins
* Update builtin/logical/database/backend.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* cleanups from feedback
* code and comment cleanups
* move db.RLock higher to protect db.GenerateCredentials call
* Return output with WALID if we failed to delete the WAL
* Update builtin/logical/database/path_creds_create.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* updates after running 'make fmt'
* update after running 'make proto'
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* update comment and remove and rearrange some dead code
* Update website/source/api/secret/databases/index.html.md
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* cleanups after review
* Update sdk/database/dbplugin/grpc_transport.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* code cleanup after feedback
* remove PasswordLastSet; it's not used
* document GenerateCredentials and SetCredentials
* Update builtin/logical/database/path_rotate_credentials.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* wrap pop and popbykey in backend methods to protect against nil cred rotation queue
* use strings.HasPrefix instead of direct equality check for path
* Forgot to commit this
* updates after feedback
* re-purpose an outdated test to now check that static and dynamic roles cannot share a name
* check for unique name across dynamic and static roles
* refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials
* remove commented out code
* refactor to have loadstaticwals filter out wals for roles that no longer exist
* return error if nil input given
* add nil check for input into setStaticAccount
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* add constant for queue tick time in seconds, used for comparrison in updates
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* code cleanup after review
* remove misplaced code comment
* remove commented out code
* create a queue in the Factory method, even if it's never used
* update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type
* document new method
* move rotation things into a specific file
* rename test file and consolidate some static account tests
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* update code comments, method names, and move more methods into rotation.go
* update comments to be capitalized
* remove the item from the queue before we try to destroy it
* findStaticWAL returns an error
* use lowercase keys when encoding WAL entries
* small cleanups
* remove vestigial static account check
* remove redundant DeleteWAL call in populate queue
* if we error on loading role, push back to queue with 10 second backoff
* poll in initqueue to make sure the backend is setup and can write/delete data
* add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false
* add code comments on read-only loop
* code comment updates
* re-push if error returned from find static wal
* add locksutil and acquire locks when pop'ing from the queue
* grab exclusive locks for updating static roles
* Add SetCredentials and GenerateCredentials stubs to mockPlugin
* add a switch in initQueue to listen for cancelation
* remove guard on zero time, it should have no affect
* create a new context in Factory to pass on and use for closing the backend queue
* restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
|
|
|
|
2017-09-20 20:05:00 +00:00
|
|
|
## Setup
|
|
|
|
|
|
|
|
Most secrets engines must be configured in advance before they can perform their
|
|
|
|
functions. These steps are usually completed by an operator or configuration
|
|
|
|
management tool.
|
|
|
|
|
|
|
|
1. Enable the database secrets engine:
|
|
|
|
|
2020-05-01 21:05:05 +00:00
|
|
|
```shell
|
2020-01-18 00:18:09 +00:00
|
|
|
$ vault secrets enable database
|
|
|
|
Success! Enabled the database secrets engine at: database/
|
|
|
|
```
|
2017-09-20 20:05:00 +00:00
|
|
|
|
2020-01-18 00:18:09 +00:00
|
|
|
By default, the secrets engine will enable at the name of the engine. To
|
|
|
|
enable the secrets engine at a different path, use the `-path` argument.
|
2017-09-20 20:05:00 +00:00
|
|
|
|
|
|
|
1. Configure Vault with the proper plugin and connection information:
|
|
|
|
|
2020-05-01 21:05:05 +00:00
|
|
|
```shell
|
2020-01-18 00:18:09 +00:00
|
|
|
$ vault write database/config/my-database \
|
|
|
|
plugin_name="..." \
|
|
|
|
connection_url="..." \
|
|
|
|
allowed_roles="..." \
|
|
|
|
username="..." \
|
|
|
|
password="..."
|
|
|
|
```
|
2017-09-20 20:05:00 +00:00
|
|
|
|
2020-10-19 21:58:09 +00:00
|
|
|
~> It is highly recommended a user within the database is created
|
2020-12-17 21:53:33 +00:00
|
|
|
specifically for Vault to use. This user will be used to manipulate
|
|
|
|
dynamic and static users within the database. This user is called the
|
|
|
|
"root" user within the documentation.
|
2020-05-01 21:05:05 +00:00
|
|
|
|
|
|
|
Vault will use the user specified here to create/update/revoke database
|
|
|
|
credentials. That user must have the appropriate permissions to perform
|
2020-10-19 21:58:09 +00:00
|
|
|
actions upon other database users (create, update credentials, delete, etc.).
|
2020-05-01 21:05:05 +00:00
|
|
|
|
2020-01-18 00:18:09 +00:00
|
|
|
This secrets engine can configure multiple database connections. For details
|
|
|
|
on the specific configuration options, please see the database-specific
|
|
|
|
documentation.
|
2017-09-20 20:05:00 +00:00
|
|
|
|
2020-10-19 21:58:09 +00:00
|
|
|
1. After configuring the root user, it is highly recommended you rotate that user's
|
2020-05-01 21:05:05 +00:00
|
|
|
password such that the vault user is not accessible by any users other than
|
|
|
|
Vault itself:
|
|
|
|
|
|
|
|
```shell
|
|
|
|
$ vault write -force database/rotate-root/my-database
|
|
|
|
```
|
2017-09-20 20:05:00 +00:00
|
|
|
|
2020-10-19 21:58:09 +00:00
|
|
|
!> When this is done, the password for the user specified in the previous step
|
2020-12-17 21:53:33 +00:00
|
|
|
is no longer accessible. Because of this, it is highly recommended that a
|
|
|
|
user is created specifically for Vault to use to manage database
|
|
|
|
users.
|
2020-05-01 21:05:05 +00:00
|
|
|
|
2020-10-19 21:58:09 +00:00
|
|
|
1. Configure a role that maps a name in Vault to a set of creation statements to
|
2020-05-01 21:05:05 +00:00
|
|
|
create the database credential:
|
|
|
|
|
|
|
|
```shell
|
2020-01-18 00:18:09 +00:00
|
|
|
$ vault write database/roles/my-role \
|
|
|
|
db_name=my-database \
|
|
|
|
creation_statements="..." \
|
|
|
|
default_ttl="1h" \
|
|
|
|
max_ttl="24h"
|
|
|
|
Success! Data written to: database/roles/my-role
|
|
|
|
```
|
2017-09-20 20:05:00 +00:00
|
|
|
|
2020-05-01 21:05:05 +00:00
|
|
|
The `{{username}}` and `{{password}}` fields will be populated by the plugin
|
2020-01-18 00:18:09 +00:00
|
|
|
with dynamically generated values. In some plugins the `{{expiration}}`
|
|
|
|
field is also be supported.
|
2017-09-20 20:05:00 +00:00
|
|
|
|
|
|
|
## Usage
|
|
|
|
|
|
|
|
After the secrets engine is configured and a user/machine has a Vault token with
|
|
|
|
the proper permission, it can generate credentials.
|
|
|
|
|
2020-01-18 00:18:09 +00:00
|
|
|
1. Generate a new credential by reading from the `/creds` endpoint with the name
|
|
|
|
of the role:
|
2017-09-20 20:05:00 +00:00
|
|
|
|
2020-05-01 21:05:05 +00:00
|
|
|
```shell
|
2017-09-20 20:05:00 +00:00
|
|
|
$ vault read database/creds/my-role
|
|
|
|
Key Value
|
|
|
|
--- -----
|
|
|
|
lease_id database/creds/my-role/2f6a614c-4aa2-7b19-24b9-ad944a8d4de6
|
|
|
|
lease_duration 1h
|
|
|
|
lease_renewable true
|
2020-10-19 21:58:09 +00:00
|
|
|
password FSREZ1S0kFsZtLat-y94
|
|
|
|
username v-vaultuser-e2978cd0-ugp7iqI2hdlff5hfjylJ-1602537260
|
2017-09-20 20:05:00 +00:00
|
|
|
```
|
2019-10-14 17:31:03 +00:00
|
|
|
|
2020-05-01 21:05:05 +00:00
|
|
|
## Database Capabilities
|
2020-12-17 21:53:33 +00:00
|
|
|
|
2020-11-06 19:16:48 +00:00
|
|
|
As of Vault 1.6, all databases support dynamic roles and static roles. All plugins except MongoDB Atlas support rotating
|
|
|
|
the root user's credentials. MongoDB Atlas cannot support rotating the root user's credentials because it uses a public
|
|
|
|
and private key pair to authenticate.
|
2020-10-19 21:58:09 +00:00
|
|
|
|
|
|
|
| Database | Root Credential Rotation | Dynamic Roles | Static Roles |
|
|
|
|
| ----------------------------------------------------- | ------------------------ | ------------- | ------------ |
|
|
|
|
| [Cassandra](/docs/secrets/databases/cassandra) | Yes | Yes | Yes (1.6+) |
|
|
|
|
| [Couchbase](/docs/secrets/databases/couchbase) | Yes | Yes | Yes |
|
|
|
|
| [Elasticsearch](/docs/secrets/databases/elasticdb) | Yes | Yes | Yes (1.6+) |
|
|
|
|
| [HanaDB](/docs/secrets/databases/hanadb) | Yes (1.6+) | Yes | Yes (1.6+) |
|
|
|
|
| [InfluxDB](/docs/secrets/databases/influxdb) | Yes | Yes | Yes (1.6+) |
|
|
|
|
| [MongoDB](/docs/secrets/databases/mongodb) | Yes | Yes | Yes |
|
2020-11-06 19:16:48 +00:00
|
|
|
| [MongoDB Atlas](/docs/secrets/databases/mongodbatlas) | No | Yes | Yes |
|
2020-10-19 21:58:09 +00:00
|
|
|
| [MSSQL](/docs/secrets/databases/mssql) | Yes | Yes | Yes |
|
|
|
|
| [MySQL/MariaDB](/docs/secrets/databases/mysql-maria) | Yes | Yes | Yes |
|
|
|
|
| [Oracle](/docs/secrets/databases/oracle) | Yes | Yes | Yes |
|
|
|
|
| [PostgreSQL](/docs/secrets/databases/postgresql) | Yes | Yes | Yes |
|
|
|
|
| [Redshift](/docs/secrets/databases/redshift) | Yes | Yes | Yes |
|
2020-05-01 21:05:05 +00:00
|
|
|
|
2017-06-06 13:49:49 +00:00
|
|
|
## Custom Plugins
|
|
|
|
|
2017-09-20 20:05:00 +00:00
|
|
|
This secrets engine allows custom database types to be run through the exposed
|
2020-07-29 17:39:53 +00:00
|
|
|
plugin interface. Please see the [custom database plugin](/docs/secrets/databases/custom)
|
|
|
|
for more information.
|
2017-06-06 13:49:49 +00:00
|
|
|
|
2019-09-26 16:53:07 +00:00
|
|
|
## Password Generation
|
|
|
|
|
2020-10-19 21:58:09 +00:00
|
|
|
Passwords are generated via [Password Policies](/docs/concepts/password-policies).
|
|
|
|
Databases can optionally set a password policy for use across all roles for that database.
|
|
|
|
In other words, each time you call `vault write database/config/my-database` you can specify
|
|
|
|
a password policy for all roles using `my-database`. Each database has a default password
|
|
|
|
policy defined as: 20 characters with at least 1 uppercase character, at least 1 lowercase
|
|
|
|
character, at least 1 number, and at least 1 dash character.
|
|
|
|
|
|
|
|
You cannot specify a password policy on a specific role as the purpose of password policies is
|
|
|
|
to adhere to password requirements of systems (such as a database), not making passwords
|
|
|
|
for specific users.
|
|
|
|
|
|
|
|
The default password generation can be represented as the following password policy:
|
|
|
|
|
|
|
|
```hcl
|
|
|
|
length = 20
|
|
|
|
|
|
|
|
rule "charset" {
|
|
|
|
charset = "abcdefghijklmnopqrstuvwxyz"
|
|
|
|
min-chars = 1
|
|
|
|
}
|
|
|
|
rule "charset" {
|
|
|
|
charset = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
|
|
|
|
min-chars = 1
|
|
|
|
}
|
|
|
|
rule "charset" {
|
|
|
|
charset = "0123456789"
|
|
|
|
min-chars = 1
|
|
|
|
}
|
|
|
|
rule "charset" {
|
|
|
|
charset = "-"
|
|
|
|
min-chars = 1
|
|
|
|
}
|
|
|
|
```
|
2019-09-26 16:53:07 +00:00
|
|
|
|
2019-10-14 17:31:03 +00:00
|
|
|
## Learn
|
|
|
|
|
|
|
|
Refer to the following step-by-step tutorials for more information:
|
|
|
|
|
|
|
|
- [Secrets as a Service: Dynamic Secrets](https://learn.hashicorp.com/vault/secrets-management/sm-dynamic-secrets)
|
|
|
|
- [Database Root Credential Rotation](https://learn.hashicorp.com/vault/secrets-management/db-root-rotation)
|
|
|
|
- [Database Static Roles and Credential Rotation](https://learn.hashicorp.com/vault/secrets-management/db-creds-rotation)
|
|
|
|
|
2017-05-03 05:24:31 +00:00
|
|
|
## API
|
|
|
|
|
2017-09-20 20:05:00 +00:00
|
|
|
The database secrets engine has a full HTTP API. Please see the [Database secret
|
2020-01-22 20:05:41 +00:00
|
|
|
secrets engine API](/api/secret/databases) for more details.
|