289 lines
14 KiB
Plaintext
289 lines
14 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Custom - Database - Secrets Engines
|
|
sidebar_title: Custom
|
|
description: |-
|
|
The database secrets engine allows new functionality to be added through a
|
|
plugin interface without needing to modify vault's core code. This allows you
|
|
write your own code to generate credentials in any database you wish. It also
|
|
allows databases that require dynamically linked libraries to be used as
|
|
plugins while keeping Vault itself statically linked.
|
|
---
|
|
|
|
# Custom Database Secrets Engines
|
|
|
|
~> The interface for custom database plugins has changed in Vault 1.6. Vault will
|
|
continue to recognize the now deprecated version of this interface for some time.
|
|
If you are using a plugin with the deprecated interface, you should upgrade to the
|
|
newest version. See [Upgrading database plugins](#upgrading-database-plugins)
|
|
for more details.
|
|
|
|
~> **Advanced topic!** Plugin development is a highly advanced topic in Vault,
|
|
and is not required knowledge for day-to-day usage. If you don't plan on writing
|
|
any plugins, we recommend not reading this section of the documentation.
|
|
|
|
The database secrets engine allows new functionality to be added through a
|
|
plugin interface without needing to modify vault's core code. This allows you
|
|
write your own code to generate credentials in any database you wish. It also
|
|
allows databases that require dynamically linked libraries to be used as plugins
|
|
while keeping Vault itself statically linked.
|
|
|
|
Please read the [Plugins internals](/docs/internals/plugins) docs for more
|
|
information about the plugin system before getting started building your
|
|
Database plugin.
|
|
|
|
## Plugin Interface
|
|
|
|
All plugins for the database secrets engine must implement the same interface. This interface
|
|
is found in `sdk/database/dbplugin/v5/database.go`
|
|
|
|
```go
|
|
type Database interface {
|
|
// Initialize the database plugin. This is the equivalent of a constructor for the
|
|
// database object itself.
|
|
Initialize(ctx context.Context, req InitializeRequest) (InitializeResponse, error)
|
|
|
|
// NewUser creates a new user within the database. This user is temporary in that it
|
|
// will exist until the TTL expires.
|
|
NewUser(ctx context.Context, req NewUserRequest) (NewUserResponse, error)
|
|
|
|
// UpdateUser updates an existing user within the database.
|
|
UpdateUser(ctx context.Context, req UpdateUserRequest) (UpdateUserResponse, error)
|
|
|
|
// DeleteUser from the database. This should not error if the user didn't
|
|
// exist prior to this call.
|
|
DeleteUser(ctx context.Context, req DeleteUserRequest) (DeleteUserResponse, error)
|
|
|
|
// Type returns the Name for the particular database backend implementation.
|
|
// This type name is usually set as a constant within the database backend
|
|
// implementation, e.g. "mysql" for the MySQL database backend. This is used
|
|
// for things like metrics and logging. No behavior is switched on this.
|
|
Type() (string, error)
|
|
|
|
// Close attempts to close the underlying database connection that was
|
|
// established by the backend.
|
|
Close() error
|
|
}
|
|
```
|
|
|
|
Each of the request and response objects can also be found in `sdk/database/dbplugin/v5/database.go`.
|
|
|
|
In each of the requests, you will see at least 1 `Statements` object (in `UpdateUserRequest`
|
|
they are in sub-fields). This object represents the set of commands to run for that particular
|
|
operation. For the `NewUser` function, this is a set of commands to create the user (and often
|
|
set permissions for that user). These statements are from the following fields in the API:
|
|
|
|
| API Argument | Request Object |
|
|
| -------------------------- | -------------------------------------------------- |
|
|
| `creation_statements` | `NewUserRequest.Statements.Commands` |
|
|
| `revocation_statements` | `DeleteUserRequest.Statements.Commands` |
|
|
| `rollback_statements` | `NewUserRequest.RollbackStatements.Commands` |
|
|
| `renew_statements` | `UpdateUserRequest.Expiration.Statements.Commands` |
|
|
| `rotation_statements` | `UpdateUserRequest.Password.Statements.Commands` |
|
|
| `root_rotation_statements` | `UpdateUserRequest.Password.Statements.Commands` |
|
|
|
|
In many of the built-in plugins, they replace `{{name}}` (or `{{username}}`), `{{password}}`,
|
|
and/or `{{expiration}}` with the associated values. It is up to your plugin to perform these
|
|
string replacements. There is a helper function located in `sdk/database/helper/dbutil`
|
|
called `QueryHelper` that assists in doing this string replacement. You are not required to
|
|
use it, but it will make your plugin's behavior consistent with the built-in plugins.
|
|
|
|
The `InitializeRequest` object contains a map of keys to values. This data is what the
|
|
user specified as the configuration for the plugin. Your plugin should use this
|
|
data to make connections to the database. The response object contains a similar configuration
|
|
map. The response object should contain the configuration map that should be saved within Vault.
|
|
This allows the plugin to manipulate the configuration prior to saving it.
|
|
|
|
It is also passed a boolean value (`InitializeRequest.VerifyConnection`) indicating if your
|
|
plugin should initialize a connection to the database during the `Initialize` call. This
|
|
function is called when the configuration is written. This allows the user to know whether
|
|
the configuration is valid and able to connect to the database in question. If this is set to
|
|
false, no connection should be made during the `Initialize` call, but subsequent calls to the
|
|
other functions will need to open a connection.
|
|
|
|
## Serving your plugin
|
|
|
|
The plugin runs as a separate binary outside of Vault, so the plugin itself will need a `main`
|
|
function. Use the `Serve` function within `sdk/database/dbplugin/v5` to serve your plugin. You
|
|
will also need to pass some TLS configuration information that Vault uses when initializing the
|
|
plugin. Below is an example setup:
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"github.com/hashicorp/vault/api/plugins"
|
|
dbplugin "github.com/hashicorp/vault/sdk/database/v5"
|
|
)
|
|
|
|
func main() {
|
|
apiClientMeta := &api.PluginAPIClientMeta{}
|
|
flags := apiClientMeta.FlagSet()
|
|
flags.Parse(os.Args[1:])
|
|
|
|
err := Run(apiClientMeta.GetTLSConfig())
|
|
if err != nil {
|
|
log.Println(err)
|
|
os.Exit(1)
|
|
}
|
|
}
|
|
|
|
func Run(apiTLSConfig *api.TLSConfig) error {
|
|
dbType, err := New()
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
dbplugin.Serve(dbType.(dbplugin.Database), api.VaultPluginTLSProvider(apiTLSConfig))
|
|
|
|
return nil
|
|
}
|
|
|
|
func New() (interface{}, error) {
|
|
db, err := newDatabase()
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
|
|
// This middleware isn't strictly required, but highly recommended to prevent accidentally exposing
|
|
// values such as passwords in error messages. An example of this is included below
|
|
db = dbplugin.NewDatabaseErrorSanitizerMiddleware(db, db.secretValues)
|
|
return db, nil
|
|
}
|
|
|
|
type MyDatabase struct {
|
|
// Variables for the database
|
|
password string
|
|
}
|
|
|
|
func newDatabase() (MyDatabase, error) {
|
|
// ...
|
|
db := &MyDatabase{
|
|
// ...
|
|
}
|
|
return db, nil
|
|
}
|
|
|
|
func (db *MyDatabase) secretValues() map[string]string {
|
|
return map[string]string{
|
|
db.password: "[password]",
|
|
}
|
|
}
|
|
```
|
|
|
|
Replacing `MyDatabase` with the actual implementation of your database plugin.
|
|
|
|
## Running your plugin
|
|
|
|
The above main package, once built, will supply you with a binary of your
|
|
plugin. We also recommend if you are planning on distributing your plugin to
|
|
build with [gox](https://github.com/mitchellh/gox) for cross platform builds.
|
|
|
|
To use your plugin with the database secrets engine you need to place the binary in the
|
|
plugin directory as specified in the [plugin internals](/docs/internals/plugins) docs.
|
|
|
|
You should now be able to register your plugin into the vault catalog. To do
|
|
this your token will need sudo permissions.
|
|
|
|
```shell-session
|
|
$ vault write sys/plugins/catalog/database/mydatabase-database-plugin \
|
|
sha256="..." \
|
|
command="mydatabase"
|
|
Success! Data written to: sys/plugins/catalog/database/mydatabase-database-plugin
|
|
```
|
|
|
|
Now you should be able to configure your plugin like any other:
|
|
|
|
```shell-session
|
|
$ vault write database/config/mydatabase \
|
|
plugin_name=mydatabase-database-plugin \
|
|
allowed_roles="readonly" \
|
|
myplugins_connection_details="..."
|
|
```
|
|
|
|
## Upgrading database plugins
|
|
|
|
### Background
|
|
|
|
In Vault 1.6, the database interface changed. The new version is referred to as version 5
|
|
and the previous version as version 4. This is due to prior versioning of the interface
|
|
that was not explicitly exposed.
|
|
|
|
The new interface was introduced for several reasons:
|
|
|
|
1. [Password policies](/docs/concepts/password-policies) introduced in Vault 1.5 required
|
|
that Vault be responsible for generating passwords. In the prior version, the database
|
|
plugin was responsible for generating passwords. This prevented integration with
|
|
password policies.
|
|
2. Passwords needed to be generated by database plugins. This meant that plugin authors
|
|
were responsible for generating secure passwords. This should be done with a helper
|
|
function available within the Vault SDK, however there was nothing preventing an
|
|
author from generating insecure passwords.
|
|
3. There were a number of inconsistencies within the version 4 interface that made it
|
|
confusing for authors. For instance: passwords were handled in 3 different ways.
|
|
`CreateUser` generated a password and returned it, `SetCredentials` receives a password
|
|
via a configuration struct and returns it, and `RotateRootCredentials` generated a
|
|
password and was expected to return an updated copy of its entire configuration
|
|
with the new password.
|
|
4. The `SetCredentials` and `RotateRootCredentials` used for static credential rotation,
|
|
and rotating the root user's credentials respectively were essentially the same operation:
|
|
change a user's password. The only practical difference was which user it was referring
|
|
to. This was especially evident when `SetCredentials` was used when rotating root
|
|
credentials (unless static credential rotation wasn't supported by the plugin in question).
|
|
5. The old interface included both `Init` and `Initialize` adding to the confusion.
|
|
|
|
The new interface is roughly modeled after a [gRPC](https://grpc.io/) interface. It has improved
|
|
future compatibility by not requiring changes to the interface definition to add additional data
|
|
in the requests or responses. It also simplifies the interface by merging several into a single
|
|
function call.
|
|
|
|
### Upgrading your custom database
|
|
|
|
Vault 1.6 supports both version 4 and version 5 database plugins. The support for version 4
|
|
plugins will be removed in a future release. To determine if a plugin is using version 4 or
|
|
version 5, the following is a list of changes in no particular order that you can check against
|
|
your plugin to determine the version:
|
|
|
|
1. The import path for version 4 is `github.com/hashicorp/vault/sdk/database/dbplugin`
|
|
whereas the import path for version 5 is `github.com/hashicorp/vault/sdk/database/dbplugin/v5`
|
|
2. Version 4 has the following functions: `Initialize`, `Init`, `CreateUser`, `RenewUser`,
|
|
`RevokeUser`, `SetCredentials`, `RotateRootCredentials`, `Type`, and `Close`. You can see the
|
|
full function signatures in `sdk/database/dbplugin/plugin.go`.
|
|
3. Version 5 has the following functions: `Initialize`, `NewUser`, `UpdateUser`, `DeleteUser`,
|
|
`Type`, and `Close`. You can see the full function signatures in
|
|
`sdk/database/dbplugin/v5/database.go`.
|
|
|
|
If you are using a version 4 custom database plugin, the following are basic instructions
|
|
for upgrading to version 5.
|
|
|
|
-> In version 4, password generation was the responsibility of the plugin. This is no longer
|
|
the case with version 5. Vault is responsible for generating passwords and passing them to
|
|
the plugin via `NewUserRequest.Password` and `UpdateUserRequest.Password.NewPassword`.
|
|
|
|
1. Change the import path from `github.com/hashicorp/vault/sdk/database/dbplugin` to
|
|
`github.com/hashicorp/vault/sdk/database/dbplugin/v5`. The package name is the same, so any
|
|
references to `dbplugin` can remain as long as those symbols exist within the new package
|
|
(such as the `Serve` function).
|
|
2. An easy way to see what functions need to be implemented is to put the following as a
|
|
global variable within your package: `var _ dbplugin.Database = (*MyDatabase)(nil)`. This
|
|
will fail to compile if the `MyDatabase` type does not adhere to the `dbplugin.Database` interface.
|
|
3. Replace `Init` and `Initialize` with the new `Initialize` function definition. The fields that
|
|
`Init` was taking (`config` and `verifyConnection`) are now wrapped into `InitializeRequest`.
|
|
The returned `map[string]interface{}` object is now wrapped into `InitializeResponse`.
|
|
Only `Initialize` is needed to adhere to the `Database` interface.
|
|
4. Update `CreateUser` to `NewUser`. The `NewUserRequest` object contains the username and
|
|
password of the user to be created. It also includes a list of statements for creating the
|
|
user as well as several other fields that may or may not be applicable. Your custom plugin
|
|
should use the password provided in the request, not generate one. If you generate a password
|
|
instead, Vault will not know about it and will give the caller the wrong password.
|
|
5. `SetCredentials`, `RotateRootCredentials`, and `RenewUser` are combined into `UpdateUser`.
|
|
The request object, `UpdateUserRequest` contains three parts: the username to change, a
|
|
`ChangePassword` and a `ChangeExpiration` object. When one of the objects is not nil, this
|
|
indicates that particular field (password or expiration) needs to change. For instance, if
|
|
the `ChangePassword` field is not-nil, the user's password should be changed. This is
|
|
equivalent to calling `SetCredentials`. If the `ChangeExpiration` field is not-nil, the
|
|
user's expiration date should be changed. This is equivalent to calling `RenewUser`.
|
|
Many databases don't need to do anything with the updated expiration.
|
|
6. Update `RevokeUser` to `DeleteUser`. This is the simplest change. The username to be
|
|
deleted is enclosed in the `DeleteUserRequest` object.
|