open-vault/website/pages/docs/secrets/databases/custom.mdx

289 lines
14 KiB
Plaintext

---
layout: docs
page_title: Custom - Database - Secrets Engines
sidebar_title: Custom
description: |-
The database secrets engine allows new functionality to be added through a
plugin interface without needing to modify vault's core code. This allows you
write your own code to generate credentials in any database you wish. It also
allows databases that require dynamically linked libraries to be used as
plugins while keeping Vault itself statically linked.
---
# Custom Database Secrets Engines
~> The interface for custom database plugins has changed in Vault 1.6. Vault will
continue to recognize the now deprecated version of this interface for some time.
If you are using a plugin with the deprecated interface, you should upgrade to the
newest version. See [Upgrading database plugins](#upgrading-database-plugins)
for more details.
~> **Advanced topic!** Plugin development is a highly advanced topic in Vault,
and is not required knowledge for day-to-day usage. If you don't plan on writing
any plugins, we recommend not reading this section of the documentation.
The database secrets engine allows new functionality to be added through a
plugin interface without needing to modify vault's core code. This allows you
write your own code to generate credentials in any database you wish. It also
allows databases that require dynamically linked libraries to be used as plugins
while keeping Vault itself statically linked.
Please read the [Plugins internals](/docs/internals/plugins) docs for more
information about the plugin system before getting started building your
Database plugin.
## Plugin Interface
All plugins for the database secrets engine must implement the same interface. This interface
is found in `sdk/database/dbplugin/v5/database.go`
```go
type Database interface {
// Initialize the database plugin. This is the equivalent of a constructor for the
// database object itself.
Initialize(ctx context.Context, req InitializeRequest) (InitializeResponse, error)
// NewUser creates a new user within the database. This user is temporary in that it
// will exist until the TTL expires.
NewUser(ctx context.Context, req NewUserRequest) (NewUserResponse, error)
// UpdateUser updates an existing user within the database.
UpdateUser(ctx context.Context, req UpdateUserRequest) (UpdateUserResponse, error)
// DeleteUser from the database. This should not error if the user didn't
// exist prior to this call.
DeleteUser(ctx context.Context, req DeleteUserRequest) (DeleteUserResponse, error)
// Type returns the Name for the particular database backend implementation.
// This type name is usually set as a constant within the database backend
// implementation, e.g. "mysql" for the MySQL database backend. This is used
// for things like metrics and logging. No behavior is switched on this.
Type() (string, error)
// Close attempts to close the underlying database connection that was
// established by the backend.
Close() error
}
```
Each of the request and response objects can also be found in `sdk/database/dbplugin/v5/database.go`.
In each of the requests, you will see at least 1 `Statements` object (in `UpdateUserRequest`
they are in sub-fields). This object represents the set of commands to run for that particular
operation. For the `NewUser` function, this is a set of commands to create the user (and often
set permissions for that user). These statements are from the following fields in the API:
| API Argument | Request Object |
| -------------------------- | -------------------------------------------------- |
| `creation_statements` | `NewUserRequest.Statements.Commands` |
| `revocation_statements` | `DeleteUserRequest.Statements.Commands` |
| `rollback_statements` | `NewUserRequest.RollbackStatements.Commands` |
| `renew_statements` | `UpdateUserRequest.Expiration.Statements.Commands` |
| `rotation_statements` | `UpdateUserRequest.Password.Statements.Commands` |
| `root_rotation_statements` | `UpdateUserRequest.Password.Statements.Commands` |
In many of the built-in plugins, they replace `{{name}}` (or `{{username}}`), `{{password}}`,
and/or `{{expiration}}` with the associated values. It is up to your plugin to perform these
string replacements. There is a helper function located in `sdk/database/helper/dbutil`
called `QueryHelper` that assists in doing this string replacement. You are not required to
use it, but it will make your plugin's behavior consistent with the built-in plugins.
The `InitializeRequest` object contains a map of keys to values. This data is what the
user specified as the configuration for the plugin. Your plugin should use this
data to make connections to the database. The response object contains a similar configuration
map. The response object should contain the configuration map that should be saved within Vault.
This allows the plugin to manipulate the configuration prior to saving it.
It is also passed a boolean value (`InitializeRequest.VerifyConnection`) indicating if your
plugin should initialize a connection to the database during the `Initialize` call. This
function is called when the configuration is written. This allows the user to know whether
the configuration is valid and able to connect to the database in question. If this is set to
false, no connection should be made during the `Initialize` call, but subsequent calls to the
other functions will need to open a connection.
## Serving your plugin
The plugin runs as a separate binary outside of Vault, so the plugin itself will need a `main`
function. Use the `Serve` function within `sdk/database/dbplugin/v5` to serve your plugin. You
will also need to pass some TLS configuration information that Vault uses when initializing the
plugin. Below is an example setup:
```go
package main
import (
"github.com/hashicorp/vault/api/plugins"
dbplugin "github.com/hashicorp/vault/sdk/database/v5"
)
func main() {
apiClientMeta := &api.PluginAPIClientMeta{}
flags := apiClientMeta.FlagSet()
flags.Parse(os.Args[1:])
err := Run(apiClientMeta.GetTLSConfig())
if err != nil {
log.Println(err)
os.Exit(1)
}
}
func Run(apiTLSConfig *api.TLSConfig) error {
dbType, err := New()
if err != nil {
return err
}
dbplugin.Serve(dbType.(dbplugin.Database), api.VaultPluginTLSProvider(apiTLSConfig))
return nil
}
func New() (interface{}, error) {
db, err := newDatabase()
if err != nil {
return nil, err
}
// This middleware isn't strictly required, but highly recommended to prevent accidentally exposing
// values such as passwords in error messages. An example of this is included below
db = dbplugin.NewDatabaseErrorSanitizerMiddleware(db, db.secretValues)
return db, nil
}
type MyDatabase struct {
// Variables for the database
password string
}
func newDatabase() (MyDatabase, error) {
// ...
db := &MyDatabase{
// ...
}
return db, nil
}
func (db *MyDatabase) secretValues() map[string]string {
return map[string]string{
db.password: "[password]",
}
}
```
Replacing `MyDatabase` with the actual implementation of your database plugin.
## Running your plugin
The above main package, once built, will supply you with a binary of your
plugin. We also recommend if you are planning on distributing your plugin to
build with [gox](https://github.com/mitchellh/gox) for cross platform builds.
To use your plugin with the database secrets engine you need to place the binary in the
plugin directory as specified in the [plugin internals](/docs/internals/plugins) docs.
You should now be able to register your plugin into the vault catalog. To do
this your token will need sudo permissions.
```shell-session
$ vault write sys/plugins/catalog/database/mydatabase-database-plugin \
sha256="..." \
command="mydatabase"
Success! Data written to: sys/plugins/catalog/database/mydatabase-database-plugin
```
Now you should be able to configure your plugin like any other:
```shell-session
$ vault write database/config/mydatabase \
plugin_name=mydatabase-database-plugin \
allowed_roles="readonly" \
myplugins_connection_details="..."
```
## Upgrading database plugins
### Background
In Vault 1.6, the database interface changed. The new version is referred to as version 5
and the previous version as version 4. This is due to prior versioning of the interface
that was not explicitly exposed.
The new interface was introduced for several reasons:
1. [Password policies](/docs/concepts/password-policies) introduced in Vault 1.5 required
that Vault be responsible for generating passwords. In the prior version, the database
plugin was responsible for generating passwords. This prevented integration with
password policies.
2. Passwords needed to be generated by database plugins. This meant that plugin authors
were responsible for generating secure passwords. This should be done with a helper
function available within the Vault SDK, however there was nothing preventing an
author from generating insecure passwords.
3. There were a number of inconsistencies within the version 4 interface that made it
confusing for authors. For instance: passwords were handled in 3 different ways.
`CreateUser` generated a password and returned it, `SetCredentials` receives a password
via a configuration struct and returns it, and `RotateRootCredentials` generated a
password and was expected to return an updated copy of its entire configuration
with the new password.
4. The `SetCredentials` and `RotateRootCredentials` used for static credential rotation,
and rotating the root user's credentials respectively were essentially the same operation:
change a user's password. The only practical difference was which user it was referring
to. This was especially evident when `SetCredentials` was used when rotating root
credentials (unless static credential rotation wasn't supported by the plugin in question).
5. The old interface included both `Init` and `Initialize` adding to the confusion.
The new interface is roughly modeled after a [gRPC](https://grpc.io/) interface. It has improved
future compatibility by not requiring changes to the interface definition to add additional data
in the requests or responses. It also simplifies the interface by merging several into a single
function call.
### Upgrading your custom database
Vault 1.6 supports both version 4 and version 5 database plugins. The support for version 4
plugins will be removed in a future release. To determine if a plugin is using version 4 or
version 5, the following is a list of changes in no particular order that you can check against
your plugin to determine the version:
1. The import path for version 4 is `github.com/hashicorp/vault/sdk/database/dbplugin`
whereas the import path for version 5 is `github.com/hashicorp/vault/sdk/database/dbplugin/v5`
2. Version 4 has the following functions: `Initialize`, `Init`, `CreateUser`, `RenewUser`,
`RevokeUser`, `SetCredentials`, `RotateRootCredentials`, `Type`, and `Close`. You can see the
full function signatures in `sdk/database/dbplugin/plugin.go`.
3. Version 5 has the following functions: `Initialize`, `NewUser`, `UpdateUser`, `DeleteUser`,
`Type`, and `Close`. You can see the full function signatures in
`sdk/database/dbplugin/v5/database.go`.
If you are using a version 4 custom database plugin, the following are basic instructions
for upgrading to version 5.
-> In version 4, password generation was the responsibility of the plugin. This is no longer
the case with version 5. Vault is responsible for generating passwords and passing them to
the plugin via `NewUserRequest.Password` and `UpdateUserRequest.Password.NewPassword`.
1. Change the import path from `github.com/hashicorp/vault/sdk/database/dbplugin` to
`github.com/hashicorp/vault/sdk/database/dbplugin/v5`. The package name is the same, so any
references to `dbplugin` can remain as long as those symbols exist within the new package
(such as the `Serve` function).
2. An easy way to see what functions need to be implemented is to put the following as a
global variable within your package: `var _ dbplugin.Database = (*MyDatabase)(nil)`. This
will fail to compile if the `MyDatabase` type does not adhere to the `dbplugin.Database` interface.
3. Replace `Init` and `Initialize` with the new `Initialize` function definition. The fields that
`Init` was taking (`config` and `verifyConnection`) are now wrapped into `InitializeRequest`.
The returned `map[string]interface{}` object is now wrapped into `InitializeResponse`.
Only `Initialize` is needed to adhere to the `Database` interface.
4. Update `CreateUser` to `NewUser`. The `NewUserRequest` object contains the username and
password of the user to be created. It also includes a list of statements for creating the
user as well as several other fields that may or may not be applicable. Your custom plugin
should use the password provided in the request, not generate one. If you generate a password
instead, Vault will not know about it and will give the caller the wrong password.
5. `SetCredentials`, `RotateRootCredentials`, and `RenewUser` are combined into `UpdateUser`.
The request object, `UpdateUserRequest` contains three parts: the username to change, a
`ChangePassword` and a `ChangeExpiration` object. When one of the objects is not nil, this
indicates that particular field (password or expiration) needs to change. For instance, if
the `ChangePassword` field is not-nil, the user's password should be changed. This is
equivalent to calling `SetCredentials`. If the `ChangeExpiration` field is not-nil, the
user's expiration date should be changed. This is equivalent to calling `RenewUser`.
Many databases don't need to do anything with the updated expiration.
6. Update `RevokeUser` to `DeleteUser`. This is the simplest change. The username to be
deleted is enclosed in the `DeleteUserRequest` object.