open-vault/website/content/docs/secrets/databases/custom.mdx

---
layout: docs
page_title: Custom - Database - Secrets Engines
description: |-
  The database secrets engine allows new functionality to be added through a
  plugin interface without needing to modify Vault's core code. This allows you
  write your own code to generate credentials in any database you wish. It also
  allows databases that require dynamically linked libraries to be used as
  plugins while keeping Vault itself statically linked.
---

# Custom Database Secrets Engines

~> The interface for custom database plugins has changed in Vault 1.6. Vault will
continue to recognize the now deprecated version of this interface for some time.
If you are using a plugin with the deprecated interface, you should upgrade to the
newest version. See [Upgrading database plugins](#upgrading-database-plugins)
for more details.

~> **Advanced topic!** Plugin development is a highly advanced topic in Vault,
and is not required knowledge for day-to-day usage. If you don't plan on writing
any plugins, feel free to skip this section of the documentation.

The database secrets engine allows new functionality to be added through a
plugin interface without needing to modify Vault's core code. This allows you
write your own code to generate credentials in any database you wish. It also
allows databases that require dynamically linked libraries to be used as plugins
while keeping Vault itself statically linked.

Please read the [Plugins internals](/docs/plugins) docs for more
information about the plugin system before getting started building your
Database plugin.

Database plugins can be made to implement
[plugin multiplexing](/docs/plugins/plugin-architecture#plugin-multiplexing)
which allows a single plugin process to be used for multiple database
connections. To enable multiplexing, the plugin must be compiled with the
`ServeMultiplex` function call from Vault's `dbplugin` package.


## Plugin Interface

All plugins for the database secrets engine must implement the same interface. This interface
is found in `sdk/database/dbplugin/v5/database.go`

```go
type Database interface {
	// Initialize the database plugin. This is the equivalent of a constructor for the
	// database object itself.
	Initialize(ctx context.Context, req InitializeRequest) (InitializeResponse, error)

	// NewUser creates a new user within the database. This user is temporary in that it
	// will exist until the TTL expires.
	NewUser(ctx context.Context, req NewUserRequest) (NewUserResponse, error)

	// UpdateUser updates an existing user within the database.
	UpdateUser(ctx context.Context, req UpdateUserRequest) (UpdateUserResponse, error)

	// DeleteUser from the database. This should not error if the user didn't
	// exist prior to this call.
	DeleteUser(ctx context.Context, req DeleteUserRequest) (DeleteUserResponse, error)

	// Type returns the Name for the particular database backend implementation.
	// This type name is usually set as a constant within the database backend
	// implementation, e.g. "mysql" for the MySQL database backend. This is used
	// for things like metrics and logging. No behavior is switched on this.
	Type() (string, error)

	// Close attempts to close the underlying database connection that was
	// established by the backend.
	Close() error
}
```

Each of the request and response objects can also be found in `sdk/database/dbplugin/v5/database.go`.

In each of the requests, you will see at least 1 `Statements` object (in `UpdateUserRequest`
they are in sub-fields). This object represents the set of commands to run for that particular
operation. For the `NewUser` function, this is a set of commands to create the user (and often
set permissions for that user). These statements are from the following fields in the API:

| API Argument               | Request Object                                     |
| -------------------------- | -------------------------------------------------- |
| `creation_statements`      | `NewUserRequest.Statements.Commands`               |
| `revocation_statements`    | `DeleteUserRequest.Statements.Commands`            |
| `rollback_statements`      | `NewUserRequest.RollbackStatements.Commands`       |
| `renew_statements`         | `UpdateUserRequest.Expiration.Statements.Commands` |
| `rotation_statements`      | `UpdateUserRequest.Password.Statements.Commands`   |
| `root_rotation_statements` | `UpdateUserRequest.Password.Statements.Commands`   |

In many of the built-in plugins, they replace `{{name}}` (or `{{username}}`), `{{password}}`,
and/or `{{expiration}}` with the associated values. It is up to your plugin to perform these
string replacements. There is a helper function located in `sdk/database/helper/dbutil`
called `QueryHelper` that assists in doing this string replacement. You are not required to
use it, but it will make your plugin's behavior consistent with the built-in plugins.

The `InitializeRequest` object contains a map of keys to values. This data is what the
user specified as the configuration for the plugin. Your plugin should use this
data to make connections to the database. The response object contains a similar configuration
map. The response object should contain the configuration map that should be saved within Vault.
This allows the plugin to manipulate the configuration prior to saving it.

It is also passed a boolean value (`InitializeRequest.VerifyConnection`) indicating if your
plugin should initialize a connection to the database during the `Initialize` call. This
function is called when the configuration is written. This allows the user to know whether
the configuration is valid and able to connect to the database in question. If this is set to
false, no connection should be made during the `Initialize` call, but subsequent calls to the
other functions will need to open a connection.

## Serving A Plugin

### Serving A Plugin with Multiplexing

~> Plugin multiplexing requires `github.com/hashicorp/vault/sdk v0.4.0` or above.

The plugin runs as a separate binary outside of Vault, so the plugin itself
will need a `main` function. Use the `ServeMultiplex` function within
`sdk/database/dbplugin/v5` to serve your multiplexed plugin. You will also need
to pass some TLS configuration information that Vault uses when initializing
the plugin.

Below is an example setup:

```go
package main

import (
	"github.com/hashicorp/vault/api"
	dbplugin "github.com/hashicorp/vault/sdk/database/dbplugin/v5"
)

func main() {
	apiClientMeta := &api.PluginAPIClientMeta{}
	flags := apiClientMeta.FlagSet()
	flags.Parse(os.Args[1:])

	err := Run()
	if err != nil {
		log.Println(err)
		os.Exit(1)
	}
}

func Run() error {
	dbplugin.ServeMultiplex(dbType.(dbplugin.New))

	return nil
}

func New() (interface{}, error) {
	db, err := newDatabase()
	if err != nil {
		return nil, err
	}

	// This middleware isn't strictly required, but highly recommended to prevent accidentally exposing
	// values such as passwords in error messages. An example of this is included below
	db = dbplugin.NewDatabaseErrorSanitizerMiddleware(db, db.secretValues)
	return db, nil
}

type MyDatabase struct {
	// Variables for the database
	password string
}

func newDatabase() (MyDatabase, error) {
	// ...
	db := &MyDatabase{
		// ...
	}
	return db, nil
}

func (db *MyDatabase) secretValues() map[string]string {
	return map[string]string{
		db.password: "[password]",
	}
}
```

Replacing `MyDatabase` with the actual implementation of your database plugin.

### Serving A Plugin without Multiplexing

Serving a plugin without multiplexing requires calling the `Serve` function
from `sdk/database/dbplugin/v5` to serve your plugin. You will also need to
pass some TLS configuration information that Vault uses when initializing the
plugin.

The setup is exactly the same as the multiplexed case above, except for the
`Run` function:

```go
func Run() error {
	dbType, err := New()
	if err != nil {
		return err
	}

	dbplugin.Serve(dbType.(dbplugin.Database))

	return nil
}
```

## Running your plugin

The above main package, once built, will supply you with a binary of your
plugin. We also recommend if you are planning on distributing your plugin to
build with [gox](https://github.com/mitchellh/gox) for cross platform builds.

To use your plugin with the database secrets engine you need to place the binary in the
plugin directory as specified in the [plugin internals](/docs/plugins) docs.

You should now be able to register your plugin into the vault catalog. To do
this your token will need sudo permissions.

```shell-session
$ vault write sys/plugins/catalog/database/mydatabase-database-plugin \
    sha256="..." \
    command="mydatabase"
Success! Data written to: sys/plugins/catalog/database/mydatabase-database-plugin
```

Now you should be able to configure your plugin like any other:

```shell-session
$ vault write database/config/mydatabase \
    plugin_name=mydatabase-database-plugin \
    allowed_roles="readonly" \
    myplugins_connection_details="..."
```

## Upgrading database plugins to leverage plugin multiplexing

### Background

Scaling many external plugins can become resource intensive. To address
performance problems with scaling external plugins, database plugins can be
made to implement [plugin multiplexing](/docs/plugins/plugin-architecture#plugin-multiplexing)
which allows a single plugin process to be used for multiple database
connections. To enable multiplexing, the plugin must be compiled with the
`ServeMultiplex` function call from Vault's `dbplugin` package.

### Upgrading your database plugin to leverage plugin multiplexing

There is only one step required to upgrade from a non-multiplexed to a
multiplexed database plugin: Change the `Serve` function call to `ServeMultiplex`.

This will run the RPC server for the plugin just as before. However, the
`ServeMultiplex` function takes the factory function directly as its argument.
This factory function is a function that returns an object that implements the
[`dbplugin.Database` interface](/docs/secrets/databases/custom#plugin-interface).

### When should plugin multiplexing be avoided?

Some use cases that should avoid plugin multiplexing might include:

* Plugin process level separation is required
* Avoiding restart across all mounts/database connections for a plugin type on
  crashes or plugin reload calls

## Upgrading database plugins to the V5 interface

### Background

In Vault 1.6, the database interface changed. The new version is referred to as version 5
and the previous version as version 4. This is due to prior versioning of the interface
that was not explicitly exposed.

The new interface was introduced for several reasons:

1. [Password policies](/docs/concepts/password-policies) introduced in Vault 1.5 required
   that Vault be responsible for generating passwords. In the prior version, the database
   plugin was responsible for generating passwords. This prevented integration with
   password policies.
2. Passwords needed to be generated by database plugins. This meant that plugin authors
   were responsible for generating secure passwords. This should be done with a helper
   function available within the Vault SDK, however there was nothing preventing an
   author from generating insecure passwords.
3. There were a number of inconsistencies within the version 4 interface that made it
   confusing for authors. For instance: passwords were handled in 3 different ways.
   `CreateUser` generated a password and returned it, `SetCredentials` receives a password
   via a configuration struct and returns it, and `RotateRootCredentials` generated a
   password and was expected to return an updated copy of its entire configuration
   with the new password.
4. The `SetCredentials` and `RotateRootCredentials` used for static credential rotation,
   and rotating the root user's credentials respectively were essentially the same operation:
   change a user's password. The only practical difference was which user it was referring
   to. This was especially evident when `SetCredentials` was used when rotating root
   credentials (unless static credential rotation wasn't supported by the plugin in question).
5. The old interface included both `Init` and `Initialize` adding to the confusion.

The new interface is roughly modeled after a [gRPC](https://grpc.io/) interface. It has improved
future compatibility by not requiring changes to the interface definition to add additional data
in the requests or responses. It also simplifies the interface by merging several into a single
function call.

### Upgrading your custom database

Vault 1.6 supports both version 4 and version 5 database plugins. The support for version 4
plugins will be removed in a future release. Version 5 database plugins will not function with
Vault prior to version 1.6. If you upgrade your database plugins, ensure that you are only using
Vault 1.6 or later. To determine if a plugin is using version 4 or version 5, the following is a
list of changes in no particular order that you can check against your plugin to determine
the version:

1. The import path for version 4 is `github.com/hashicorp/vault/sdk/database/dbplugin`
   whereas the import path for version 5 is `github.com/hashicorp/vault/sdk/database/dbplugin/v5`
2. Version 4 has the following functions: `Initialize`, `Init`, `CreateUser`, `RenewUser`,
   `RevokeUser`, `SetCredentials`, `RotateRootCredentials`, `Type`, and `Close`. You can see the
   full function signatures in `sdk/database/dbplugin/plugin.go`.
3. Version 5 has the following functions: `Initialize`, `NewUser`, `UpdateUser`, `DeleteUser`,
   `Type`, and `Close`. You can see the full function signatures in
   `sdk/database/dbplugin/v5/database.go`.

If you are using a version 4 custom database plugin, the following are basic instructions
for upgrading to version 5.

-> In version 4, password generation was the responsibility of the plugin. This is no longer
the case with version 5. Vault is responsible for generating passwords and passing them to
the plugin via `NewUserRequest.Password` and `UpdateUserRequest.Password.NewPassword`.

1. Change the import path from `github.com/hashicorp/vault/sdk/database/dbplugin` to
   `github.com/hashicorp/vault/sdk/database/dbplugin/v5`. The package name is the same, so any
   references to `dbplugin` can remain as long as those symbols exist within the new package
   (such as the `Serve` function).
2. An easy way to see what functions need to be implemented is to put the following as a
   global variable within your package: `var _ dbplugin.Database = (*MyDatabase)(nil)`. This
   will fail to compile if the `MyDatabase` type does not adhere to the `dbplugin.Database` interface.
3. Replace `Init` and `Initialize` with the new `Initialize` function definition. The fields that
   `Init` was taking (`config` and `verifyConnection`) are now wrapped into `InitializeRequest`.
   The returned `map[string]interface{}` object is now wrapped into `InitializeResponse`.
   Only `Initialize` is needed to adhere to the `Database` interface.
4. Update `CreateUser` to `NewUser`. The `NewUserRequest` object contains the username and
   password of the user to be created. It also includes a list of statements for creating the
   user as well as several other fields that may or may not be applicable. Your custom plugin
   should use the password provided in the request, not generate one. If you generate a password
   instead, Vault will not know about it and will give the caller the wrong password.
5. `SetCredentials`, `RotateRootCredentials`, and `RenewUser` are combined into `UpdateUser`.
   The request object, `UpdateUserRequest` contains three parts: the username to change, a
   `ChangePassword` and a `ChangeExpiration` object. When one of the objects is not nil, this
   indicates that particular field (password or expiration) needs to change. For instance, if
   the `ChangePassword` field is not-nil, the user's password should be changed. This is
   equivalent to calling `SetCredentials`. If the `ChangeExpiration` field is not-nil, the
   user's expiration date should be changed. This is equivalent to calling `RenewUser`.
   Many databases don't need to do anything with the updated expiration.
6. Update `RevokeUser` to `DeleteUser`. This is the simplest change. The username to be
   deleted is enclosed in the `DeleteUserRequest` object.