open-vault/plugins/database/influxdb/connection_producer.go

276 lines
8.4 KiB
Go
Raw Normal View History

package influxdb
import (
"context"
"crypto/tls"
"fmt"
"sync"
"time"
"github.com/hashicorp/errwrap"
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"github.com/hashicorp/vault/sdk/database/dbplugin"
2019-04-15 18:10:07 +00:00
"github.com/hashicorp/vault/sdk/database/helper/connutil"
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"github.com/hashicorp/vault/sdk/database/helper/dbutil"
"github.com/hashicorp/vault/sdk/helper/certutil"
"github.com/hashicorp/vault/sdk/helper/parseutil"
2019-04-12 22:26:54 +00:00
"github.com/hashicorp/vault/sdk/helper/tlsutil"
influx "github.com/influxdata/influxdb/client/v2"
"github.com/mitchellh/mapstructure"
)
// influxdbConnectionProducer implements ConnectionProducer and provides an
// interface for influxdb databases to make connections.
type influxdbConnectionProducer struct {
Host string `json:"host" structs:"host" mapstructure:"host"`
Username string `json:"username" structs:"username" mapstructure:"username"`
Password string `json:"password" structs:"password" mapstructure:"password"`
Port string `json:"port" structs:"port" mapstructure:"port"` //default to 8086
TLS bool `json:"tls" structs:"tls" mapstructure:"tls"`
InsecureTLS bool `json:"insecure_tls" structs:"insecure_tls" mapstructure:"insecure_tls"`
ConnectTimeoutRaw interface{} `json:"connect_timeout" structs:"connect_timeout" mapstructure:"connect_timeout"`
TLSMinVersion string `json:"tls_min_version" structs:"tls_min_version" mapstructure:"tls_min_version"`
PemBundle string `json:"pem_bundle" structs:"pem_bundle" mapstructure:"pem_bundle"`
PemJSON string `json:"pem_json" structs:"pem_json" mapstructure:"pem_json"`
connectTimeout time.Duration
certificate string
privateKey string
issuingCA string
rawConfig map[string]interface{}
Initialized bool
Type string
client influx.Client
sync.Mutex
}
func (i *influxdbConnectionProducer) Initialize(ctx context.Context, conf map[string]interface{}, verifyConnection bool) error {
_, err := i.Init(ctx, conf, verifyConnection)
return err
}
func (i *influxdbConnectionProducer) Init(ctx context.Context, conf map[string]interface{}, verifyConnection bool) (map[string]interface{}, error) {
i.Lock()
defer i.Unlock()
i.rawConfig = conf
err := mapstructure.WeakDecode(conf, i)
if err != nil {
return nil, err
}
if i.ConnectTimeoutRaw == nil {
i.ConnectTimeoutRaw = "0s"
}
if i.Port == "" {
i.Port = "8086"
}
i.connectTimeout, err = parseutil.ParseDurationSecond(i.ConnectTimeoutRaw)
if err != nil {
return nil, errwrap.Wrapf("invalid connect_timeout: {{err}}", err)
}
switch {
case len(i.Host) == 0:
return nil, fmt.Errorf("host cannot be empty")
case len(i.Username) == 0:
return nil, fmt.Errorf("username cannot be empty")
case len(i.Password) == 0:
return nil, fmt.Errorf("password cannot be empty")
}
var certBundle *certutil.CertBundle
var parsedCertBundle *certutil.ParsedCertBundle
switch {
case len(i.PemJSON) != 0:
parsedCertBundle, err = certutil.ParsePKIJSON([]byte(i.PemJSON))
if err != nil {
return nil, errwrap.Wrapf("could not parse given JSON; it must be in the format of the output of the PKI backend certificate issuing command: {{err}}", err)
}
certBundle, err = parsedCertBundle.ToCertBundle()
if err != nil {
return nil, errwrap.Wrapf("Error marshaling PEM information: {{err}}", err)
}
i.certificate = certBundle.Certificate
i.privateKey = certBundle.PrivateKey
i.issuingCA = certBundle.IssuingCA
i.TLS = true
case len(i.PemBundle) != 0:
parsedCertBundle, err = certutil.ParsePEMBundle(i.PemBundle)
if err != nil {
return nil, errwrap.Wrapf("Error parsing the given PEM information: {{err}}", err)
}
certBundle, err = parsedCertBundle.ToCertBundle()
if err != nil {
return nil, errwrap.Wrapf("Error marshaling PEM information: {{err}}", err)
}
i.certificate = certBundle.Certificate
i.privateKey = certBundle.PrivateKey
i.issuingCA = certBundle.IssuingCA
i.TLS = true
}
// Set initialized to true at this point since all fields are set,
// and the connection can be established at a later time.
i.Initialized = true
if verifyConnection {
if _, err := i.Connection(ctx); err != nil {
return nil, errwrap.Wrapf("error verifying connection: {{err}}", err)
}
}
return conf, nil
}
func (i *influxdbConnectionProducer) Connection(_ context.Context) (interface{}, error) {
if !i.Initialized {
return nil, connutil.ErrNotInitialized
}
// If we already have a DB, return it
if i.client != nil {
return i.client, nil
}
cli, err := i.createClient()
if err != nil {
return nil, err
}
// Store the session in backend for reuse
i.client = cli
return cli, nil
}
func (i *influxdbConnectionProducer) Close() error {
// Grab the write lock
i.Lock()
defer i.Unlock()
if i.client != nil {
i.client.Close()
}
i.client = nil
return nil
}
func (i *influxdbConnectionProducer) createClient() (influx.Client, error) {
clientConfig := influx.HTTPConfig{
Addr: fmt.Sprintf("http://%s:%s", i.Host, i.Port),
Username: i.Username,
Password: i.Password,
UserAgent: "vault-influxdb-plugin",
Timeout: i.connectTimeout,
}
if i.TLS {
var tlsConfig *tls.Config
if len(i.certificate) > 0 || len(i.issuingCA) > 0 {
if len(i.certificate) > 0 && len(i.privateKey) == 0 {
return nil, fmt.Errorf("found certificate for TLS authentication but no private key")
}
certBundle := &certutil.CertBundle{}
if len(i.certificate) > 0 {
certBundle.Certificate = i.certificate
certBundle.PrivateKey = i.privateKey
}
if len(i.issuingCA) > 0 {
certBundle.IssuingCA = i.issuingCA
}
parsedCertBundle, err := certBundle.ToParsedCertBundle()
if err != nil {
return nil, errwrap.Wrapf("failed to parse certificate bundle: {{err}}", err)
}
tlsConfig, err = parsedCertBundle.GetTLSConfig(certutil.TLSClient)
if err != nil || tlsConfig == nil {
return nil, errwrap.Wrapf(fmt.Sprintf("failed to get TLS configuration: tlsConfig:%#v err:{{err}}", tlsConfig), err)
}
tlsConfig.InsecureSkipVerify = i.InsecureTLS
if i.TLSMinVersion != "" {
var ok bool
tlsConfig.MinVersion, ok = tlsutil.TLSLookup[i.TLSMinVersion]
if !ok {
return nil, fmt.Errorf("invalid 'tls_min_version' in config")
}
} else {
// MinVersion was not being set earlier. Reset it to
// zero to gracefully handle upgrades.
tlsConfig.MinVersion = 0
}
}
clientConfig.TLSConfig = tlsConfig
clientConfig.Addr = fmt.Sprintf("https://%s:%s", i.Host, i.Port)
}
cli, err := influx.NewHTTPClient(clientConfig)
if err != nil {
return nil, errwrap.Wrapf("error creating client: {{err}}", err)
}
// Checking server status
_, _, err = cli.Ping(i.connectTimeout)
if err != nil {
return nil, errwrap.Wrapf("error checking cluster status: {{err}}", err)
}
// verifying infos about the connection
isAdmin, err := isUserAdmin(cli, i.Username)
if err != nil {
return nil, errwrap.Wrapf("error getting if provided username is admin: {{err}}", err)
}
if !isAdmin {
return nil, fmt.Errorf("the provided user is not an admin of the influxDB server")
}
return cli, nil
}
func (i *influxdbConnectionProducer) secretValues() map[string]interface{} {
return map[string]interface{}{
i.Password: "[password]",
i.PemBundle: "[pem_bundle]",
i.PemJSON: "[pem_json]",
}
}
func isUserAdmin(cli influx.Client, user string) (bool, error) {
q := influx.NewQuery("SHOW USERS", "", "")
response, err := cli.Query(q)
if err != nil {
return false, err
}
if response.Error() != nil {
return false, response.Error()
}
for _, res := range response.Results {
for _, serie := range res.Series {
for _, val := range serie.Values {
if val[0].(string) == user && val[1].(bool) == true {
return true, nil
}
}
}
}
return false, fmt.Errorf("the provided username is not a valid user in the influxdb")
}
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
// SetCredentials uses provided information to set/create a user in the
// database. Unlike CreateUser, this method requires a username be provided and
// uses the name given, instead of generating a name. This is used for creating
// and setting the password of static accounts, as well as rolling back
// passwords in the database in the event an updated database fails to save in
// Vault's storage.
func (i *influxdbConnectionProducer) SetCredentials(ctx context.Context, statements dbplugin.Statements, staticUser dbplugin.StaticUserConfig) (username, password string, err error) {
return "", "", dbutil.Unimplemented()
}