open-vault/builtin/logical/database/path_roles_test.go

861 lines
24 KiB
Go
Raw Normal View History

Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
package database
import (
"context"
"encoding/json"
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"errors"
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
"strings"
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"testing"
"time"
"github.com/go-test/deep"
"github.com/hashicorp/vault/helper/namespace"
postgreshelper "github.com/hashicorp/vault/helper/testhelpers/postgresql"
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
v5 "github.com/hashicorp/vault/sdk/database/dbplugin/v5"
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"github.com/hashicorp/vault/sdk/logical"
"github.com/stretchr/testify/assert"
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
"github.com/stretchr/testify/mock"
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
)
func TestBackend_Roles_CredentialTypes(t *testing.T) {
config := logical.TestBackendConfig()
config.System = logical.TestSystemView()
config.StorageView = &logical.InmemStorage{}
b, err := Factory(context.Background(), config)
if err != nil {
t.Fatal(err)
}
type args struct {
credentialType v5.CredentialType
credentialConfig map[string]string
}
tests := []struct {
name string
args args
wantErr bool
expectedResp map[string]interface{}
}{
{
name: "role with invalid credential type",
args: args{
credentialType: v5.CredentialType(10),
},
wantErr: true,
},
{
name: "role with invalid credential type and valid credential config",
args: args{
credentialType: v5.CredentialType(7),
credentialConfig: map[string]string{
"password_policy": "test-policy",
},
},
wantErr: true,
},
{
name: "role with password credential type",
args: args{
credentialType: v5.CredentialTypePassword,
},
expectedResp: map[string]interface{}{
"credential_type": v5.CredentialTypePassword.String(),
"credential_config": nil,
},
},
{
name: "role with password credential type and configuration",
args: args{
credentialType: v5.CredentialTypePassword,
credentialConfig: map[string]string{
"password_policy": "test-policy",
},
},
expectedResp: map[string]interface{}{
"credential_type": v5.CredentialTypePassword.String(),
"credential_config": map[string]interface{}{
"password_policy": "test-policy",
},
},
},
{
name: "role with rsa_private_key credential type and default configuration",
args: args{
credentialType: v5.CredentialTypeRSAPrivateKey,
},
expectedResp: map[string]interface{}{
"credential_type": v5.CredentialTypeRSAPrivateKey.String(),
"credential_config": map[string]interface{}{
"key_bits": json.Number("2048"),
"format": "pkcs8",
},
},
},
{
name: "role with rsa_private_key credential type and 2048 bit configuration",
args: args{
credentialType: v5.CredentialTypeRSAPrivateKey,
credentialConfig: map[string]string{
"key_bits": "2048",
},
},
expectedResp: map[string]interface{}{
"credential_type": v5.CredentialTypeRSAPrivateKey.String(),
"credential_config": map[string]interface{}{
"key_bits": json.Number("2048"),
"format": "pkcs8",
},
},
},
{
name: "role with rsa_private_key credential type and 3072 bit configuration",
args: args{
credentialType: v5.CredentialTypeRSAPrivateKey,
credentialConfig: map[string]string{
"key_bits": "3072",
},
},
expectedResp: map[string]interface{}{
"credential_type": v5.CredentialTypeRSAPrivateKey.String(),
"credential_config": map[string]interface{}{
"key_bits": json.Number("3072"),
"format": "pkcs8",
},
},
},
{
name: "role with rsa_private_key credential type and 4096 bit configuration",
args: args{
credentialType: v5.CredentialTypeRSAPrivateKey,
credentialConfig: map[string]string{
"key_bits": "4096",
},
},
expectedResp: map[string]interface{}{
"credential_type": v5.CredentialTypeRSAPrivateKey.String(),
"credential_config": map[string]interface{}{
"key_bits": json.Number("4096"),
"format": "pkcs8",
},
},
},
{
name: "role with rsa_private_key credential type invalid key_bits configuration",
args: args{
credentialType: v5.CredentialTypeRSAPrivateKey,
credentialConfig: map[string]string{
"key_bits": "256",
},
},
wantErr: true,
},
{
name: "role with rsa_private_key credential type invalid format configuration",
args: args{
credentialType: v5.CredentialTypeRSAPrivateKey,
credentialConfig: map[string]string{
"format": "pkcs1",
},
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
req := &logical.Request{
Operation: logical.CreateOperation,
Path: "roles/test",
Storage: config.StorageView,
Data: map[string]interface{}{
"db_name": "test-database",
"creation_statements": "CREATE USER {{name}}",
"credential_type": tt.args.credentialType.String(),
"credential_config": tt.args.credentialConfig,
},
}
// Create the role
resp, err := b.HandleRequest(context.Background(), req)
if tt.wantErr {
assert.True(t, resp.IsError(), "expected error")
return
}
assert.False(t, resp.IsError())
assert.Nil(t, err)
// Read the role
req.Operation = logical.ReadOperation
resp, err = b.HandleRequest(context.Background(), req)
assert.False(t, resp.IsError())
assert.Nil(t, err)
for k, v := range tt.expectedResp {
assert.Equal(t, v, resp.Data[k])
}
// Delete the role
req.Operation = logical.DeleteOperation
resp, err = b.HandleRequest(context.Background(), req)
assert.False(t, resp.IsError())
assert.Nil(t, err)
})
}
}
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
func TestBackend_StaticRole_Config(t *testing.T) {
cluster, sys := getCluster(t)
defer cluster.Cleanup()
config := logical.TestBackendConfig()
config.StorageView = &logical.InmemStorage{}
config.System = sys
lb, err := Factory(context.Background(), config)
if err != nil {
t.Fatal(err)
}
b, ok := lb.(*databaseBackend)
if !ok {
t.Fatal("could not convert to db backend")
}
defer b.Cleanup(context.Background())
cleanup, connURL := postgreshelper.PrepareTestContainer(t, "")
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
defer cleanup()
// create the database user
createTestPGUser(t, connURL, dbUser, "password", testRoleStaticCreate)
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
// Configure a connection
data := map[string]interface{}{
"connection_url": connURL,
"plugin_name": "postgresql-database-plugin",
"verify_connection": false,
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
"allowed_roles": []string{"plugin-role-test"},
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"name": "plugin-test",
}
req := &logical.Request{
Operation: logical.UpdateOperation,
Path: "config/plugin-test",
Storage: config.StorageView,
Data: data,
}
resp, err := b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
// Test static role creation scenarios. Uses a map, so there is no guaranteed
// ordering, so each case cleans up by deleting the role
testCases := map[string]struct {
account map[string]interface{}
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
path string
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
expected map[string]interface{}
err error
}{
"basic": {
account: map[string]interface{}{
"username": dbUser,
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"rotation_period": "5400s",
},
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
path: "plugin-role-test",
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
expected: map[string]interface{}{
"username": dbUser,
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"rotation_period": float64(5400),
},
},
"missing rotation period": {
account: map[string]interface{}{
"username": dbUser,
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
},
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
path: "plugin-role-test",
err: errors.New("rotation_period is required to create static accounts"),
},
"disallowed role config": {
account: map[string]interface{}{
"username": dbUser,
"rotation_period": "5400s",
},
path: "disallowed-role",
err: errors.New("\"disallowed-role\" is not an allowed role"),
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
},
}
for name, tc := range testCases {
t.Run(name, func(t *testing.T) {
data := map[string]interface{}{
"name": "plugin-role-test",
"db_name": "plugin-test",
"rotation_statements": testRoleStaticUpdate,
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
}
for k, v := range tc.account {
data[k] = v
}
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
path := "static-roles/" + tc.path
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
req := &logical.Request{
Operation: logical.CreateOperation,
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
Path: path,
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
if tc.err == nil {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
if err != nil && tc.err.Error() == err.Error() {
// errors match
return
}
if err == nil && tc.err.Error() == resp.Error().Error() {
// errors match
return
}
t.Fatalf("expected err message: (%s), got (%s), response error: (%s)", tc.err, err, resp.Error())
}
if tc.err != nil {
if err == nil || (resp == nil || !resp.IsError()) {
t.Fatal("expected error, got none")
}
}
// Read the role
data = map[string]interface{}{}
req = &logical.Request{
Operation: logical.ReadOperation,
Path: "static-roles/plugin-role-test",
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
expected := tc.expected
actual := make(map[string]interface{})
dataKeys := []string{"username", "password", "last_vault_rotation", "rotation_period"}
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
for _, key := range dataKeys {
if v, ok := resp.Data[key]; ok {
actual[key] = v
}
}
if len(tc.expected) > 0 {
// verify a password is returned, but we don't care what it's value is
if actual["password"] == "" {
t.Fatalf("expected result to contain password, but none found")
}
if v, ok := actual["last_vault_rotation"].(time.Time); !ok {
t.Fatalf("expected last_vault_rotation to be set to time.Time type, got: %#v", v)
}
// delete these values before the comparison, since we can't know them in
// advance
delete(actual, "password")
delete(actual, "last_vault_rotation")
if diff := deep.Equal(expected, actual); diff != nil {
t.Fatal(diff)
}
}
if len(tc.expected) == 0 && resp.Data["static_account"] != nil {
t.Fatalf("got unexpected static_account info: %#v", actual)
}
if diff := deep.Equal(resp.Data["db_name"], "plugin-test"); diff != nil {
t.Fatal(diff)
}
// Delete role for next run
req = &logical.Request{
Operation: logical.DeleteOperation,
Path: "static-roles/plugin-role-test",
Storage: config.StorageView,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
})
}
}
func TestBackend_StaticRole_Updates(t *testing.T) {
cluster, sys := getCluster(t)
defer cluster.Cleanup()
config := logical.TestBackendConfig()
config.StorageView = &logical.InmemStorage{}
config.System = sys
lb, err := Factory(context.Background(), config)
if err != nil {
t.Fatal(err)
}
b, ok := lb.(*databaseBackend)
if !ok {
t.Fatal("could not convert to db backend")
}
defer b.Cleanup(context.Background())
cleanup, connURL := postgreshelper.PrepareTestContainer(t, "")
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
defer cleanup()
// create the database user
createTestPGUser(t, connURL, dbUser, "password", testRoleStaticCreate)
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
// Configure a connection
data := map[string]interface{}{
"connection_url": connURL,
"plugin_name": "postgresql-database-plugin",
"verify_connection": false,
"allowed_roles": []string{"*"},
"name": "plugin-test",
}
req := &logical.Request{
Operation: logical.UpdateOperation,
Path: "config/plugin-test",
Storage: config.StorageView,
Data: data,
}
resp, err := b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
data = map[string]interface{}{
"name": "plugin-role-test-updates",
"db_name": "plugin-test",
"rotation_statements": testRoleStaticUpdate,
"username": dbUser,
"rotation_period": "5400s",
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
}
req = &logical.Request{
Operation: logical.CreateOperation,
Path: "static-roles/plugin-role-test-updates",
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
// Read the role
data = map[string]interface{}{}
req = &logical.Request{
Operation: logical.ReadOperation,
Path: "static-roles/plugin-role-test-updates",
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
rotation := resp.Data["rotation_period"].(float64)
// capture the password to verify it doesn't change
req = &logical.Request{
Operation: logical.ReadOperation,
Path: "static-creds/plugin-role-test-updates",
Storage: config.StorageView,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
username := resp.Data["username"].(string)
password := resp.Data["password"].(string)
if username == "" || password == "" {
t.Fatalf("expected both username/password, got (%s), (%s)", username, password)
}
// update rotation_period
updateData := map[string]interface{}{
"name": "plugin-role-test-updates",
"db_name": "plugin-test",
"username": dbUser,
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"rotation_period": "6400s",
}
req = &logical.Request{
Operation: logical.UpdateOperation,
Path: "static-roles/plugin-role-test-updates",
Storage: config.StorageView,
Data: updateData,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
// re-read the role
data = map[string]interface{}{}
req = &logical.Request{
Operation: logical.ReadOperation,
Path: "static-roles/plugin-role-test-updates",
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
newRotation := resp.Data["rotation_period"].(float64)
if newRotation == rotation {
t.Fatalf("expected change in rotation, but got old value: %#v", newRotation)
}
// re-capture the password to ensure it did not change
req = &logical.Request{
Operation: logical.ReadOperation,
Path: "static-creds/plugin-role-test-updates",
Storage: config.StorageView,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
if username != resp.Data["username"].(string) {
t.Fatalf("usernames dont match!: (%s) / (%s)", username, resp.Data["username"].(string))
}
if password != resp.Data["password"].(string) {
t.Fatalf("passwords dont match!: (%s) / (%s)", password, resp.Data["password"].(string))
}
// verify that rotation_period is only required when creating
updateData = map[string]interface{}{
"name": "plugin-role-test-updates",
"db_name": "plugin-test",
"username": dbUser,
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
"rotation_statements": testRoleStaticUpdateRotation,
}
req = &logical.Request{
Operation: logical.UpdateOperation,
Path: "static-roles/plugin-role-test-updates",
Storage: config.StorageView,
Data: updateData,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
// verify updating static username returns an error
updateData = map[string]interface{}{
"name": "plugin-role-test-updates",
"db_name": "plugin-test",
"username": "statictestmodified",
}
req = &logical.Request{
Operation: logical.UpdateOperation,
Path: "static-roles/plugin-role-test-updates",
Storage: config.StorageView,
Data: updateData,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || !resp.IsError() {
t.Fatal("expected error on updating name")
}
err = resp.Error()
if err.Error() != "cannot update static account username" {
t.Fatalf("expected error on updating name, got: %s", err)
}
}
func TestBackend_StaticRole_Role_name_check(t *testing.T) {
cluster, sys := getCluster(t)
defer cluster.Cleanup()
config := logical.TestBackendConfig()
config.StorageView = &logical.InmemStorage{}
config.System = sys
lb, err := Factory(context.Background(), config)
if err != nil {
t.Fatal(err)
}
b, ok := lb.(*databaseBackend)
if !ok {
t.Fatal("could not convert to db backend")
}
defer b.Cleanup(context.Background())
cleanup, connURL := postgreshelper.PrepareTestContainer(t, "")
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
defer cleanup()
// create the database user
createTestPGUser(t, connURL, dbUser, "password", testRoleStaticCreate)
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
// Configure a connection
data := map[string]interface{}{
"connection_url": connURL,
"plugin_name": "postgresql-database-plugin",
"verify_connection": false,
"allowed_roles": []string{"*"},
"name": "plugin-test",
}
req := &logical.Request{
Operation: logical.UpdateOperation,
Path: "config/plugin-test",
Storage: config.StorageView,
Data: data,
}
resp, err := b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
// non-static role
data = map[string]interface{}{
"name": "plugin-role-test",
"db_name": "plugin-test",
"creation_statements": testRoleStaticCreate,
"rotation_statements": testRoleStaticUpdate,
"revocation_statements": defaultRevocationSQL,
"default_ttl": "5m",
"max_ttl": "10m",
}
req = &logical.Request{
Operation: logical.CreateOperation,
Path: "roles/plugin-role-test",
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
// create a static role with the same name, and expect failure
// static role
data = map[string]interface{}{
"name": "plugin-role-test",
"db_name": "plugin-test",
"creation_statements": testRoleStaticCreate,
"rotation_statements": testRoleStaticUpdate,
"revocation_statements": defaultRevocationSQL,
}
req = &logical.Request{
Operation: logical.CreateOperation,
Path: "static-roles/plugin-role-test",
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil {
t.Fatal(err)
}
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
if resp == nil || !resp.IsError() {
t.Fatalf("expected error, got none")
}
// repeat, with a static role first
data = map[string]interface{}{
"name": "plugin-role-test-2",
"db_name": "plugin-test",
"rotation_statements": testRoleStaticUpdate,
"username": dbUser,
"rotation_period": "1h",
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
}
req = &logical.Request{
Operation: logical.CreateOperation,
Path: "static-roles/plugin-role-test-2",
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil || (resp != nil && resp.IsError()) {
t.Fatalf("err:%s resp:%#v\n", err, resp)
}
// create a non-static role with the same name, and expect failure
data = map[string]interface{}{
"name": "plugin-role-test-2",
"db_name": "plugin-test",
"creation_statements": testRoleStaticCreate,
"revocation_statements": defaultRevocationSQL,
"default_ttl": "5m",
"max_ttl": "10m",
}
req = &logical.Request{
Operation: logical.CreateOperation,
Path: "roles/plugin-role-test-2",
Storage: config.StorageView,
Data: data,
}
resp, err = b.HandleRequest(namespace.RootContext(nil), req)
if err != nil {
t.Fatal(err)
}
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
if resp == nil || !resp.IsError() {
t.Fatalf("expected error, got none")
}
}
Port: Premature Rotation For autorotate (#12563) * port of ldap fix for early cred rotation * some more porting * another couple lines to port * final commits before report * remove deadlock * needs testing * updates * Sync with OpenLDAP PR * Update the update error handling for items not found in the queue * WIP unit tests * Need to configure DB mount correctly, with db type mockv5 * Need to find a way to inject errors into that mock db * throw error on role creation failure * do not swallow error on role creation * comment out wip tests and add in a test for disallowed role * Use newly generated password in WAL Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> * return err on popFromRotationQueueByKey error; cleanup on setStaticAccount * test: fix TestPlugin_lifecycle * Uncomment and fix unit tests * Use mock database plugin to inject errors * Tidy test code to rely less on code internals where possible * Some stronger test assertions * Undo logging updates * Add changelog * Remove ticker and background threads from WAL tests * Keep pre-existing API behaviour of allowing update static role to act as a create * Switch test back to update operation * Revert my revert, and fix some test bugs * Fix TestBackend_StaticRole_LockRegression * clean up defer on TestPlugin_lifecycle * unwrap reqs on cleanup * setStaticAccount: don't hold a write lock * TestStoredWALsCorrectlyProcessed: set replication state to unknown Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com> Co-authored-by: Michael Golowka <72365+pcman312@users.noreply.github.com> Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2021-09-22 00:45:04 +00:00
func TestWALsStillTrackedAfterUpdate(t *testing.T) {
ctx := context.Background()
b, storage, mockDB := getBackend(t)
defer b.Cleanup(ctx)
configureDBMount(t, storage)
createRole(t, b, storage, mockDB, "hashicorp")
generateWALFromFailedRotation(t, b, storage, mockDB, "hashicorp")
requireWALs(t, storage, 1)
resp, err := b.HandleRequest(ctx, &logical.Request{
Operation: logical.UpdateOperation,
Path: "static-roles/hashicorp",
Storage: storage,
Data: map[string]interface{}{
"username": "hashicorp",
"rotation_period": "600s",
},
})
if err != nil || (resp != nil && resp.IsError()) {
t.Fatal(resp, err)
}
walIDs := requireWALs(t, storage, 1)
// Now when we trigger a manual rotate, it should use the WAL's new password
// which will tell us that the in-memory structure still kept track of the
// WAL in addition to it still being in storage.
wal, err := b.findStaticWAL(ctx, storage, walIDs[0])
if err != nil {
t.Fatal(err)
}
rotateRole(t, b, storage, mockDB, "hashicorp")
role, err := b.StaticRole(ctx, storage, "hashicorp")
if err != nil {
t.Fatal(err)
}
if role.StaticAccount.Password != wal.NewPassword {
t.Fatal()
}
requireWALs(t, storage, 0)
}
func TestWALsDeletedOnRoleCreationFailed(t *testing.T) {
ctx := context.Background()
b, storage, mockDB := getBackend(t)
defer b.Cleanup(ctx)
configureDBMount(t, storage)
for i := 0; i < 3; i++ {
mockDB.On("UpdateUser", mock.Anything, mock.Anything).
Return(v5.UpdateUserResponse{}, errors.New("forced error")).
Once()
resp, err := b.HandleRequest(ctx, &logical.Request{
Operation: logical.CreateOperation,
Path: "static-roles/hashicorp",
Storage: storage,
Data: map[string]interface{}{
"username": "hashicorp",
"db_name": "mockv5",
"rotation_period": "5s",
},
})
if err == nil {
t.Fatal("expected error from DB")
}
if !strings.Contains(err.Error(), "forced error") {
t.Fatal("expected forced error message", resp, err)
}
}
requireWALs(t, storage, 0)
}
func TestWALsDeletedOnRoleDeletion(t *testing.T) {
ctx := context.Background()
b, storage, mockDB := getBackend(t)
defer b.Cleanup(ctx)
configureDBMount(t, storage)
// Create the roles
roleNames := []string{"hashicorp", "2"}
for _, roleName := range roleNames {
createRole(t, b, storage, mockDB, roleName)
}
// Fail to rotate the roles
for _, roleName := range roleNames {
generateWALFromFailedRotation(t, b, storage, mockDB, roleName)
}
// Should have 2 WALs hanging around
requireWALs(t, storage, 2)
// Delete one of the static roles
resp, err := b.HandleRequest(ctx, &logical.Request{
Operation: logical.DeleteOperation,
Path: "static-roles/hashicorp",
Storage: storage,
})
if err != nil || (resp != nil && resp.IsError()) {
t.Fatal(resp, err)
}
// 1 WAL should be cleared by the delete
requireWALs(t, storage, 1)
}
func createRole(t *testing.T, b *databaseBackend, storage logical.Storage, mockDB *mockNewDatabase, roleName string) {
t.Helper()
mockDB.On("UpdateUser", mock.Anything, mock.Anything).
Return(v5.UpdateUserResponse{}, nil).
Once()
resp, err := b.HandleRequest(context.Background(), &logical.Request{
Operation: logical.CreateOperation,
Path: "static-roles/" + roleName,
Storage: storage,
Data: map[string]interface{}{
"username": roleName,
"db_name": "mockv5",
"rotation_period": "86400s",
},
})
if err != nil || (resp != nil && resp.IsError()) {
t.Fatal(resp, err)
}
}
Combined Database Backend: Static Accounts (#6834) * Add priority queue to sdk * fix issue of storing pointers and now copy * update to use copy structure * Remove file, put Item struct def. into other file * add link * clean up docs * refactor internal data structure to hide heap method implementations. Other cleanup after feedback * rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods * updates after feedback * refactoring/renaming * guard against pushing a nil item * minor updates after feedback * Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC * Initial Combined database backend implementation of static accounts and automatic rotation * vendor updates * initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation * add lock and setup of rotation queue * vendor the queue * rebase on new method signature of queue * remove mongo tests for now * update default role sql * gofmt after rebase * cleanup after rebasing to remove checks for ErrNotFound error * rebase cdcr-priority-queue * vendor dependencies with 'go mod vendor' * website database docs for Static Role support * document the rotate-role API endpoint * postgres specific static role docs * use constants for paths * updates from review * remove dead code * combine and clarify error message for older plugins * Update builtin/logical/database/backend.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups from feedback * code and comment cleanups * move db.RLock higher to protect db.GenerateCredentials call * Return output with WALID if we failed to delete the WAL * Update builtin/logical/database/path_creds_create.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * updates after running 'make fmt' * update after running 'make proto' * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update comment and remove and rearrange some dead code * Update website/source/api/secret/databases/index.html.md Co-Authored-By: Jim Kalafut <jim@kalafut.net> * cleanups after review * Update sdk/database/dbplugin/grpc_transport.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * code cleanup after feedback * remove PasswordLastSet; it's not used * document GenerateCredentials and SetCredentials * Update builtin/logical/database/path_rotate_credentials.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * wrap pop and popbykey in backend methods to protect against nil cred rotation queue * use strings.HasPrefix instead of direct equality check for path * Forgot to commit this * updates after feedback * re-purpose an outdated test to now check that static and dynamic roles cannot share a name * check for unique name across dynamic and static roles * refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials * remove commented out code * refactor to have loadstaticwals filter out wals for roles that no longer exist * return error if nil input given * add nil check for input into setStaticAccount * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * add constant for queue tick time in seconds, used for comparrison in updates * Update builtin/logical/database/path_roles.go Co-Authored-By: Jim Kalafut <jim@kalafut.net> * code cleanup after review * remove misplaced code comment * remove commented out code * create a queue in the Factory method, even if it's never used * update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type * document new method * move rotation things into a specific file * rename test file and consolidate some static account tests * Update builtin/logical/database/path_roles.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * Update builtin/logical/database/rotation.go Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com> * update code comments, method names, and move more methods into rotation.go * update comments to be capitalized * remove the item from the queue before we try to destroy it * findStaticWAL returns an error * use lowercase keys when encoding WAL entries * small cleanups * remove vestigial static account check * remove redundant DeleteWAL call in populate queue * if we error on loading role, push back to queue with 10 second backoff * poll in initqueue to make sure the backend is setup and can write/delete data * add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false * add code comments on read-only loop * code comment updates * re-push if error returned from find static wal * add locksutil and acquire locks when pop'ing from the queue * grab exclusive locks for updating static roles * Add SetCredentials and GenerateCredentials stubs to mockPlugin * add a switch in initQueue to listen for cancelation * remove guard on zero time, it should have no affect * create a new context in Factory to pass on and use for closing the backend queue * restore master copy of vendor dir
2019-06-19 19:45:39 +00:00
const testRoleStaticCreate = `
CREATE ROLE "{{name}}" WITH
LOGIN
PASSWORD '{{password}}';
`
const testRoleStaticUpdate = `
ALTER USER "{{name}}" WITH PASSWORD '{{password}}';
`
const testRoleStaticUpdateRotation = `
ALTER USER "{{name}}" WITH PASSWORD '{{password}}';GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO "{{name}}";
`