Commit Graph

34 Commits

Author SHA1 Message Date
Becca Petrin 0e0c16b11a
Postgres: Correct parsing of multiline statements (#8512)
* add test reproducing issue

* add code fixing issue

* check for END in unquoted string frags

* move delimiters inside parens

* begin checking with stmt

* PR feedback

* fix comment

* add tests with templates

* update test name

* remove unnecessary backslashes from test
2020-03-17 12:45:25 -07:00
Jeff Malnick f46769b441
fix: rotateStatents -> rotateStatements (#8306) 2020-02-06 14:05:00 -08:00
Michael Golowka 70bcd2cc05
plugins/database: Allow both {{name}} and {{username}} in MySQL & Postgres (#8240)
* Allow {{name}} or {{username}} in psql templates

* Fix default rotation bug; allow {{user}} and {{username}}
2020-02-03 13:57:28 -07:00
Clint cd6b0b2de5 Combined Database backend: Add GenerateCredentials to the CredentialsProducer Interface (#7010)
* Add GenerateCredentials to the CredentialsProducer Interface, add default implementation

* Remove GenerateCredentials implementation from database plugins
2019-07-05 14:34:47 -04:00
Clint 4b00597609
Combined Database backend: remove create/delete support (#6951)
* remove create/update database user for static accounts

* update tests after create/delete removed

* small cleanups

* update postgresql setcredentials test
2019-06-23 15:58:07 -05:00
Clint b55303eddb
Combined Database Backend: Static Accounts (#6834)
* Add priority queue to sdk

* fix issue of storing pointers and now copy

* update to use copy structure

* Remove file, put Item struct def. into other file

* add link

* clean up docs

* refactor internal data structure to hide heap method implementations. Other cleanup after feedback

* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods

* updates after feedback

* refactoring/renaming

* guard against pushing a nil item

* minor updates after feedback

* Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC

* Initial Combined database backend implementation of static accounts and automatic rotation

* vendor updates

* initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation

* add lock and setup of rotation queue

* vendor the queue

* rebase on new method signature of queue

* remove mongo tests for now

* update default role sql

* gofmt after rebase

* cleanup after rebasing to remove checks for ErrNotFound error

* rebase cdcr-priority-queue

* vendor dependencies with 'go mod vendor'

* website database docs for Static Role support

* document the rotate-role API endpoint

* postgres specific static role docs

* use constants for paths

* updates from review

* remove dead code

* combine and clarify error message for older plugins

* Update builtin/logical/database/backend.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* cleanups from feedback

* code and comment cleanups

* move db.RLock higher to protect db.GenerateCredentials call

* Return output with WALID if we failed to delete the WAL

* Update builtin/logical/database/path_creds_create.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* updates after running 'make fmt'

* update after running 'make proto'

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* update comment and remove and rearrange some dead code

* Update website/source/api/secret/databases/index.html.md

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* cleanups after review

* Update sdk/database/dbplugin/grpc_transport.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* code cleanup after feedback

* remove PasswordLastSet; it's not used

* document GenerateCredentials and SetCredentials

* Update builtin/logical/database/path_rotate_credentials.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* wrap pop and popbykey in backend methods to protect against nil cred rotation queue

* use strings.HasPrefix instead of direct equality check for path

* Forgot to commit this

* updates after feedback

* re-purpose an outdated test to now check that static and dynamic roles cannot share a name

* check for unique name across dynamic and static roles

* refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials

* remove commented out code

* refactor to have loadstaticwals filter out wals for roles that no longer exist

* return error if nil input given

* add nil check for input into setStaticAccount

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* add constant for queue tick time in seconds, used for comparrison in updates

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* code cleanup after review

* remove misplaced code comment

* remove commented out code

* create a queue in the Factory method, even if it's never used

* update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type

* document new method

* move rotation things into a specific file

* rename test file and consolidate some static account tests

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* update code comments, method names, and move more methods into rotation.go

* update comments to be capitalized

* remove the item from the queue before we try to destroy it

* findStaticWAL returns an error

* use lowercase keys when encoding WAL entries

* small cleanups

* remove vestigial static account check

* remove redundant DeleteWAL call in populate queue

* if we error on loading role, push back to queue with 10 second backoff

* poll in initqueue to make sure the backend is setup and can write/delete data

* add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false

* add code comments on read-only loop

* code comment updates

* re-push if error returned from find static wal

* add locksutil and acquire locks when pop'ing from the queue

* grab exclusive locks for updating static roles

* Add SetCredentials and GenerateCredentials stubs to mockPlugin

* add a switch in initQueue to listen for cancelation

* remove guard on zero time, it should have no affect

* create a new context in Factory to pass on and use for closing the backend queue

* restore master copy of vendor dir
2019-06-19 14:45:39 -05:00
ncabatoff 06574da57a
Merge multiple functions for creating consul containers into one. (#6612)
Merge both functions for creating mongodb containers into one.
Add retries to docker container cleanups.
Require $VAULT_ACC be set to enable AWS tests.
2019-04-22 12:26:10 -04:00
Jeff Mitchell 213b9fd1cf Update to api 1.0.1 and sdk 0.1.8 2019-04-15 14:10:07 -04:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell 7a89a2f7e3 Fix more tests 2019-04-12 22:14:50 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
T.K 453f1ac109 changed misspelled english words (#6432) 2019-03-19 09:32:45 -04:00
Calvin Leung Huang b5c8b3d4c5 Clean up container on connection failure, switch to ory/dockertest on package postgresql (#5050) 2018-08-07 09:56:33 -07:00
Jeff Mitchell 4b354e1110
Re-add dockertest and fix up imports and update script (#4909) 2018-07-11 17:49:13 -04:00
Calvin Leung Huang 7d214d2a3a
Purge opened connections on retries during tests (#4452) 2018-04-26 11:28:58 -04:00
Becca Petrin b3b7fba67e
Release database resources on each iteration of a loop (#4305) 2018-04-17 16:31:09 -07:00
Chris Hoffman e4832fdbcf
Database Root Credential Rotation (#3976)
* redoing connection handling

* a little more cleanup

* empty implementation of rotation

* updating rotate signature

* signature update

* updating interfaces again :(

* changing back to interface

* adding templated url support and rotation for postgres

* adding correct username

* return updates

* updating statements to be a list

* adding error sanitizing middleware

* fixing log sanitizier

* adding postgres rotate test

* removing conf from rotate

* adding rotate command

* adding mysql rotate

* finishing up the endpoint in the db backend for rotate

* no more structs, just store raw config

* fixing tests

* adding db instance lock

* adding support for statement list in cassandra

* wip redoing interface to support BC

* adding falllback for Initialize implementation

* adding backwards compat for statements

* fix tests

* fix more tests

* fixing up tests, switching to new fields in statements

* fixing more tests

* adding mssql and mysql

* wrapping all the things in middleware, implementing templating for mongodb

* wrapping all db servers with error santizer

* fixing test

* store the name with the db instance

* adding rotate to cassandra

* adding compatibility translation to both server and plugin

* reordering a few things

* store the name with the db instance

* reordering

* adding a few more tests

* switch secret values from slice to map

* addressing some feedback

* reinstate execute plugin after resetting connection

* set database connection to closed

* switching secret values func to map[string]interface for potential future uses

* addressing feedback
2018-03-21 15:05:56 -04:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Brian Kassouf 9358540d50
plugins/database: use context with plugins that use database/sql package (#3691) 2017-12-15 10:26:17 -08:00
Brian Kassouf afe53eb862
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Jeff Mitchell 45d4facb29 Add some delay to postgres create user test to verify expiration isn't immediate 2017-11-30 09:35:47 -05:00
Calvin Leung Huang 3136fbb678 Fix flag parsing on database plugins, exit on parse error (#3305)
* Change FlagSet error handling to flag.ExitOnError

* Ignore os.Args[0] (the command itself) when parsing flags

* Revert to using flag.ContinueOnError
2017-09-07 16:30:00 -04:00
Brian Kassouf aca4736d7d Move database drivers from test files into the plugin files (#2986) 2017-07-07 10:38:12 -07:00
Graham Davison 0bd78602fa Moved drivers from `plugins/helper/database/connutil/sql.go` to driver test files (#2980) 2017-07-07 09:09:48 -04:00
Brian Kassouf ef979cc02b Use WeakDecode to decode the initialize values (#2871) 2017-06-14 18:59:27 -07:00
Brian Kassouf 606fe393be Use the role name in the db username (#2812) 2017-06-06 09:49:49 -04:00
Andrew Paulin 776019963b Support custom renewal statements in Postgres (#2788)
* Support custom renewal statements in Postgres

* Refactored out default/custom renew methods
2017-06-01 13:18:16 -07:00
Brian Kassouf 37bd3ed76e Use log to output errors instead of fmt 2017-05-03 13:01:05 -07:00
Brian Kassouf 29d9b831d3 Update the api for serving plugins and provide a utility to pass TLS data for commuinicating with the vault process 2017-05-02 14:40:11 -07:00
Brian Kassouf ca7ff89bcb Fix documentation 2017-05-02 02:22:06 -07:00
Brian Kassouf c8bbea9f37 Rename NewPluginServer to just Serve 2017-05-02 02:00:39 -07:00
Brian Kassouf 43cf619871 Update the username length for postgresql 2017-04-27 23:02:33 -07:00
Brian Kassouf 6f9d178370 Calls to builtin plugins now go directly to the implementation instead of go-plugin 2017-04-20 18:46:41 -07:00
Brian Kassouf 4e9f89430c Move plugins into main vault repo 2017-04-13 13:48:32 -07:00