I probably left this cleanup commented out as part of debugging test
errors in #6789 and forgot to uncomment it, so actually cleaning up the
test user.
* secret/aws: Pass policy ARNs to AssumedRole and FederationToken roles
AWS now allows you to pass policy ARNs as well as, and in addition to,
policy documents for AssumeRole and GetFederationToken (see
https://aws.amazon.com/about-aws/whats-new/2019/05/session-permissions/).
Vault already collects policy ARNs for iam_user credential types; now it
will allow policy ARNs for assumed_role and federation_token credential
types and plumb them through to the appropriate AWS calls.
This brings along a minor breaking change. Vault roles of the
federation_token credential type are now required to have either a
policy_document or a policy_arns specified. This was implicit
previously; a missing policy_document would result in a validation error
from the AWS SDK when retrieving credentials. However, it would still
allow creating a role that didn't have a policy_document specified and
then later specifying it, after which retrieving the AWS credentials
would work. Similar workflows in which the Vault role didn't have a
policy_document specified for some period of time, such as deleting the
policy_document and then later adding it back, would also have worked
previously but will now be broken.
The reason for this breaking change is because a credential_type of
federation_token without either a policy_document or policy_arns
specified will return credentials that have equivalent permissions to
the credentials the Vault server itself is using. This is quite
dangerous (e.g., it could allow Vault clients access to retrieve
credentials that could modify Vault's underlying storage) and so should
be discouraged. This scenario is still possible when passing in an
appropriate policy_document or policy_arns parameter, but clients should
be explicitly aware of what they are doing and opt in to it by passing
in the appropriate role parameters.
* Error out on dangerous federation token retrieval
The AWS secrets role code now disallows creation of a dangerous role
configuration; however, pre-existing roles could have existed that would
trigger this now-dangerous code path, so also adding a check for this
configuration at credential retrieval time.
* Run makefmt
* Fix tests
* Fix comments/docs
This refactors role validation for the AWS secrets engine to be in a
separate method. Previously, all validation was interspersed with the
parsing of parameters when creating/updating a role, which led to a high
degree of complexity. Now, all validation is centralized which makes it
easier to understand and also easier to test (and so a number of test
cases have been added).
* Add maximum amount of random characters requested at any given time
* Readability changes
* Removing sys/tools/random from the default policy
* Setting the maxBytes value as const
* Declaring maxBytes in the package to use it everywhere
* Using maxBytes in the error message
* upgrade aws roles
* test upgrade aws roles
* Initialize aws credential backend at mount time
* add a TODO
* create end-to-end test for builtin/credential/aws
* fix bug in initializer
* improve comments
* add Initialize() to logical.Backend
* use Initialize() in Core.enableCredentialInternal()
* use InitializeRequest to call Initialize()
* improve unit testing for framework.Backend
* call logical.Backend.Initialize() from all of the places that it needs to be called.
* implement backend.proto changes for logical.Backend.Initialize()
* persist current role storage version when upgrading aws roles
* format comments correctly
* improve comments
* use postUnseal funcs to initialize backends
* simplify test suite
* improve test suite
* simplify logic in aws role upgrade
* simplify aws credential initialization logic
* simplify logic in aws role upgrade
* use the core's activeContext for initialization
* refactor builtin/plugin/Backend
* use a goroutine to upgrade the aws roles
* misc improvements and cleanup
* do not run AWS role upgrade on DR Secondary
* always call logical.Backend.Initialize() when loading a plugin.
* improve comments
* on standbys and DR secondaries we do not want to run any kind of upgrade logic
* fix awsVersion struct
* clarify aws version upgrade
* make the upgrade logic for aws auth more explicit
* aws upgrade is now called from a switch
* fix fallthrough bug
* simplify logic
* simplify logic
* rename things
* introduce currentAwsVersion const to track aws version
* improve comments
* rearrange things once more
* conglomerate things into one function
* stub out aws auth initialize e2e test
* improve aws auth initialize e2e test
* finish aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* fix typo in test suite
* simplify logic a tad
* rearrange assignment
* Fix a few lifecycle related issues in #7025 (#7075)
* Fix panic when plugin fails to load
* Implement SetCredentials for MongoDB, adding support for static accounts
* rework SetCredentials to split from CreateUser, and to parse the url for database
* Add integration test for mongodb static account rotation
* check the length of the password results to avoid out-of-bounds
* remove unused method
* use the pre-existing test helper for this. Add parse method to helper
* remove unused command
This was inspired by #7022 but has the advantage of avoiding
double-locking and needing to perform lock upgrades while also
simplifying the logic and being faster.
Original, #7022, this:
goos: linux
goarch: amd64
pkg: github.com/hashicorp/vault/builtin/audit/file
BenchmarkAuditFile_request-4 30000 60734 ns/op
PASS
ok github.com/hashicorp/vault/builtin/audit/file 2.428s
goos: linux
goarch: amd64
pkg: github.com/hashicorp/vault/builtin/audit/file
BenchmarkAuditFile_request-4 50000 34772 ns/op
PASS
ok github.com/hashicorp/vault/builtin/audit/file 2.086s
goos: linux
goarch: amd64
pkg: github.com/hashicorp/vault/builtin/audit/file
BenchmarkAuditFile_request-4 50000 25302 ns/op
PASS
ok github.com/hashicorp/vault/builtin/audit/file 1.542s
Fixes#7014Closes#7022
If only a non-_token field is provided we don't want to clear out the
Token version of the params, we want to set both. Otherwise we can't
rely on using the Token version of the parameter when creating the Auth
struct.
* remove create/update database user for static accounts
* update tests after create/delete removed
* small cleanups
* update postgresql setcredentials test
* add kmip engine
* adjust where kmip engine is mounted and sketch out routes
* add secret mount path service to share params to engines
* move list-controller and list-route mixins to core addon and adjust imports
* properly link kmip secrets from the secrets list page
* tweak routes and add list controllers
* stub out some models and adapters
* fix mixin exports
* move a bunch of components into the core addon
* use new empty yield in list-view in the namespace template
* scopes list using list-view and list-item components
* simplify and flatten routes, templates for all of the list pages
* role show route and template and scope create template
* add ember-router-helpers
* add more packages to the dependencies of the core addon
* add field-group-show component for listing fields from a model
* move more components to the shared addon
* make configure and configuration routes work and save a generated model
* save and list scopes
* role create, list, read
* list credentials properly
* move allowed attributes to field group
* show allowed operations on role details page
* add kmip logo to mount secrets engine list page
* add role edit page
* show all model attributes on role show page
* enable role edit
* fix newFields error by creating open api role model on the role list route
* only show selected fields on role edit page
* do not send scope and backend attrs to api
* move path-or-array to core addon
* move string-list component to core addon
* remove extra top border when there is only one field group
* add icons for all of the list pages
* update kmip config model so defaultValue doesn't error
* generate credentials
* credential create and show
* only show kmip when feature is enabled
* fix saving of TTL fields generated from Open API
* move masked-input and list-pagination components to core addon
* add param on edit form to allow for calling onSave after render happens
* polish credential show page and redirect there after generating credentials
* add externalLink for kmip engine
* add kmip-breadcrumb component
* use kmip-breadcrumb component
* add linkPrefix param to linked-block component to allow for routing programmatically inside an engine
* redirect to the right place when enabling kmip
* fix linting
* review feedback
* update signature for path-help usage
* fix ttl field expansion test
* remove role filed from role form, fix generate redirect
* remove field-group-show because it's in the core addon
* remove bottom rule from show pages
* fix Max TTL displayAttrs for ssh role
* update edit-form to take fields or attrs
* fix linting
* remove listenAddrs and set default val on ttl if a val is passed in
* Use a role cache to avoid separate locking paths
Due to the various locked/nonlocked paths we had a case where we weren't
always checking for secondary status before trying to upgrade. This
broadly simplifies things by using a cache to store the current role
values (avoiding a lot of storage hits) and updating the cache on any
write, delete, or invalidation.
* Add priority queue to sdk
* fix issue of storing pointers and now copy
* update to use copy structure
* Remove file, put Item struct def. into other file
* add link
* clean up docs
* refactor internal data structure to hide heap method implementations. Other cleanup after feedback
* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods
* updates after feedback
* refactoring/renaming
* guard against pushing a nil item
* minor updates after feedback
* Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC
* Initial Combined database backend implementation of static accounts and automatic rotation
* vendor updates
* initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation
* add lock and setup of rotation queue
* vendor the queue
* rebase on new method signature of queue
* remove mongo tests for now
* update default role sql
* gofmt after rebase
* cleanup after rebasing to remove checks for ErrNotFound error
* rebase cdcr-priority-queue
* vendor dependencies with 'go mod vendor'
* website database docs for Static Role support
* document the rotate-role API endpoint
* postgres specific static role docs
* use constants for paths
* updates from review
* remove dead code
* combine and clarify error message for older plugins
* Update builtin/logical/database/backend.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* cleanups from feedback
* code and comment cleanups
* move db.RLock higher to protect db.GenerateCredentials call
* Return output with WALID if we failed to delete the WAL
* Update builtin/logical/database/path_creds_create.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* updates after running 'make fmt'
* update after running 'make proto'
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* update comment and remove and rearrange some dead code
* Update website/source/api/secret/databases/index.html.md
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* cleanups after review
* Update sdk/database/dbplugin/grpc_transport.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* code cleanup after feedback
* remove PasswordLastSet; it's not used
* document GenerateCredentials and SetCredentials
* Update builtin/logical/database/path_rotate_credentials.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* wrap pop and popbykey in backend methods to protect against nil cred rotation queue
* use strings.HasPrefix instead of direct equality check for path
* Forgot to commit this
* updates after feedback
* re-purpose an outdated test to now check that static and dynamic roles cannot share a name
* check for unique name across dynamic and static roles
* refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials
* remove commented out code
* refactor to have loadstaticwals filter out wals for roles that no longer exist
* return error if nil input given
* add nil check for input into setStaticAccount
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* add constant for queue tick time in seconds, used for comparrison in updates
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Jim Kalafut <jim@kalafut.net>
* code cleanup after review
* remove misplaced code comment
* remove commented out code
* create a queue in the Factory method, even if it's never used
* update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type
* document new method
* move rotation things into a specific file
* rename test file and consolidate some static account tests
* Update builtin/logical/database/path_roles.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update builtin/logical/database/rotation.go
Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
* update code comments, method names, and move more methods into rotation.go
* update comments to be capitalized
* remove the item from the queue before we try to destroy it
* findStaticWAL returns an error
* use lowercase keys when encoding WAL entries
* small cleanups
* remove vestigial static account check
* remove redundant DeleteWAL call in populate queue
* if we error on loading role, push back to queue with 10 second backoff
* poll in initqueue to make sure the backend is setup and can write/delete data
* add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false
* add code comments on read-only loop
* code comment updates
* re-push if error returned from find static wal
* add locksutil and acquire locks when pop'ing from the queue
* grab exclusive locks for updating static roles
* Add SetCredentials and GenerateCredentials stubs to mockPlugin
* add a switch in initQueue to listen for cancelation
* remove guard on zero time, it should have no affect
* create a new context in Factory to pass on and use for closing the backend queue
* restore master copy of vendor dir
