Update the AWS auth backend acceptance tests to account for the new
`iam_tags` field that comes back on responses.
* marked only tests requiring creds as acceptance
Renamed TestBackend_* to TestAcceptanceBackend_* if the test requires
AWS credentials. Otherwise left the name as TestBackend_* and set
`AcceptanceTest: false`.
* ensure generated names aren't too long
IAM roles and users have a 64 character limit, and adding Acceptance
to the test names was putting some over the length limit, so modified
generateUniqueName() to take a max length parameter and added
functions for each type of name generation (user, role, group).
* Added support for iam_tags for AWS secret roles
This change allows iam_users generated by the secrets engine
to add custom tags in the form of key-value pairs to users
that are created.
Allows vault roles to be associated with IAM groups in the AWS
secrets engine, since IAM groups are a recommended way to manage
IAM user policies. IAM users generated against a vault role will
be added to the IAM Groups. For a credential type of
`assumed_role` or `federation_token`, the policies sent to the
corresponding AWS call (sts:AssumeRole or sts:GetFederation) will
be the policies from each group in `iam_groups` combined with the
`policy_document` and `policy_arns` parameters.
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
* secrets/aws: Support permissions boundaries on iam_user creds
This allows configuring Vault to attach a permissions boundary policy to
IAM users that it creates, configured on a per-Vault-role basis.
* Fix indentation of policy in docs
Use spaces instead of tabs
I probably left this cleanup commented out as part of debugging test
errors in #6789 and forgot to uncomment it, so actually cleaning up the
test user.
* secret/aws: Pass policy ARNs to AssumedRole and FederationToken roles
AWS now allows you to pass policy ARNs as well as, and in addition to,
policy documents for AssumeRole and GetFederationToken (see
https://aws.amazon.com/about-aws/whats-new/2019/05/session-permissions/).
Vault already collects policy ARNs for iam_user credential types; now it
will allow policy ARNs for assumed_role and federation_token credential
types and plumb them through to the appropriate AWS calls.
This brings along a minor breaking change. Vault roles of the
federation_token credential type are now required to have either a
policy_document or a policy_arns specified. This was implicit
previously; a missing policy_document would result in a validation error
from the AWS SDK when retrieving credentials. However, it would still
allow creating a role that didn't have a policy_document specified and
then later specifying it, after which retrieving the AWS credentials
would work. Similar workflows in which the Vault role didn't have a
policy_document specified for some period of time, such as deleting the
policy_document and then later adding it back, would also have worked
previously but will now be broken.
The reason for this breaking change is because a credential_type of
federation_token without either a policy_document or policy_arns
specified will return credentials that have equivalent permissions to
the credentials the Vault server itself is using. This is quite
dangerous (e.g., it could allow Vault clients access to retrieve
credentials that could modify Vault's underlying storage) and so should
be discouraged. This scenario is still possible when passing in an
appropriate policy_document or policy_arns parameter, but clients should
be explicitly aware of what they are doing and opt in to it by passing
in the appropriate role parameters.
* Error out on dangerous federation token retrieval
The AWS secrets role code now disallows creation of a dangerous role
configuration; however, pre-existing roles could have existed that would
trigger this now-dangerous code path, so also adding a check for this
configuration at credential retrieval time.
* Run makefmt
* Fix tests
* Fix comments/docs
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
* Added role-option max_sts_ttl to cap TTL for AWS STS credentials.
* Allow for setting max_sts_ttl to 0 after it has been set already.
* Fixed message in error response for default_sts_ttl > max_sts_ttl.
* Allow specifying role-default TTLs in AWS secret engine
* Add an acceptance test
* Add docs for AWS secret role-default TTLs
* Rename default_ttl to default_sts_ttl
* Return default_ttl as int64 instead of time.Duration
* Fix broken tests
The merge of #5383 broke the tests due to some changes in the test style
that didn't actually cause a git merge conflict. This updates the tests
to the new style.
* Add AWS Secret Engine Root Credential Rotation
This allows the AWS Secret Engine to rotate its credentials used to
access AWS. This will only work when the AWS Secret Engine has been
provided explicit IAM credentials via the config/root endpoint, and
further, when the IAM credentials provided are the only access key on
the IAM user associated wtih the access key (because AWS allows a
maximum of 2 access keys per user).
Fixes#4385
* Add test for AWS root credential rotation
Also fix a typo in the root credential rotation code
* Add docs for AWS root rotation
* Add locks around reading and writing config/root
And wire the backend up in a bunch of places so the config can get the
lock
* Respond to PR feedback
* Fix casing in error messages
* Fix merge errors
* Fix locking bugs
* Parallelize a couple AWS acceptance tests
Starting an effort to paralleize AWS secret engine acceptance tests.
Currently they take over a minute to run, and this parallelizes the two
that explicitly call a 10-second sleep, reulting in a 10-second speedup
in test time.
* Parameterize IAM user name
Probably not needed, but future-proofing the code
* Make remainder of tests parallel
AWS_ACCOUNT_ID environment variable is no longer being used; global
mutable state is a recipe for disaster when trying to run things in
parallel, and parallelizing the tests exposed a race condition in which
they were depending on the AWS_ACCOUNT_ID environment variable to be set
before they were run.
AWS_DEFAULT_REGION is still left as an environment variable because it
is required by AWS SDKs, but its configuration is now protected by a
sync.Once to ensure it only ever gets called a single time.
* Replace generateUnique*Name with testhelpers method
* Initial implemntation of returning 529 for rate limits
- bump aws iam and sts packages to v1.14.31 to get mocking interface
- promote the iam and sts clients to the aws backend struct, for mocking in tests
- this also promotes some functions to methods on the Backend struct, so
that we can use the injected client
Generating creds requires reading config/root for credentials to contact
IAM. Here we make pathConfigRoot a method on aws/backend so we can clear
the clients on successful update of config/root path. Adds a mutex to
safely clear the clients
* refactor locking and unlocking into methods on *backend
* refactor/simply the locking
* check client after grabbing lock
* Make AWS credential types more explicit
The AWS secret engine had a lot of confusing overloading with role
paramemters and how they mapped to each of the three credential types
supported. This now adds parameters to remove the overloading while
maintaining backwards compatibility.
With the change, it also becomes easier to add other feature requests.
Attaching multiple managed policies to IAM users and adding a policy
document to STS AssumedRole credentials is now also supported.
Fixes#4229Fixes#3751Fixes#2817
* Add missing write action to STS endpoint
* Allow unsetting policy_document with empty string
This allows unsetting the policy_document by passing in an empty string.
Previously, it would fail because the empty string isn't a valid JSON
document.
* Respond to some PR feedback
* Refactor and simplify role reading/upgrading
This gets rid of the duplicated role upgrade code between both role
reading and role writing by handling the upgrade all in the role
reading.
* Eliminate duplicated AWS secret test code
The testAccStepReadUser and testAccStepReadSTS were virtually identical,
so they are consolidated into a single method with the path passed in.
* Switch to use AWS ARN parser
* Accept temp creds in AWS secret backend acceptance tests
The AWS secret backend acceptance tests implicitly accepted long-lived
AWS credentials (i.e., AWS IAM user and/or root credentials) in two
ways:
1. It expected credentials to be passed in via the AWS_ACCESS_KEY_ID and
AWS_SECRET_ACCESS_KEY environment variables. By not accepting
AWS_SESSION_TOKEN or AWS_SECURITY_TOKEN, temporary credentials could
not be passed in. (This also forced all credentials to be passed in
via environment variables, which is a bit ugly).
2. The AWS sts:GetFederationToken call is only allowed from long-term
credentials. This is called by the Vault code which the acceptance
tests exercise.
1 is solved by deleting explicit references to credentials, which allows
the SDK to do one of the things it does best -- find credentials via the
default chain.
2 is a little more complicated. Rather than pass in whatever creds the
acceptance test was run under to the backend, the acceptance test now
creates a new IAM user and gets an access key from it, then passes the
IAM user's creds back to the backend so that it can call
sts:GetFederationToken (and then tries to clean up afterwards).
* Fix Travis build failure
The Travis build was failing because the user creation was happening
regardless of whether it was running in acceptance test mode or not.
This moves the user creation into the acceptance test precheck, which
requires lazily evaluating the credentials when configuring the backend
in the STS accetpance test, and so moving that to a PreFlight closure.
* Reduce blind sleeps in AWS secret backend acceptance tests
This removes a blind "sleep 10 seconds and then attempt to reuse the
credential" codepath and instead just keeps attemtping to reuse the
credential for 10 seconds and fails if there aren't any successful uses
after 10 seconds. This adds a few seconds speedup of acceptance test
runs from my experiments.
Support use cases where you want to provision STS tokens
using Vault, but, you need to call AWS APIs that are blocked
for federated tokens. For example, STS federated tokens cannot
invoke IAM APIs, such as Terraform scripts containing
`aws_iam_*` resources.
The secretAccessKeysRevoke revoke function now asserts that it is
not dealing with STS keys by checking a new internal data flag. Defaults
to IAM when the flag is not found.
Factored out genUsername into its own function to share between STS and
IAM secret creation functions.
Fixed bad call to "WriteOperation" instead of "UpdateOperation" in
aws/backend_test
This strips out http.DefaultClient everywhere I could immediately find
it. Too many things use it and then modify it in incompatible ways.
Fixes#700, I believe.
Theron Voran