Adds debug and warn logging around AWS credential chain generation,
specifically to help users debugging auto-unseal problems on AWS, by
logging which role is being used in the case of a webidentity token.
Adds a deferred call to flush the log output as well, to ensure logs
are output in the event of an initialization failure.
* upgrade aws roles
* test upgrade aws roles
* Initialize aws credential backend at mount time
* add a TODO
* create end-to-end test for builtin/credential/aws
* fix bug in initializer
* improve comments
* add Initialize() to logical.Backend
* use Initialize() in Core.enableCredentialInternal()
* use InitializeRequest to call Initialize()
* improve unit testing for framework.Backend
* call logical.Backend.Initialize() from all of the places that it needs to be called.
* implement backend.proto changes for logical.Backend.Initialize()
* persist current role storage version when upgrading aws roles
* format comments correctly
* improve comments
* use postUnseal funcs to initialize backends
* simplify test suite
* improve test suite
* simplify logic in aws role upgrade
* simplify aws credential initialization logic
* simplify logic in aws role upgrade
* use the core's activeContext for initialization
* refactor builtin/plugin/Backend
* use a goroutine to upgrade the aws roles
* misc improvements and cleanup
* do not run AWS role upgrade on DR Secondary
* always call logical.Backend.Initialize() when loading a plugin.
* improve comments
* on standbys and DR secondaries we do not want to run any kind of upgrade logic
* fix awsVersion struct
* clarify aws version upgrade
* make the upgrade logic for aws auth more explicit
* aws upgrade is now called from a switch
* fix fallthrough bug
* simplify logic
* simplify logic
* rename things
* introduce currentAwsVersion const to track aws version
* improve comments
* rearrange things once more
* conglomerate things into one function
* stub out aws auth initialize e2e test
* improve aws auth initialize e2e test
* finish aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* tinker with aws auth initialize e2e test
* fix typo in test suite
* simplify logic a tad
* rearrange assignment
* Fix a few lifecycle related issues in #7025 (#7075)
* Fix panic when plugin fails to load
* Use a role cache to avoid separate locking paths
Due to the various locked/nonlocked paths we had a case where we weren't
always checking for secondary status before trying to upgrade. This
broadly simplifies things by using a cache to store the current role
values (avoiding a lot of storage hits) and updating the cache on any
write, delete, or invalidation.
* Update aws auth docs with new semantics
Moving away from implicitly globbed bound_iam_role_arn and
bound_iam_instance_profile_arn variables to make them explicit
* Refactor tests to reduce duplication
auth/aws EC2 login tests had the same flow duplicated a few times, so
refactoring to reduce duplication
* Add tests for aws auth explicit wildcard constraints
* Remove implicit prefix matching from AWS auth backend
In the aws auth backend, bound_iam_role_arn and
bound_iam_instance_profile_arn were ALWAYS prefix matched, and there was
no way to opt out of this implicit prefix matching. This now makes the
implicit prefix matching an explicit opt-in feature by requiring users
to specify a * at the end of an ARN if they want the prefix matching.
* auth/aws: Allow binding by EC2 instance IDs
This allows specifying a list of EC2 instance IDs that are allowed to
bind to the role. To keep style formatting with the other bindings, this
is still called bound_ec2_instance_id rather than bound_ec2_instance_ids
as I intend to convert the other bindings to accept lists as well (where
it makes sense) and keeping them with singular names would be the
easiest for backwards compatibility.
Partially fixes#3797
* auth/aws: Allow lists in binds
In the aws auth method, allow a number of binds to take in lists
instead of a single string value. The intended semantic is that, for
each bind type set, clients must match at least one of each of the bind
types set in order to authenticate.
* auth/aws: Fix error with empty bound_iam_principal_arn
In cases where there doesn't need to be a bound_iam_principal_arn, i.e.,
either auth_type is ec2 or there are other bindings with the iam
auth_type, but it is specified explicitly anyway, Vault tried to parse
it to resolve to internal unique IDs. This now checks to ensure that
bound_iam_principal_arn is non-empty before attempting to resolve it.
Fixes#3837
* Fix extraneous newline
* Start work on passing context to backends
* More work on passing context
* Unindent logical system
* Unindent token store
* Unindent passthrough
* Unindent cubbyhole
* Fix tests
* use requestContext in rollback and expiration managers
* Add backend plugin changes
* Fix totp backend plugin tests
* Fix logical/plugin InvalidateKey test
* Fix plugin catalog CRUD test, fix NoopBackend
* Clean up commented code block
* Fix system backend mount test
* Set plugin_name to omitempty, fix handleMountTable config parsing
* Clean up comments, keep shim connections alive until cleanup
* Include pluginClient, disallow LookupPlugin call from within a plugin
* Add wrapper around backendPluginClient for proper cleanup
* Add logger shim tests
* Add logger, storage, and system shim tests
* Use pointer receivers for system view shim
* Use plugin name if no path is provided on mount
* Enable plugins for auth backends
* Add backend type attribute, move builtin/plugin/package
* Fix merge conflict
* Fix missing plugin name in mount config
* Add integration tests on enabling auth backend plugins
* Remove dependency cycle on mock-plugin
* Add passthrough backend plugin, use logical.BackendType to determine lease generation
* Remove vault package dependency on passthrough package
* Add basic impl test for passthrough plugin
* Incorporate feedback; set b.backend after shims creation on backendPluginServer
* Fix totp plugin test
* Add plugin backends docs
* Fix tests
* Fix builtin/plugin tests
* Remove flatten from PluginRunner fields
* Move mock plugin to logical/plugin, remove totp and passthrough plugins
* Move pluginMap into newPluginClient
* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck
* Change shim logger's Fatal to no-op
* Change BackendType to uint32, match UX backend types
* Change framework.Backend Setup signature
* Add Setup func to logical.Backend interface
* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments
* Remove commented var in plugin package
* RegisterLicense on logical.Backend interface (#3017)
* Add RegisterLicense to logical.Backend interface
* Update RegisterLicense to use callback func on framework.Backend
* Refactor framework.Backend.RegisterLicense
* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs
* plugin: Revert BackendType to remove TypePassthrough and related references
* Fix typo in plugin backends docs
* Rename builtin/credential/aws-ec2 to aws
The aws-ec2 authentication backend is being expanded and will become the
generic aws backend. This is a small rename commit to keep the commit
history clean.
* Expand aws-ec2 backend to more generic aws
This adds the ability to authenticate arbitrary AWS IAM principals using
AWS's sts:GetCallerIdentity method. The AWS-EC2 auth backend is being to
just AWS with the expansion.
* Add missing aws auth handler to CLI
This was omitted from the previous commit
* aws auth backend general variable name cleanup
Also fixed a bug where allowed auth types weren't being checked upon
login, and added tests for it.
* Update docs for the aws auth backend
* Refactor aws bind validation
* Fix env var override in aws backend test
Intent is to override the AWS environment variables with the TEST_*
versions if they are set, but the reverse was happening.
* Update docs on use of IAM authentication profile
AWS now allows you to change the instance profile of a running instance,
so the use case of "a long-lived instance that's not in an instance
profile" no longer means you have to use the the EC2 auth method. You
can now just change the instance profile on the fly.
* Fix typo in aws auth cli help
* Respond to PR feedback
* More PR feedback
* Respond to additional PR feedback
* Address more feedback on aws auth PR
* Make aws auth_type immutable per role
* Address more aws auth PR feedback
* Address more iam auth PR feedback
* Rename aws-ec2.html.md to aws.html.md
Per PR feedback, to go along with new backend name.
* Add MountType to logical.Request
* Make default aws auth_type dependent upon MountType
When MountType is aws-ec2, default to ec2 auth_type for backwards
compatibility with legacy roles. Otherwise, default to iam.
* Pass MountPoint and MountType back up to the core
Previously the request router reset the MountPoint and MountType back to
the empty string before returning to the core. This ensures they get set
back to the correct values.
