1) In backends, ensure they are now using TokenPolicies
2) Don't reassign auth.Policies until after expmgr registration as we
don't need them at that point
Fixes#4829
This change makes it so that if a lease is revoked through user action,
we set the expiration time to now and update pending, just as we do with
tokens. This allows the normal retry logic to apply in these cases as
well, instead of just erroring out immediately. The idea being that once
you tell Vault to revoke something it should keep doing its darndest to
actually make that happen.
* Add an idle timeout for the server
Because tidy operations can be long-running, this also changes all tidy
operations to behave the same operationally (kick off the process, get a
warning back, log errors to server log) and makes them all run in a
goroutine.
This could mean a sort of hard stop if Vault gets sealed because the
function won't have the read lock. This should generally be okay
(running tidy again should pick back up where it left off), but future
work could use cleanup funcs to trigger the functions to stop.
* Fix up tidy test
* Add deadline to cluster connections and an idle timeout to the cluster server, plus add readheader/read timeout to api server
* This changes the way policies are reported in audit logs.
Previously, only policies tied to tokens would be reported. This could
make it difficult to perform after-the-fact analysis based on both the
initial response entry and further requests. Now, the full set of
applicable policies from both the token and any derived policies from
Identity are reported.
To keep things consistent, token authentications now also return the
full set of policies in api.Secret.Auth responses, so this both makes it
easier for users to understand their actual full set, and it matches
what the audit logs now report.
* Fix panic due to metadata being nil
* added a nil check
* Added a test
* ensure metadata is never nil
* Remove unnecessary allocation
* revert back to early initialization
Taking inspiration from
https://github.com/golang/go/issues/17604#issuecomment-256384471
suggests that taking the address of a stack variable for use in atomics
works (at least, the race detector doesn't complain) but is doing it
wrong.
The only other change is a change in Leader() detecting if HA is enabled
to fast-path out. This value never changes after NewCore, so we don't
need to grab the read lock to check it.
* make invalid role_id a 400 error
* remove single-use validateCredentials function
* remove single-use validateBindSecretID function
* adjust the error message for CIDR check failure
* locking updates as review feedback
* govet cleanup in token store
* adding general ttl handling to login requests
* consolidating TTL calculation to system view
* deprecate LeaseExtend
* deprecate LeaseExtend
* set the increment to the correct value
* move calculateTTL out of SystemView
* remove unused value
* add back clearing of lease id
* implement core ttl in some backends
* removing increment and issue time from lease options
* adding ttl tests, fixing some compile issue
* adding ttl tests
* fixing some explicit max TTL logic
* fixing up some tests
* removing unneeded test
* off by one errors...
* adding back some logic for bc
* adding period to return on renewal
* tweaking max ttl capping slightly
* use the appropriate precision for ttl calculation
* deprecate proto fields instead of delete
* addressing feedback
* moving TTL handling for backends to core
* mongo is a secret backend not auth
* adding estimated ttl for backends that also manage the expiration time
* set the estimate values before calling the renew request
* moving calculate TTL to framework, revert removal of increment and issue time from logical
* minor edits
* addressing feedback
* address more feedback
* logbridge with hclog and identical output
* Initial search & replace
This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.
* strip logbridge
* fix majority of tests
* update logxi aliases
* WIP fixing tests
* more test fixes
* Update test to hclog
* Fix format
* Rename hclog -> log
* WIP making hclog and logxi love each other
* update logger_test.go
* clean up merged comments
* Replace RawLogger interface with a Logger
* Add some logger names
* Replace Trace with Debug
* update builtin logical logging patterns
* Fix build errors
* More log updates
* update log approach in command and builtin
* More log updates
* update helper, http, and logical directories
* Update loggers
* Log updates
* Update logging
* Update logging
* Update logging
* Update logging
* update logging in physical
* prefixing and lowercase
* Update logging
* Move phyisical logging name to server command
* Fix som tests
* address jims feedback so far
* incorporate brians feedback so far
* strip comments
* move vault.go to logging package
* update Debug to Trace
* Update go-plugin deps
* Update logging based on review comments
* Updates from review
* Unvendor logxi
* Remove null_logger.go
* Update aws auth docs with new semantics
Moving away from implicitly globbed bound_iam_role_arn and
bound_iam_instance_profile_arn variables to make them explicit
* Refactor tests to reduce duplication
auth/aws EC2 login tests had the same flow duplicated a few times, so
refactoring to reduce duplication
* Add tests for aws auth explicit wildcard constraints
* Remove implicit prefix matching from AWS auth backend
In the aws auth backend, bound_iam_role_arn and
bound_iam_instance_profile_arn were ALWAYS prefix matched, and there was
no way to opt out of this implicit prefix matching. This now makes the
implicit prefix matching an explicit opt-in feature by requiring users
to specify a * at the end of an ARN if they want the prefix matching.
* auth/aws: Allow binding by EC2 instance IDs
This allows specifying a list of EC2 instance IDs that are allowed to
bind to the role. To keep style formatting with the other bindings, this
is still called bound_ec2_instance_id rather than bound_ec2_instance_ids
as I intend to convert the other bindings to accept lists as well (where
it makes sense) and keeping them with singular names would be the
easiest for backwards compatibility.
Partially fixes#3797
* auth/aws: Allow lists in binds
In the aws auth method, allow a number of binds to take in lists
instead of a single string value. The intended semantic is that, for
each bind type set, clients must match at least one of each of the bind
types set in order to authenticate.
Previously the functional test was only testing the PCKS7-signed identity
document, not the detached RSA signature, so adding a test for that in the
functional test suite.
* Verify DNS SANs if PermittedDNSDomains is set
* Use DNSNames check and not PermittedDNSDomains on leaf certificate
* Document the check
* Add RFC link
* Test for success case
* fix the parameter name
* rename the test
* remove unneeded commented code
strings.HasPrefix is more correct; if a tag part value ended up
containing the expected prefix of another part, it could cause incorrect
parsing. I don't think that these values would be semantically legal
today, but it's probably better to be defensive.
* Differentiate between user/internal error in AppRole login.
This allows us to properly pass through internal errors back up into
core.
* Separate out error cases
* auth/aws: Fix error with empty bound_iam_principal_arn
In cases where there doesn't need to be a bound_iam_principal_arn, i.e.,
either auth_type is ec2 or there are other bindings with the iam
auth_type, but it is specified explicitly anyway, Vault tried to parse
it to resolve to internal unique IDs. This now checks to ensure that
bound_iam_principal_arn is non-empty before attempting to resolve it.
Fixes#3837
* Fix extraneous newline
* Don't call LeaseExtend on login renewal paths when period is provided
* WIP tests
* NoopBackend accept backend ttl values
* Test period value on credentials backend
* Use t.Fatalf instead
* Remove mockCoreExpiration
* Add login renewal test for approle backend
* Add resp.Auth.Period check on aws and cert backend tests
* Pass in approle's period via role's period
* Correctly set period in valid-role's role
* Add period renewal test using TestCluster and approle backend
* Check for ttl values after renewals on test
* Use SHA2-256 hash with prefix to upgrade the paths
* test the SHA1 upgrade to SHA256
* Remove hash identifier and the delimiter; use 's' instead
* Added API test to verify the correctness of the fix
* Fix broken test
* remove unneeded test
* Support JSON lists for Okta user groups+policies.
Migrate the manually-parsed comma-separated string field types for user
groups and user policies to TypeCommaStringSlice. This means user
endpoints now accept proper lists as input for these fields in addition
to comma-separated string values. The value for reads remains a list.
Update the Okta API documentation for users and groups to reflect that
both user group and user/group policy fields are list-valued.
Update the Okta acceptance tests to cover passing a list value for the
user policy field, and require the OKTA_API_TOKEN env var to be set
(required for the "everyone" policy tests to pass).
* Fix typo, add comma-separated docs.
* Start work on passing context to backends
* More work on passing context
* Unindent logical system
* Unindent token store
* Unindent passthrough
* Unindent cubbyhole
* Fix tests
* use requestContext in rollback and expiration managers
* Add logic for using Auth.Period when handling auth login/renew requests
* Set auth.TTL if not set in handleLoginRequest
* Always set auth.TTL = te.TTL on handleLoginRequest, check TTL and period against sys values on RenewToken
* Get sysView from le.Path, revert tests
* Add back auth.Policies
* Fix TokenStore tests, add resp warning when capping values
* Use switch for ttl/period check on RenewToken
* Move comments around
* avoid race conditions in approle
* return a warning from role read if secondary index is missing
* Create a role ID index if a role is missing one
* Fix locking in approle read and add test
* address review feedback
* external identity groups
* add local LDAP groups as well to group aliases
* add group aliases for okta credential backend
* Fix panic in tests
* fix build failure
* remove duplicated struct tag
* add test steps to test out removal of group member during renewals
* Add comment for having a prefix check in router
* fix tests
* s/parent_id/canonical_id
* s/parent/canonical in comments and errors
This removes all references I could find to:
- credential provider
- authentication backend
- authentication provider
- auth provider
- auth backend
in favor of the unified:
- auth method
This was previously part of the very long command/auth.go file, where it
mimmicked the same API as other handlers. By making it a builtin
credential, we can remove a lot of conditional logic for token-based
authentication.
Some of the path help documentation was incorrect for auth/aws/role as
behavior changed during PR development and the help wasn't updated. This
fixes incorrect information and makes the path help somewhat more
consistent.
Fixes#3472
* Adding support for base_url for Okta api
* addressing feedback suggestions, bringing back optional group query
* updating docs
* cleaning up the login method
* clear out production flag if base_url is set
* docs updates
* docs updates
The parseIamArn method was making assumptions about the input arn being
properly formatted and of a certain type. If users tried to pass a
bound_iam_principal_arn that was malformed (or was the ARN of the root
user), it would cause a panic. parseIamArn now explicitly checks the
assumptions it's making and tests are added to ensure it properly errors
out (rather than panic'ing) on malformed input.
* Compare groups case-insensitively at login time, since Okta groups are
case-insensitive but preserving.
* Make other group operations case-preserving but otherwise
case-insensitive. New groups will be written in lowercase.
* Add backend plugin changes
* Fix totp backend plugin tests
* Fix logical/plugin InvalidateKey test
* Fix plugin catalog CRUD test, fix NoopBackend
* Clean up commented code block
* Fix system backend mount test
* Set plugin_name to omitempty, fix handleMountTable config parsing
* Clean up comments, keep shim connections alive until cleanup
* Include pluginClient, disallow LookupPlugin call from within a plugin
* Add wrapper around backendPluginClient for proper cleanup
* Add logger shim tests
* Add logger, storage, and system shim tests
* Use pointer receivers for system view shim
* Use plugin name if no path is provided on mount
* Enable plugins for auth backends
* Add backend type attribute, move builtin/plugin/package
* Fix merge conflict
* Fix missing plugin name in mount config
* Add integration tests on enabling auth backend plugins
* Remove dependency cycle on mock-plugin
* Add passthrough backend plugin, use logical.BackendType to determine lease generation
* Remove vault package dependency on passthrough package
* Add basic impl test for passthrough plugin
* Incorporate feedback; set b.backend after shims creation on backendPluginServer
* Fix totp plugin test
* Add plugin backends docs
* Fix tests
* Fix builtin/plugin tests
* Remove flatten from PluginRunner fields
* Move mock plugin to logical/plugin, remove totp and passthrough plugins
* Move pluginMap into newPluginClient
* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck
* Change shim logger's Fatal to no-op
* Change BackendType to uint32, match UX backend types
* Change framework.Backend Setup signature
* Add Setup func to logical.Backend interface
* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments
* Remove commented var in plugin package
* RegisterLicense on logical.Backend interface (#3017)
* Add RegisterLicense to logical.Backend interface
* Update RegisterLicense to use callback func on framework.Backend
* Refactor framework.Backend.RegisterLicense
* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs
* plugin: Revert BackendType to remove TypePassthrough and related references
* Fix typo in plugin backends docs
* Let AWS auth CLI helper only generate login data
This will be useful to other golang clients so they can manage the login
process themselves.
Also helps for #2855
* Respond to PR feedback
In auth/aws/config/client, when only the iam_server_id_header_value was
being updated on an existing config, it wouldn't get stored because I
was trying to avoid unnecessarily flushing the cache of AWS clients, and
the flag to not flush the cache also meant that the updated entry didn't
get written back to the storage. This now adds a new flag for when
other changes occur that don't require flushing the cache but do require
getting written to the storage. It also adds a test for this explicitly.
Fixes#3004
Previously, the renew method would ALWAYS check to ensure the
authenticated IAM principal ARN matched the bound ARN. However, there
is a valid use case in which no bound_iam_principal_arn is specified and
all bindings are done through inferencing. When a role is configured
like this, clients won't be able to renew their token because of the
check.
This now checks to ensure that the bound_iam_principal_arn is not empty
before requriing that it match the originally authenticated client.
Fixes#2781
A change in copystructure has caused some panics due to the custom copy
function. I'm more nervous about production panics than I am about
keeping some bad code wiping out some existing warnings, so remove the
custom copy function and just allow direct setting of Warnings.
* Refactor to consolidate constraints on the matching chain
* Add CN prefix/suffix constraint
* Maintain backwards compatibility (pick a random cert if multiple match)
* Vendor go-glob
* Replace cn_prefix/suffix with required_name/globbing
Move all the new tests to acceptance-capable tests instead of embedding in the CRL test
* Allow authenticating against a single cert
* Add new params to documentation
* Add CLI support for new param
* Refactor for style
* Support multiple (ORed) name patterns
* Rename required_names to allowed_names
* Update docs for parameter rename
* Use the new TypeCommaStringSlice
* Rename builtin/credential/aws-ec2 to aws
The aws-ec2 authentication backend is being expanded and will become the
generic aws backend. This is a small rename commit to keep the commit
history clean.
* Expand aws-ec2 backend to more generic aws
This adds the ability to authenticate arbitrary AWS IAM principals using
AWS's sts:GetCallerIdentity method. The AWS-EC2 auth backend is being to
just AWS with the expansion.
* Add missing aws auth handler to CLI
This was omitted from the previous commit
* aws auth backend general variable name cleanup
Also fixed a bug where allowed auth types weren't being checked upon
login, and added tests for it.
* Update docs for the aws auth backend
* Refactor aws bind validation
* Fix env var override in aws backend test
Intent is to override the AWS environment variables with the TEST_*
versions if they are set, but the reverse was happening.
* Update docs on use of IAM authentication profile
AWS now allows you to change the instance profile of a running instance,
so the use case of "a long-lived instance that's not in an instance
profile" no longer means you have to use the the EC2 auth method. You
can now just change the instance profile on the fly.
* Fix typo in aws auth cli help
* Respond to PR feedback
* More PR feedback
* Respond to additional PR feedback
* Address more feedback on aws auth PR
* Make aws auth_type immutable per role
* Address more aws auth PR feedback
* Address more iam auth PR feedback
* Rename aws-ec2.html.md to aws.html.md
Per PR feedback, to go along with new backend name.
* Add MountType to logical.Request
* Make default aws auth_type dependent upon MountType
When MountType is aws-ec2, default to ec2 auth_type for backwards
compatibility with legacy roles. Otherwise, default to iam.
* Pass MountPoint and MountType back up to the core
Previously the request router reset the MountPoint and MountType back to
the empty string before returning to the core. This ensures they get set
back to the correct values.
* approle: secret-id listing lock sanity
* Skip processing an empty secretIDHMAC item during the iteration
* approle: use dedicated lock for listing of secret-id-accessors
In order for Vault to map IAM instance profiles to roles, Vault
must query the 'iam:GetInstanceProfile' API, so update the documentation
and help to include the additional permissions needed.
This fixes#1911 but not directly; it doesn't address the cause of the
panic. However, it turns out that this is the correct fix anyways,
because it ensures that the value being logged is RFC3339 format, which
is what the time turns into in JSON but not the normal time string
value, so what we audit log (and HMAC) matches what we are returning.
Use the stored information to validate the source address and credential issue time.
Correct the logic used to verify BoundCIDRList on the role.
Reverify the subset requirements between secret ID and role during credential issue time.
