* Fix using wrong public key in sign-self-issued
* Change behavior of TTL in sign-intermediate
This allows signing CA certs with an expiration past the signer's
NotAfter.
It also change sign-self-issued to replace the Issuer, since it's
potentially RFC legal but stacks won't validate it.
Ref: https://groups.google.com/d/msg/vault-tool/giP69-n2o20/FfhRpW1vAQAJ
The parseIamArn method was making assumptions about the input arn being
properly formatted and of a certain type. If users tried to pass a
bound_iam_principal_arn that was malformed (or was the ARN of the root
user), it would cause a panic. parseIamArn now explicitly checks the
assumptions it's making and tests are added to ensure it properly errors
out (rather than panic'ing) on malformed input.
* Lazy load plugins to avoid setup-unwrap cycle
* Remove commented blocks
* Refactor NewTestCluster, use single core cluster on basic plugin tests
* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly
* Add special path to mock plugin
* Move ensureCoresSealed to vault/testing.go
* Use same method for EnsureCoresSealed and Cleanup
* Bump ensureCoresSealed timeout to 60s
* Correctly handle nil opts on NewTestCluster
* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap
* Check metadata flag directly on the plugin process
* Plumb isMetadataMode down to PluginRunner
* Add NOOP shims when running in metadata mode
* Remove unused flag from the APIMetadata object
* Remove setupSecretPlugins and setupCredentialPlugins functions
* Move when we setup rollback manager to after the plugins are initialized
* Fix tests
* Fix merge issue
* start rollback manager after the credential setup
* Add guards against running certain client and server functions while in metadata mode
* Call initialize once a plugin is loaded on the fly
* Add more tests, update basic secret/auth plugin tests to trigger lazy loading
* Skip mount if plugin removed from catalog
* Fixup
* Remove commented line on LookupPlugin
* Fail on mount operation if plugin is re-added to catalog and mount is on existing path
* Check type and special paths on startBackend
* Fix merge conflicts
* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
* Add pki/root/sign-self-issued.
This is useful for root CA rolling, and is also suitably dangerous.
Along the way I noticed we weren't setting the authority key IDs
anywhere, so I addressed that.
* Add tests
* Compare groups case-insensitively at login time, since Okta groups are
case-insensitive but preserving.
* Make other group operations case-preserving but otherwise
case-insensitive. New groups will be written in lowercase.
* Add automatic plugin reload
* Refactor builtin/backend
* Remove plugin reload at the core level
* Refactor plugin tests
* Add auto-reload test case
* Change backend to use sync.RWMutex, fix dangling test plugin processes
* Add a canary to plugin backends to avoid reloading many times (#3174)
* Call setupPluginCatalog before mount-related operations in postUnseal
* Don't create multiple system backends since core only holds a reference (#3176)
to one.
gRPC doesn't have a handler for recovering from a panic like a normal
HTTP request so a panic will actually kill Vault's listener. This
basically copies the net/http logic for managing this.
The SSH-specific logic is removed here as the underlying issue is caused
by the request forwarding mechanism.
* Add backend plugin changes
* Fix totp backend plugin tests
* Fix logical/plugin InvalidateKey test
* Fix plugin catalog CRUD test, fix NoopBackend
* Clean up commented code block
* Fix system backend mount test
* Set plugin_name to omitempty, fix handleMountTable config parsing
* Clean up comments, keep shim connections alive until cleanup
* Include pluginClient, disallow LookupPlugin call from within a plugin
* Add wrapper around backendPluginClient for proper cleanup
* Add logger shim tests
* Add logger, storage, and system shim tests
* Use pointer receivers for system view shim
* Use plugin name if no path is provided on mount
* Enable plugins for auth backends
* Add backend type attribute, move builtin/plugin/package
* Fix merge conflict
* Fix missing plugin name in mount config
* Add integration tests on enabling auth backend plugins
* Remove dependency cycle on mock-plugin
* Add passthrough backend plugin, use logical.BackendType to determine lease generation
* Remove vault package dependency on passthrough package
* Add basic impl test for passthrough plugin
* Incorporate feedback; set b.backend after shims creation on backendPluginServer
* Fix totp plugin test
* Add plugin backends docs
* Fix tests
* Fix builtin/plugin tests
* Remove flatten from PluginRunner fields
* Move mock plugin to logical/plugin, remove totp and passthrough plugins
* Move pluginMap into newPluginClient
* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck
* Change shim logger's Fatal to no-op
* Change BackendType to uint32, match UX backend types
* Change framework.Backend Setup signature
* Add Setup func to logical.Backend interface
* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments
* Remove commented var in plugin package
* RegisterLicense on logical.Backend interface (#3017)
* Add RegisterLicense to logical.Backend interface
* Update RegisterLicense to use callback func on framework.Backend
* Refactor framework.Backend.RegisterLicense
* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs
* plugin: Revert BackendType to remove TypePassthrough and related references
* Fix typo in plugin backends docs
* Let AWS auth CLI helper only generate login data
This will be useful to other golang clients so they can manage the login
process themselves.
Also helps for #2855
* Respond to PR feedback
In auth/aws/config/client, when only the iam_server_id_header_value was
being updated on an existing config, it wouldn't get stored because I
was trying to avoid unnecessarily flushing the cache of AWS clients, and
the flag to not flush the cache also meant that the updated entry didn't
get written back to the storage. This now adds a new flag for when
other changes occur that don't require flushing the cache but do require
getting written to the storage. It also adds a test for this explicitly.
Fixes#3004
Previously, the renew method would ALWAYS check to ensure the
authenticated IAM principal ARN matched the bound ARN. However, there
is a valid use case in which no bound_iam_principal_arn is specified and
all bindings are done through inferencing. When a role is configured
like this, clients won't be able to renew their token because of the
check.
This now checks to ensure that the bound_iam_principal_arn is not empty
before requriing that it match the originally authenticated client.
Fixes#2781
A change in copystructure has caused some panics due to the custom copy
function. I'm more nervous about production panics than I am about
keeping some bad code wiping out some existing warnings, so remove the
custom copy function and just allow direct setting of Warnings.
* Initialized basic outline of TOTP backend using Postgresql backend as template
* Updated TOTP backend.go's structure and help string
* Updated TOTP path_roles.go's structure and help strings
* Updated TOTP path_role_create.go's structure and help strings
* Fixed typo in path_roles.go
* Fixed errors in path_role_create.go and path_roles.go
* Added TOTP secret backend information to cli commands
* Fixed build errors in path_roles.go and path_role_create.go
* Changed field values of period and digits from uint to int, added uint conversion of period when generating passwords
* Initialized TOTP test file based on structure of postgresql test file
* Added enforcement of input values
* Added otp library to vendor folder
* Added test steps and cleaned up errors
* Modified read credential test step, not working yet
* Use of vendored package not allowed - Test error
* Removed vendor files for TOTP library
* Revert "Removed vendor files for TOTP library"
This reverts commit fcd030994bc1741dbf490f3995944e091b11da61.
* Hopefully fixed vendor folder issue with TOTP Library
* Added additional tests for TOTP backend
* Cleaned up comments in TOTP backend_test.go
* Added default values of period, algorithm and digits to field schema
* Changed account_name and issuer fields to optional
* Removed MD5 as a hash algorithm option
* Implemented requested pull request changes
* Added ability to validate TOTP codes
* Added ability to have a key generated
* Added skew, qr size and key size parameters
* Reset vendor.json prior to merge
* Readded otp and barcode libraries to vendor.json
* Modified help strings for path_role_create.go
* Fixed test issue in testAccStepReadRole
* Cleaned up error formatting, variable names and path names. Also added some additional documentation
* Moveed barcode and url output to key creation function and did some additional cleanup based on requested changes
* Added ability to pass in TOTP urls
* Added additional tests for TOTP server functions
* Removed unused QRSize, URL and Generate members of keyEntry struct
* Removed unnecessary urlstring variable from pathKeyCreate
* Added website documentation for TOTP secret backend
* Added errors if generate is true and url or key is passed, removed logger from backend, and revised parameter documentation.
* Updated website documentation and added QR example
* Added exported variable and ability to disable QR generation, cleaned up error reporting, changed default skew value, updated documentation and added additional tests
* Updated API documentation to inlude to exported variable and qr size option
* Cleaned up return statements in path_code, added error handling while validating codes and clarified documentation for generate parameters in path_keys
* Refactor to consolidate constraints on the matching chain
* Add CN prefix/suffix constraint
* Maintain backwards compatibility (pick a random cert if multiple match)
* Vendor go-glob
* Replace cn_prefix/suffix with required_name/globbing
Move all the new tests to acceptance-capable tests instead of embedding in the CRL test
* Allow authenticating against a single cert
* Add new params to documentation
* Add CLI support for new param
* Refactor for style
* Support multiple (ORed) name patterns
* Rename required_names to allowed_names
* Update docs for parameter rename
* Use the new TypeCommaStringSlice
* Rename builtin/credential/aws-ec2 to aws
The aws-ec2 authentication backend is being expanded and will become the
generic aws backend. This is a small rename commit to keep the commit
history clean.
* Expand aws-ec2 backend to more generic aws
This adds the ability to authenticate arbitrary AWS IAM principals using
AWS's sts:GetCallerIdentity method. The AWS-EC2 auth backend is being to
just AWS with the expansion.
* Add missing aws auth handler to CLI
This was omitted from the previous commit
* aws auth backend general variable name cleanup
Also fixed a bug where allowed auth types weren't being checked upon
login, and added tests for it.
* Update docs for the aws auth backend
* Refactor aws bind validation
* Fix env var override in aws backend test
Intent is to override the AWS environment variables with the TEST_*
versions if they are set, but the reverse was happening.
* Update docs on use of IAM authentication profile
AWS now allows you to change the instance profile of a running instance,
so the use case of "a long-lived instance that's not in an instance
profile" no longer means you have to use the the EC2 auth method. You
can now just change the instance profile on the fly.
* Fix typo in aws auth cli help
* Respond to PR feedback
* More PR feedback
* Respond to additional PR feedback
* Address more feedback on aws auth PR
* Make aws auth_type immutable per role
* Address more aws auth PR feedback
* Address more iam auth PR feedback
* Rename aws-ec2.html.md to aws.html.md
Per PR feedback, to go along with new backend name.
* Add MountType to logical.Request
* Make default aws auth_type dependent upon MountType
When MountType is aws-ec2, default to ec2 auth_type for backwards
compatibility with legacy roles. Otherwise, default to iam.
* Pass MountPoint and MountType back up to the core
Previously the request router reset the MountPoint and MountType back to
the empty string before returning to the core. This ensures they get set
back to the correct values.
* WIP on invalidate function
* cassandraConnectionProducer has Close()
* Delete database from connections map on successful db.Close()
* Move clear connection into its own func
* Use const for database config path
Adding a boolean field that determines whether users will be allowed to
set the ID of the signed SSH key or whether it will always be the token
display name. Preventing users from changing the ID and always using
the token name is useful for auditing who actually used a key to access
a remote host since sshd logs key IDs.
* Use constants for storage paths
* Upgrade path for public key storage
* Fix calculateValidPrincipals, upgrade ca_private_key, and other changes
* Remove a print statement
* Added tests for upgrade case
* Make exporting consistent in creation bundle
* unexporting and constants
* Move keys into a struct instead of plain string
* minor changes
* docs: input format for default_critical_options and default_extensions
* s/sshca/ssh
* Added default_critical_options and default_extensions to the read endpoint of role
* Change default time return value to 0
A static token at the beginning of a log line can help systems parse
logs better. For example, rsyslog and syslog-ng will recognize the
'@cee: ' prefix and will parse the rest of the line as a valid json message.
This is useful in environments where there is a mix of structured and
unstructured logs.
* transit: change batch input format
* transit: no json-in-json for batch response
* docs: transit: update batch input format
* transit: fix tests after changing response format
* Add /sys/config/audited-headers endpoint for configuring the headers that will be audited
* Remove some debug lines
* Add a persistant layer and refactor a bit
* update the api endpoints to be more restful
* Add comments and clean up a few functions
* Remove unneeded hash structure functionaility
* Fix existing tests
* Add tests
* Add test for Applying the header config
* Add Benchmark for the ApplyConfig method
* ResetTimer on the benchmark:
* Update the headers comment
* Add test for audit broker
* Use hyphens instead of camel case
* Add size paramater to the allocation of the result map
* Fix the tests for the audit broker
* PR feedback
* update the path and permissions on config/* paths
* Add docs file
* Fix TestSystemBackend_RootPaths test
* Transit: Support batch encryption
* Address review feedback
* Make the normal flow go through as a batch request
* Transit: Error out if encryption fails during batch processing
* Transit: Infer the 'derived' parameter based on 'context' being set
* Transit: Batch encryption doc updates
* Transit: Return a JSON string instead of []byte
* Transit: Add batch encryption tests
* Remove plaintext empty check
* Added tests for batch encryption, more coming..
* Added more batch encryption tests
* Check for base64 decoding of plaintext before encrypting
* Transit: Support batch decryption
* Transit: Added tests for batch decryption
* Transit: Doc update for batch decryption
* Transit: Sync the path-help and website docs for decrypt endpoint
* Add batch processing for rewrap
* transit: input validation for context
* transit: add rewrap batch option to docs
* Remove unnecessary variables from test
* transit: Added tests for rewrap use cases
* Address review feedback
* Address review feedback
* Address review feedback
* transit: move input checking out of critical path
* transit: allow empty plaintexts for batch encryption
* transit: use common structs for batch processing
* transit: avoid duplicate creation of structs; add omitempty to response structs
* transit: address review feedback
* transit: fix tests
* address review feedback
* transit: fix tests
* transit: rewrap encrypt user error should not error out
* transit: error out for internal errors
* approle: secret-id listing lock sanity
* Skip processing an empty secretIDHMAC item during the iteration
* approle: use dedicated lock for listing of secret-id-accessors
The rolename is now saved to the secret internal data for fetching
later during the user revocation process. No longer deriving the role
name from request path
Added support for default revoke SQL statements that will provide the
same functionality as before. If not revoke SQL statements are provided
the default statements are used.
Cleaned up personal ignores from the .gitignore file
Adds HUP support for audit log files to close and reopen. This makes it
much easier to deal with normal log rotation methods.
As part of testing this I noticed that HUP and other items that come out
of command/server.go are going to stderr, which is where our normal log
lines go. This isn't so much problematic with our normal output but as
we officially move to supporting other formats this can cause
interleaving issues, so I moved those to stdout instead.
In order for Vault to map IAM instance profiles to roles, Vault
must query the 'iam:GetInstanceProfile' API, so update the documentation
and help to include the additional permissions needed.
The removal of a user from the db is now handled similar to the
creation. The SQL is read out of a key from the role and then executed
with values substituted for username.
This fixes#1911 but not directly; it doesn't address the cause of the
panic. However, it turns out that this is the correct fix anyways,
because it ensures that the value being logged is RFC3339 format, which
is what the time turns into in JSON but not the normal time string
value, so what we audit log (and HMAC) matches what we are returning.
Use the stored information to validate the source address and credential issue time.
Correct the logic used to verify BoundCIDRList on the role.
Reverify the subset requirements between secret ID and role during credential issue time.
Both revocation times are UTC so clarify via parameter name that it's just a formatting difference. Also leave as a time.Time here, as it automatically marshals into RFC3339.
