* WIP: output policy
* Outputs example policy HCL for given request
* Simplify conditional
* Add PATCH capability
* Use OpenAPI spec and regex patterns to determine if path is sudo
* Add test for isSudoPath
* Add changelog
* Fix broken CLI tests
* Add output-policy to client cloning code
* Smaller fixes from PR comments
* Clone client instead of saving and restoring custom values
* Fix test
* Address comments
* Don't unset output-policy flag on KV requests otherwise the preflight request will fail and not populate LastOutputPolicyError
* Print errors saved in buffer from preflight KV requests
* Unescape characters in request URL
* Rename methods and properties to improve readability
* Put KV-specificness at front of KV-specific error
* Simplify logic by doing more direct returns of strings and errors
* Use precompiled regexes and move OpenAPI call to tests
* Remove commented out code
* Remove legacy MFA paths
* Remove unnecessary use of client
* Move sudo paths map to plugin helper
* Remove unused error return
* Add explanatory comment
* Remove need to pass in address
* Make {name} regex less greedy
* Use method and path instead of info from retryablerequest
* Add test for IsSudoPaths, use more idiomatic naming
* Use precompiled regexes and move OpenAPI call to tests (#15170)
* Use precompiled regexes and move OpenAPI call to tests
* Remove commented out code
* Remove legacy MFA paths
* Remove unnecessary use of client
* Move sudo paths map to plugin helper
* Remove unused error return
* Add explanatory comment
* Remove need to pass in address
* Make {name} regex less greedy
* Use method and path instead of info from retryablerequest
* Add test for IsSudoPaths, use more idiomatic naming
* Make stderr writing more obvious, fix nil pointer deref
* fix raft tls key rotation panic when rotation time in past
* add changelog entry
* push out next raft TLS rotation time in case close to elapsing
* consolidate tls key rotation duration calculation
* reduce raft getNextRotationTime padding to 10 seconds
* move tls rotation ticker reset to where its duration is calculated
* add BuildDate to version base
* populate BuildDate with ldflags
* include BuildDate in FullVersionNumber
* add BuildDate to seal-status and associated status cmd
* extend core/versions entries to include BuildDate
* include BuildDate in version-history API and CLI
* fix version history tests
* fix sys status tests
* fix TestStatusFormat
* remove extraneous LD_FLAGS from build.sh
* add BuildDate to build.bat
* fix TestSysUnseal_Reset
* attempt to add build-date to release builds
* add branch to github build workflow
* add get-build-date to build-* job needs
* fix release build command vars
* add missing quote in release build command
* Revert "add branch to github build workflow"
This reverts commit b835699ecb7c2c632757fa5fe64b3d5f60d2a886.
* add changelog entry
* fix dev-plugin-dir when backend is builtin
* use builtinRegistry.Contains
* revert aa76337
* use correct plugin type for logical backend after revert
* fix factory func default setting after revert
* add ut coverage for builtin plugin with plugin directory set
* add coverage for secrets plugin type
* use totp in tests to avoid test import cycle in ssh package
* use nomad in tests to avoid test import cycle
* remove secrets mount tests due to unavoidable test import cycle
* VAULT-5422: Add rate limit for TOTP passcode attempts
* fixing the docs
* CL
* feedback
* Additional info in doc
* rate limit is done per entity per methodID
* refactoring a test
* rate limit OSS work for policy MFA
* adding max_validation_attempts to TOTP config
* feedback
* checking for non-nil reference
* Address incorrect table metric value for local mounts
- Reported within issue #14750 as a panic, it was identified that
we were using the wrong value for local mounts within the table metrics.
* Add changelog
* Warn on upper case in policy name
* Rename name variable to be less confusing
* Use more general solution for other string issues
* Clarify changelog
* Remove unnecessary check
* Don't throw CLI warning until after past errors
* Add before and after names with quotes to show spacing changes
* remove mount accessor from MFA config
* Update login_mfa_duo_test.go
* DUO test with entity templating
* using identitytpl.PopulateString to perform templating
* minor refactoring
* fixing fmt failures in CI
* change username format to username template
* fixing username_template example
* Address slow CI causing failures in TestRateLimitQuota_Allow_WithBlock
- An attempt to fix CI runs that are extremely slow and the for loop
runs across two BlockIntervals within the rate limit window of operation.
- Increasing BlockInterval was looked at but the normal test times would
be increased due to us also validating that we are releasing clients post
BlockInterval.
* Address TestRateLimitQuota_Allow_WithBlock slowness issue (take 2)
- Increase the overall Interval value within the blocking test so that we
should always be able to request at least 17 requests within the interval
value.
- Tested by changing the time.Sleep within the for loop to 20 from 2
and could see that within the response, coming back from the rlq.allow,
that we were no longer being rate limited by going over the 1 second
interval value per host.
* Limit the number of active go routines in TestRateLimitQuota_Allow_WithBlock
* Login MFA
* ENT OSS segragation (#14088)
* Delete method id if not used in an MFA enforcement config (#14063)
* Delete an MFA methodID only if it is not used by an MFA enforcement config
* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path
* adding use_passcode field to DUO config (#14059)
* add changelog
* preventing replay attack on MFA passcodes (#14056)
* preventing replay attack on MFA passcodes
* using %w instead of %s for error
* Improve CLI command for login mfa (#14106)
CLI prints a warning message indicating the login request needs to get validated
* adding the validity period of a passcode to error messages (#14115)
* interactive CLI for mfa login
* minor fixes
* bail if no input was inserted
* change label name
* interactive CLI when single methodID is returned from login request
* minor fix
* adding changelog
* addressing feedback
* a user with a terminal should be able to choose between interactive and non-interactive. A user without a terminal should not be able to use the interactive mode.
Co-authored-by: Josh Black <raskchanky@gmail.com>
* achieve parity with ent in core.go
* add VAULT_DISABLE_LOCAL_AUTH_MOUNT_ENTITIES
* parity in build.yml with ent but without adding the +ent
* pass base version to ldflags
Co-authored-by: Kyle Penfound <kpenfound11@gmail.com>
* Various changes to try to ensure that fewer goroutines survive after a test completes:
* add Core.ShutdownWait that doesn't return until shutdown is done
* create the usedCodes cache on seal and nil it out on pre-seal so that the finalizer kills the janitor goroutine
* stop seal health checks on seal rather than wait for them to discover the active context is done
* make sure all lease-loading goroutines are done before returning from restore
* make uniquePoliciesGc discover closed quitCh immediately instead of only when the ticker fires
* make sure all loading goroutines are done before returning from loadEntities, loadCachedEntitiesOfLocalAliases
* go get plugin-secrets-kv@v0.11.0; go mod tidy
* add HTTP-level tests for KV subkeys endpoint
* check status in TestKV_Subkeys_CurrentVersion
* some test cleanup
* Update plugin-portal.mdx (#13229)
Add a Vault plugin to allow authentication via SSH certificates and public keys
* oss changes
Co-authored-by: Wim <wim@42.be>
* plugin/catalog: support plugin registration whe type is explicitly provided
* don't use database type on plugin backend test; mock doesn't satisfy the DB interface
* check multiplexing support from plugin directly on newPluginClient
* do not return mutiplexed bool on catalog helper funcs
* Login MFA
* ENT OSS segragation (#14088)
* Delete method id if not used in an MFA enforcement config (#14063)
* Delete an MFA methodID only if it is not used by an MFA enforcement config
* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path
* adding use_passcode field to DUO config (#14059)
* add changelog
* preventing replay attack on MFA passcodes (#14056)
* preventing replay attack on MFA passcodes
* using %w instead of %s for error
* Improve CLI command for login mfa (#14106)
CLI prints a warning message indicating the login request needs to get validated
* adding the validity period of a passcode to error messages (#14115)
* PR feedback
* duo to handle preventing passcode reuse
Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
* feat: DB plugin multiplexing (#13734)
* WIP: start from main and get a plugin runner from core
* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode
* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient
* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient
* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists
* add ID to dbplugin.DatabasePluginClient struct
* only create one plugin process per plugin type
* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server
* add v6 multiplexing protocol version
* WIP: backwards compat for db plugins
* Ensure locking on plugin catalog access
- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin
* use the New constructor for db plugins
* grpc server: use write lock for Close and rlock for CRUD
* cleanup MultiplexedClients on Close
* remove TODO
* fix multiplexing regression with grpc server connection
* cleanup grpc server instances on close
* embed ClientProtocol in Multiplexer interface
* use PluginClientConfig arg to make NewPluginClient plugin type agnostic
* create a new plugin process for non-muxed plugins
* feat: plugin multiplexing: handle plugin client cleanup (#13896)
* use closure for plugin client cleanup
* log and return errors; add comments
* move rpcClient wrapping to core for ID injection
* refactor core plugin client and sdk
* remove unused ID method
* refactor and only wrap clientConn on multiplexed plugins
* rename structs and do not export types
* Slight refactor of system view interface
* Revert "Slight refactor of system view interface"
This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.
* Revert "Revert "Slight refactor of system view interface""
This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.
* only provide pluginRunner arg to the internal newPluginClient method
* embed ClientProtocol in pluginClient and name logger
* Add back MLock support
* remove enableMlock arg from setupPluginCatalog
* rename plugin util interface to PluginClient
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
* feature: multiplexing: fix unit tests (#14007)
* fix grpc_server tests and add coverage
* update run_config tests
* add happy path test case for grpc_server ID from context
* update test helpers
* feat: multiplexing: handle v5 plugin compiled with new sdk
* add mux supported flag and increase test coverage
* set multiplexingSupport field in plugin server
* remove multiplexingSupport field in sdk
* revert postgres to non-multiplexed
* add comments on grpc server fields
* use pointer receiver on grpc server methods
* add changelog
* use pointer for grpcserver instance
* Use a gRPC server to determine if a plugin should be multiplexed
* Apply suggestions from code review
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* add lock to removePluginClient
* add multiplexingSupport field to externalPlugin struct
* do not send nil to grpc MultiplexingSupport
* check err before logging
* handle locking scenario for cleanupFunc
* allow ServeConfigMultiplex to dispense v5 plugin
* reposition structs, add err check and comments
* add comment on locking for cleanupExternalPlugin
* feature: multiplexing: handle error plugin shutdown
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* port SSCT OSS
* port header hmac key to ent and generate token proto without make command
* remove extra nil check in request handling
* add changelog
* add comment to router.go
* change test var to use length constants
* remove local index is 0 check and extra defer which can be removed after use of ExternalID
* adds development workflow to mirage config
* adds mirage handler and factory for mfa workflow
* adds mfa handling to auth service and cluster adapter
* moves auth success logic from form to controller
* adds mfa form component
* shows delayed auth message for all methods
* adds new code delay to mfa form
* adds error views
* fixes merge conflict
* adds integration tests for mfa-form component
* fixes auth tests
* updates mfa response handling to align with backend
* updates mfa-form to handle multiple methods and constraints
* adds noDefault arg to Select component
* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios
* adds tests
* flaky test fix attempt
* reverts test fix attempt
* adds changelog entry
* updates comments for todo items
* removes faker from mfa mirage factory and handler
* adds number to word helper
* fixes tests
* Revert "Merge branch 'main' into ui/mfa"
This reverts commit 8ee6a6aaa1b6c9ec16b985c10d91c3806819ec40, reversing
changes made to 2428dd6cca07bb41cda3f453619646ca3a88bfd0.
* format-ttl helper fix from main
* feat: DB plugin multiplexing (#13734)
* WIP: start from main and get a plugin runner from core
* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode
* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient
* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient
* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists
* add ID to dbplugin.DatabasePluginClient struct
* only create one plugin process per plugin type
* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server
* add v6 multiplexing protocol version
* WIP: backwards compat for db plugins
* Ensure locking on plugin catalog access
- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin
* use the New constructor for db plugins
* grpc server: use write lock for Close and rlock for CRUD
* cleanup MultiplexedClients on Close
* remove TODO
* fix multiplexing regression with grpc server connection
* cleanup grpc server instances on close
* embed ClientProtocol in Multiplexer interface
* use PluginClientConfig arg to make NewPluginClient plugin type agnostic
* create a new plugin process for non-muxed plugins
* feat: plugin multiplexing: handle plugin client cleanup (#13896)
* use closure for plugin client cleanup
* log and return errors; add comments
* move rpcClient wrapping to core for ID injection
* refactor core plugin client and sdk
* remove unused ID method
* refactor and only wrap clientConn on multiplexed plugins
* rename structs and do not export types
* Slight refactor of system view interface
* Revert "Slight refactor of system view interface"
This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.
* Revert "Revert "Slight refactor of system view interface""
This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.
* only provide pluginRunner arg to the internal newPluginClient method
* embed ClientProtocol in pluginClient and name logger
* Add back MLock support
* remove enableMlock arg from setupPluginCatalog
* rename plugin util interface to PluginClient
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
* feature: multiplexing: fix unit tests (#14007)
* fix grpc_server tests and add coverage
* update run_config tests
* add happy path test case for grpc_server ID from context
* update test helpers
* feat: multiplexing: handle v5 plugin compiled with new sdk
* add mux supported flag and increase test coverage
* set multiplexingSupport field in plugin server
* remove multiplexingSupport field in sdk
* revert postgres to non-multiplexed
* add comments on grpc server fields
* use pointer receiver on grpc server methods
* add changelog
* use pointer for grpcserver instance
* Use a gRPC server to determine if a plugin should be multiplexed
* Apply suggestions from code review
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* add lock to removePluginClient
* add multiplexingSupport field to externalPlugin struct
* do not send nil to grpc MultiplexingSupport
* check err before logging
* handle locking scenario for cleanupFunc
* allow ServeConfigMultiplex to dispense v5 plugin
* reposition structs, add err check and comments
* add comment on locking for cleanupExternalPlugin
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* store version history as utc; add self-heal logic
* add sys/version-history endpoint
* change version history from GET to LIST, require auth
* add "vault version-history" CLI command
* add vault-version CLI error message for version string parsing
* adding version-history API and CLI docs
* add changelog entry
* some version-history command fixes
* remove extraneous cmd args
* fix version-history command help text
* specify in docs that endpoint was added in 1.10.0
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* enforce UTC within storeVersionTimestamp directly
* fix improper use of %w in logger.Warn
* remove extra err check and erroneous return from loadVersionTimestamps
* add >= 1.10.0 warning to version-history cmd
* move sys/version-history tests
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Add check for OIDC provider to permit a non-exact redirect URI from OIDC client if it is the IPv4 or IPv6 loopback address.
* Update changelog/13871.txt
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
* Update redirectURI check to match that for the OIDC auth method.
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
Seal wrap storage entries of the ManagedKeyRegistry.
Specify the managed key registry's path in SealWrapStorage in the SystemBackend.
Enable SealWrap in the system mount entry.
This function call was previously used to generate mappings from
potential subjects (or SANs) to certificates within the TLS client
object. However, newer Go versions have deprecated this method, instead
building the mapping automatically based on present certificates at
request time. Because the corresponding client configuration field is
not used in Vault (NameToCertificate), it is safe to remove this call
and leave it nil.
See also: 67d894ee65
See also: https://pkg.go.dev/crypto/tls#Config.BuildNameToCertificate
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* check for nil signing key on rotation
* add changelog
* Update nil signing key handling
- bypass setting ExpireAt if signing key is nil in rotate
- return err if singing key is nil in signPayload
* add comment; update error msg on signPayload; refactor UT
In doing some testing I found that the listener clusteraddr isn't really used, or at least isn't as important as the top-level clusteraddr setting. As such, go-sockaddr templating needs to be implemented for the top-level `cluster_addr` setting or it's unusable for HA.
Also fix a nil pointer panic I discovered at the same time.
The auth/token/revoke will not error out if the token does not exists, it
always tries to revoke the token and return success to the client whether
or not the token exists. This makes the behavior of
auth/token/revoke-accessor coherent with this and remove the need to
check whether the token still exists.
* go get vault-plugin-secrets-kv@vault-4290-patch-metadata
* add kv metadata patch command
* add changelog entry
* success tests for kv metadata patch flags
* add more kv metadata patch flags tests
* add kv metadata patch cas warning test
* add kv-v2 key metadata patch API docs
* add kv metadata patch to docs
* prevent unintentional field overwriting in kv metadata put cmd
* like create/update ops, prevent patch to paths ending in /
* fix kv metadata patch cmd in docs
* fix flag defaults for kv metadata put
* go get vault-plugin-secrets-kv@vault-4290-patch-metadata
* fix TestKvMetadataPatchCommand_Flags test
* doc fixes
* go get vault-plugin-secrets-kv@master; go mod tidy
* Attempt to address a data race issue within identity store
* Testcase TestIdentityStore_LocalAliasInvalidations identified a data race issue.
* This reverts the previous attempt to address the issue from #13093
* Fixed null token panic from 'v1/auth/token/' endpoints and returned proper error response
* added changelog entry for PR #13233
* changed error message from 'bad token' to 'null token'
* rebased off of main
* Revert "changed error message from 'bad token' to 'null token'"
This reverts commit 381ed9b32c5ddd5e47adb1643ef7e46fb768bc76.
* changed 'bad token' error message to 'invalid token' after revert
* remove unnecessary vault-data folder
* Support clearing an identity alias' custom_metadata
Previously, an update to an entity alias supported updating the
custom_metadata as long as the update was not empty, which makes it
impossible to clear the metadata values completely.
Fixes:
- empty custom_metadata parameters are honoured on entity alias update
- update related tests
- drop dependency on mapstructure
- reformat with gofumpt
* VAULT-1564 report in-flight requests
* adding a changelog
* Changing some variable names and fixing comments
* minor style change
* adding unauthenticated support for in-flight-req
* adding documentation for the listener.profiling stanza
* adding an atomic counter for the inflight requests
addressing comments
* addressing comments
* logging completed requests
* fixing a test
* providing log_requests_info as a config option to determine at which level requests should be logged
* removing a member and a method from the StatusHeaderResponseWriter struct
* adding api docks
* revert changes in NewHTTPResponseWriter
* Fix logging invalid log_requests_info value
* Addressing comments
* Fixing a test
* use an tomic value for logRequestsInfo, and moving the CreateClientID function to Core
* fixing go.sum
* minor refactoring
* protecting InFlightRequests from data race
* another try on fixing a data race
* another try to fix a data race
* addressing comments
* fixing couple of tests
* changing log_requests_info to log_requests_level
* minor style change
* fixing a test
* removing the lock in InFlightRequests
* use single-argument form for interface assertion
* adding doc for the new configuration paramter
* adding the new doc to the nav data file
* minor fix
* Adding support for SHA3 in the transit backend.
* Adds SHA-3 tests for transit sign/verify path. Adds SHA-3 tests for logical system tools path hash functionality. Updates documentation to include SHA-3 algorithms in system tools path hashing.
* Adds changelog entry.
Co-authored-by: robison jacka <robison@packetized.io>
* See what it looks like to replace "master key" with "root key". There are two places that would require more challenging code changes: the storage path `core/master`, and its contents (the JSON-serialized EncodedKeyringtructure.)
* Restore accidentally deleted line
* Add changelog
* Update root->recovery
* Fix test
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Unify NewHTTPResponseWriter ant NewStatusHeaderResponseWriter to fix ResponseWriter issues
* adding changelog
* removing unnecessary function from the WrappingResponseWriter interface
* changing logical requests responseWriter type
* reverting change to HTTPResponseWriter
* don't log token error on DR Secondary
* stop gauge collector expiration errors on dr secondary
* don't check dr secondary for token create
* see if CI hits panic
* Revert "don't check dr secondary for token create"
This reverts commit c036a1a544d3a20d29d046f1ee239ab1563ce4d9.
* don't check dr secondary for token create
* Revert "see if CI hits panic"
This reverts commit 1e15aa535cac6e4d1684aaf47c8746c094068eb8.
* remove condition on log
* Move the ctx capture outside the goroutine to avoid a race
* refactor the toggleable wrapper to avoid races
* Move the capture back outside the goroutine
* defer
* Add a periodic test of the autoseal to detect loss of connectivity
* Keep the logic adjacent to autoseal
* imports
* typo, plus unnecessary constant time compare
* changelog
* pr feedback
* More feedback
* Add locking and a unit test
* unnecessary
* Add timeouts to encrypt/decrypt operations, capture activeContext before starting loop
* Add a block scope for the timeout
* copy/paste ftl
* Refactor to use two timeouts, and cleanup the repetitive failure code
* Readd 0ing gauge
* use millis
* Invert the unit test logic
* fix adding clientID to request in audit log
* fix boolean statement
* use standard encoding for client ID instead of urlEncoding
* change encoding in tests
* add in client counts to request handling
* remove redundant client ID generation in request handling
* directly add clientID to req after handling token usage
* removed unpublished:true for sys/internal/* endpoints
* added changelog file
* updated change log and added placeholder summary as these endpoints are not mentioned in docs.
* added documentation for internal/ui/namspaces and resultant-acl
* updated log configs
* Disallow alias creation if entity/accessor combination exists
* Add changelog
* Address review comments
* Add handling to aliasUpdate, some field renaming
* Update tests to work under new entity-alias constraint
* Add check to entity merge, other review fixes
* Log duplicated accessors only once
* Fix flaky test
* Add note about new constraint to docs
* Update entity merge warn log
* handle HTTP PATCH requests as logical.PatchOperation
* update go.mod, go.sum
* a nil response for logical.PatchOperation should result in 404
* respond with 415 for incorrect MIME type in PATCH Content-Type header
* add abstraction to handle PatchOperation requests
* add ACLs for patch
* Adding JSON Merge support to the API client
* add HTTP PATCH tests to check high level response logic
* add permission-based 'kv patch' tests in prep to add HTTP PATCH
* adding more 'kv patch' CLI command tests
* fix TestHandler_Patch_NotFound
* Fix TestKvPatchCommand_StdinValue
* add audit log test for HTTP PATCH
* patch CLI changes
* add patch CLI tests
* change JSONMergePatch func to accept a ctx
* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag
* go fmt
* add a test to verify patching works by default with the root token
* add changelog entry
* get vault-plugin-secrets-kv@add-patch-support
* PR feedback
* reorder some imports; go fmt
* add doc comment for HandlePatchOperation
* add json-patch@v5.5.0 to go.mod
* remove unnecessary cancelFunc for WriteBytes
* remove default for -method
* use stable version of json-patch; go mod tidy
* more PR feedback
* temp go get vault-plugin-secrets-kv@master until official release
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
* store unauthenticated path wildcards in map
* working unauthenticated paths with basic unit tests
* refactor wildcard logic
* add parseUnauthenticatedPaths unit tests
* use parseUnauthenticatedPaths when reloading backend
* add more wildcard test cases
* update special paths doc; add changelog
* remove buggy prefix check; add test cases
* prevent false positives for prefix matches
If we ever encounter a mismatched segment, break and set a flag to
prevent false positives for prefix matches.
If it is a match we need to do a prefix check. But we should not return
unless HasPrefix also evaluates to true. Otherwise we should let the for
loop continue to check other possibilities and only return false once
all wildcard paths have been evaluated.
* refactor switch and add more test cases
* remove comment leftover from debug session
* add more wildcard path validation and test cases
* update changelong; feature -> improvement
* simplify wildcard segment matching logic
* refactor wildcard matching into func
* fix glob matching, add more wildcard validation, refactor
* refactor common wildcard errors to func
* move doc comment to logical.Paths
* optimize wildcard paths storage with pre-split slices
* fix comment typo
* fix test case after changing wildcard paths storage type
* move prefix check to parseUnauthenticatedPaths
* tweak regex, remove unneeded array copy, refactor
* add test case around wildcard and glob matching
* Customizing HTTP headers in the config file
* Add changelog, fix bad imports
* fixing some bugs
* fixing interaction of custom headers and /ui
* Defining a member in core to set custom response headers
* missing additional file
* Some refactoring
* Adding automated tests for the feature
* Changing some error messages based on some recommendations
* Incorporating custom response headers struct into the request context
* removing some unused references
* fixing a test
* changing some error messages, removing a default header value from /ui
* fixing a test
* wrapping ResponseWriter to set the custom headers
* adding a new test
* some cleanup
* removing some extra lines
* Addressing comments
* fixing some agent tests
* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before
* Removing default custom headers, and renaming some function varibles
* some refacotring
* Refactoring and addressing comments
* removing a function and fixing comments
* filter identity token keys
* Update test cases to associate keys with roles
* use getOIDCRole helper
* add func comment and test assertion
* add changelog
* remove unnecessary code
* build list of keys to return by starting with a list of roles
* move comment
* update changelog
* remove cruft
use helper
Add a helper for getting public key sizes
wip
* error names
* Fix ecdsa
* only if trace is on
* Log listener side as well
* rename
* Add remote address
* Make the log level configurable via the env var, and a member of the Listener and thus modifiable by tests
* Fix certutil_test
* Adds ability to define an inline policy and internal metadata to tokens
* Update comment on fetchEntityAndDerivedPolicies
* Simplify handling of inline policy
* Update comment on InternalMeta
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* Improve argument name
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* Use explicit SkipIdentityInheritance token field instead of implicit InlinePolicy behavior
* Add SkipIdentityInheritance to pb struct in token store create method
* Rename SkipIdentityInheritance to NoIdentityPolicies
* Merge latest from main and make proto
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* move merge and compare states to vault core
* move MergeState, CompareStates and ParseRequiredStates to api package
* fix merge state reference in API Proxy
* move mergeStates test to api package
* add changelog
* ghost commit to trigger CI
* rename CompareStates to CompareReplicationStates
* rename MergeStates and make compareStates and parseStates private methods
* improved error messaging in parseReplicationState
* export ParseReplicationState for enterprise files
- When two entities are merged, remove the from entity ID in any
associated groups.
- When two entities are merged, also merge their associated group
memberships.
Fixes#10084
* Update Go client libraries for etcd
* Added etcd server container to run etcd3 tests automatically.
* Removed etcd2 test case: it fails the backend tests but the failure is
unrelated to the uplift. The etcd2 backend implementation does not
remove empty nested nodes when removing leaf (see comments in #11980).
* core: set namespace from the sysview's mount entry on GeneratePasswordFromPolicy
* test: update TestDynamicSystemView to be ns-aware, update tests
* add changelog entry
* [VAULT-3519] Return no_default_policy on token role read if set
* [VAULT-3519] Add changelog
* [VAULT-3519] Always return token_no_default_policy on role read
* Fix broken test
* Update role read response in docs
* Add allowed_policies_glob and disallowed_policies_glob that are the same as allowed_policies and disallowed_policies but allow glob matching.
* Update changelog, docs, tests, and comments for (dis)allowed_token_glob token role feature.
* Improve docs and unit tests for auth/token role policy globbing.
* Expose secret_id_accessor as WrappedAccessor when wrapping secret-id creation.
* Add changelog.
* Minor updates as suggested.
* Adding external test for wrapped accessor.
* Add check that mounttype is approle.
* Update changelog text to use improvement
* add keys path and initial handler
* read provider public keys
* add test cases
* remove some debug logs
* update tests after merging main
* refactor list all clients
* refactor logic to collect Key IDs
* pre-publish new signing keys for `rotation_period` of time before using
* Work In Progress: Prepublish JWKS and even cache control
* remove comments
* use math/rand instead of math/big
* update tests
* remove debug comment
* refactor cache control logic into func
* don't set expiry when create/update key
* update cachecontrol name in oidccache for test
* fix bug in periodicfunc test case
* add changelog
* remove confusing comment
* add logging and comments
* update change log from bug to improvement
Co-authored-by: Ian Ferguson <ian.ferguson@datadoghq.com>
* identity: handle creation of role without a key parameter
* update docs to not require key parameter for creation of a role
* add changelog
* require key param when creating a role
* lock create/update role; remove now redundant key check
* update changelog and UTs
* update change log to refelct actual implementation
* remove deprecated test case
* Auto-join support for IPv6 discovery
The go-discover library returns IP addresses and not URLs. It just so
happens net.URL parses "127.0.0.1", which isn't a valid URL.
Instead, we construct the URL ourselves. Being careful to check if it's
an ipv6 address and making sure it's in explicit form if so.
Fixes#12323
* feedback: addrs & ipv6 test
Rename addrs to clusterIPs to improve clarity and intent
Tighten up our IPv6 address detection to be more correct and to ensure
it's actually in implicit form
* OIDC Provider: implement discovery endpoint
* handle case when provider does not exist
* refactor providerDiscover struct and add scopes_supported
* fix authz endpoint
* initial commit
* add read and delete operations
* fix bug in delete and add list unit test
* func doc typo fix
* add existence check for assignment
* remove locking on the assignment resource
It is not needed at this time.
* convert Callbacks to Operations
- convert Callbacks to Operations
- add test case for update operations
* add CRUD operations and test cases
* add client api and tests
* remove use of oidcCache
* remove use of oidcCache
* add template validation and update tests
* remove usage of oidcCache
* refactor struct and var names
* harmonize test name conventions
* refactor struct and var names
* add changelog and refactor
- add changelog
- be more explicit in the case where we do not recieve a path field
* refactor
be more explicit in the case where a field is not provided
* remove extra period from changelog
* update scope path to be OIDC provider specific
* refactor naming conventions
* update assignment path
* update scope path
* enforce key existence on client creation
* removed unused name field
* removed unused name field
* removed unused name field
* prevent assignment deletion when ref'ed by a client
* enfoce assignment existence on client create/update
* update scope template description
* error when attempting to created scope with openid reserved name
* fix UT failures after requiring assignment existence
* disallow key deletion when ref'ed by existing client
* generate client_id and client_secret on CreateOp
* do not allow key modification on client update
* return client_id and client_secret on read ops
* small refactor
* fix bug in delete assignment op
* remove client secret get call
* OIDC Client API: add more test coverage
* change name convention in tests
* initial commit
* add read and delete operations
* fix bug in delete and add list unit test
* func doc typo fix
* add existence check for assignment
* remove locking on the assignment resource
It is not needed at this time.
* convert Callbacks to Operations
- convert Callbacks to Operations
- add test case for update operations
* add CRUD operations and test cases
* add client api and tests
* remove use of oidcCache
* remove use of oidcCache
* add template validation and update tests
* remove usage of oidcCache
* refactor struct and var names
* harmonize test name conventions
* refactor struct and var names
* add changelog and refactor
- add changelog
- be more explicit in the case where we do not recieve a path field
* refactor
be more explicit in the case where a field is not provided
* remove extra period from changelog
* update scope path to be OIDC provider specific
* refactor naming conventions
* update assignment path
* update scope path
* enforce key existence on client creation
* removed unused name field
* removed unused name field
* removed unused name field
* prevent assignment deletion when ref'ed by a client
* enfoce assignment existence on client create/update
* update scope template description
* error when attempting to created scope with openid reserved name
* fix UT failures after requiring assignment existence
* disallow key deletion when ref'ed by existing client
* generate client_id and client_secret on CreateOp
* do not allow key modification on client update
* return client_id and client_secret on read ops
* small refactor
* fix bug in delete assignment op
* remove client secret get call
* initial commit
* add read and delete operations
* fix bug in delete and add list unit test
* func doc typo fix
* add existence check for assignment
* remove locking on the assignment resource
It is not needed at this time.
* convert Callbacks to Operations
- convert Callbacks to Operations
- add test case for update operations
* add CRUD operations and test cases
* remove use of oidcCache
* remove use of oidcCache
* add template validation and update tests
* refactor struct and var names
* harmonize test name conventions
* refactor struct and var names
* add changelog and refactor
- add changelog
- be more explicit in the case where we do not recieve a path field
* refactor
be more explicit in the case where a field is not provided
* remove extra period from changelog
* update scope path to be OIDC provider specific
* update assignment path
* update scope path
* removed unused name field
* removed unused name field
* update scope template description
* error when attempting to created scope with openid reserved name
* oss part of vault 2399
* Update vault/quotas/quotas.go
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
* use OSS PR number as changelog entry as indicated by the changelog guide
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
* initial commit
* add read and delete operations
* fix bug in delete and add list unit test
* func doc typo fix
* add existence check for assignment
* remove locking on the assignment resource
It is not needed at this time.
* convert Callbacks to Operations
- convert Callbacks to Operations
- add test case for update operations
* remove use of oidcCache
* refactor struct and var names
* harmonize test name conventions
* add changelog and refactor
- add changelog
- be more explicit in the case where we do not recieve a path field
* remove extra period from changelog
* update assignment path
* removed unused name field
* VAULT-2285 adding capability to accept comma separated entries for auth enable/tune
* Adding changelog
* Adding logic to detect invalid input parameter for auth enable config
* Updating tune.mdx
* Updating secret enable/tune for comma separated parameters
* Adding further parameter checks for auth/secret tests
Fixing changelog
using builtin type for a switch statement
Fixing a possible panic scenario
* Changing a function name, using deep.Equal instead of what reflect package provides
* Fixing auth/secret enable/tune mdx files
* One more mdx file fix
* Only when users provide a single comma separated string in a curl command, split the entries by commas
* Fixing API docs for auth/mount enable/tune for comma separated entries
* updating docs, removing an unnecessary switch case
* do not allow token_ttl to be longer than verification_ttl
* add verification when updating an existing key
When updating a key, ensure any roles referencing the key do not already
have a token_ttl greater than the key's verification_ttl
* add changelog
* remove unneeded UT check and comment
* refactor based on PR comments
- remove make slice in favor of var delcaration
- remove unneeded if check
- validate expiry value during token generation
- update changelog as bug
* refactor get roles referencing target key names logic
* add note about thread safety to helper func
* update func comment
* sort array and refactor func names
* add warning to return response
* remove unnecessary code from unit test
* Update vault/identity_store_oidc.go
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
* hghaf099-VAULT-1303-Adding namespace in error when it is set
* casting ResponseWriter in handleMonitor to logical.NamespaceResponseWriter
* Casting ResponseWriter conditionally for http.Flusher
Adding changelog
* Improving changlog message
* save
* save
* save
* first round of the diagnose language pass
* capitalization
* first round of feedback
* fix bug in advise
* a few more nouns to verbs
* prototype function to retry irrevocable lease revocation, clean up comment
* setup irrevocable lease revoke retry on expiration manager creation
* reduce expiration lag to 1 hour, use core context and timeout on retry attempts
* add lease namespace to revocation call
* start irrevocable revocation attempts in setupExpiration
* grab reference to channel to avoid race test failures
* add leases path to sudo required set
* update TestSystemBackend_RootPaths with new special privilege paths
* note that list-leases requires sudo
* minor typo fixes
* diagnose: Add seal transit tls check
* Fixing the path to the config file and the path to the cert files
* Addressing comment
* Addressing seal transit tls check comments
* tls verification bugfix
* tls verification bugfix
* allow diagnose fail to report status when there are also warnings
* allow diagnose fail to report status when there are also warnings
* Update vault/diagnose/helpers_test.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* comments
Co-authored-by: swayne275 <swayne275@gmail.com>
* Checking Validity of all Certs in the chain
* Addressing Comments for TLS cert validation
* Fixing tls_verification tests
* Fixing minor issue in tls_verification tests
* Addressing Comments, Rebasing with main
* Adding comment on top of a test
* Actually call config.Validate in diagnose
* Wire configuration checks into diagnose and fix resulting bugs.
* go mod vendor
* Merge to vendorless version
* Remove sentinel section to allow diagnose_ok to pass
* Fix unit tests
* raft file and quorum checks
* raft checks
* backup
* raft file checks test
* address comments and add more raft and file and process checks
* syntax issues
* modularize functions to compile differently on different os
* compile raft checks everywhere
* more build tag issues
* raft-diagnose
* correct file permission checks
* upgrade tests and add a getConfigOffline test that currently does not work
* comment
* update file checks method signature on windows
* Update physical/raft/raft_test.go
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* raft tests
* add todo comment for windows root ownership
* voter count message
* raft checks test fixes
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* initial refactoring of unseal step in run
* remove waitgroup
* remove waitgroup
* backup work
* backup
* backup
* completely modularize run and move into diagnose
* add diagnose errors for incorrect number of unseal keys
* comment tests back in
* backup
* first subspan
* finished subspanning but running into error with timeouts
* remove runtime checks
* merge main branch
* meeting updates
* remove telemetry block
* roy comment
* subspans for seal finalization and wrapping diagnose latency checks
* backup while I fix something else
* fix storage latency test errors
* runtime checks
* diagnose with timeout on seal
* Acquire a per-lock lease to make renew and revoke atomic wrt each other.
This means we don't have to hold pendingLock during I/O.
* Attempted fix for deadlock in token revocation.
* Comment fix.
* Fix error checking in loadEntry.
* Add benchmark
* Add a few additional locking locations
* Improve benchmark slightly
* Update vault/expiration.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Update vault/expiration.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Add a lease lock into tidy
Co-authored-by: Scott Miller <smiller@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
* wip
* wip
* Finish implementing advice handling and word wrapping
* Properly word wrap messages and warnings
* Remove debugging
* Remove debugging
* Remove unnecessary test
* unit test bug
* go vendor
* Segment out disk checks to disable on openbsd/arm
Also add a spot skipped helper.
* Expected results may be fewer than actual because of variable length tests like disk usage
* Move to os_common and build on windows
* build out lease count (not fully working), start lease list
* build out irrevocable lease list
* bookkeeping
* test irrevocable lease counts for API/CLI
* fix listIrrevocableLeases, test listIrrevocableLeases, cleanup
* test expiration API limit
* namespace tweaks, test force flag on lease list
* integration test leases/count API, plenty of fixes and improvements
* test lease list API, fixes and improvements
* test force flag for irrevocable lease list API
* i guess this wasn't saved on the last refactor...
* fixes and improvements found during my review
* better test error msg
* Update vault/logical_system_paths.go
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* return warning with data if more than default leases to list without force flag
* make api doc more generalized
* list leases in general, not by mount point
* change force flag to include_large_results
* sort leases by LeaseID for consistent API response
* switch from bool flag for API limit to string value
* sort first by leaseID, then stable sort by expiration
* move some utils to be in oss and ent
* improve sort efficiency for API response
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* Add ulimit check, and tidy unit test cases to avoid needing to have all results and perfect ordering
* Make order independent check recursive
* Fix unit tests
* Try a 5s request timeout
Checking if a plugin is a builtin by comparing it's type to those in the builtin
registry allows for a custom plugin loaded with the same name to be considered a
builtin during error handling of the mounting process. This can cause the vault
installation to brick itself because it cannot be unsealed to register a new
sha256 or file path for a previously loaded custom plugin. Improve this logic
by checking the plugin catalog rather than the builtin registry.
Fixes#11687
* Disk usage checks
* Move disk free earlier
* Move logic to helpers
* Bring over test logic from the ulimit PR
* imports
* Report error
* Get unit tests working
* initial refactoring of unseal step in run
* remove waitgroup
* remove waitgroup
* backup work
* backup
* backup
* completely modularize run and move into diagnose
* add diagnose errors for incorrect number of unseal keys
* comment tests back in
* backup
* first subspan
* finished subspanning but running into error with timeouts
* remove runtime checks
* meeting updates
* remove telemetry block
* roy comment
* subspans for seal finalization and wrapping diagnose latency checks
* fix storage latency test errors
* review comments
* use random uuid for latency checks instead of static id
* Create helpers which integrate with OpenTelemetry for diagnose collection
* Go mod vendor
* Comments
* Update vault/diagnose/helpers.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Add unit test/example
* tweak output
* More comments
* add spot check concept
* Get unit tests working on Result structs
* wip
* Fix unit test
* Get unit tests working, and make diagnose sessions local rather than global
* Comments
* Last comments
* No need for init
* :|
* Fix helpers_test
* wip
* wip
* wip
* Revendor otel
* Fix merge related problems
* imports
* Fix unit tests
Co-authored-by: swayne275 <swayne275@gmail.com>
* Add infrastructure for skipping tests
* Add infrastructure for skipping tests
* Set it
* Update vault/diagnose/helpers.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Implement type alias for test functions
Co-authored-by: swayne275 <swayne275@gmail.com>
* shrink generic error message
* move zombie loading to updatePendingInternal from loadEntryInternal
* probably the right metric/lease behavior for irrevocable leases...
* comment improvements
* test total lease count with valid and irrevocable leases
* The main driver for this change was to make the read from a.newFragmentCh timeout quickly rather than waiting for the test timeout (much longer). While testing the change I observed a panic during shutdown, but it was swallowed and moreover there was no stack trace so it wasn't obvious. I'm hoping we can get rid of the recover, so I fixed the issue in the activitylog tests that needed it.
* Add an Int64 type
* Use the new Int64 type so that even 32 bit builds can specify max_operations above 2^31
* Missed a spot
* go mod vendor
* fix cast
* changelog
* Update unit test to ensure this works on both 32 and 64-bit archs
* build out zombie lease system
* add typo for CI
* undo test CI commit
* time equality test isn't working on CI, so let's see what this does...
* add unrecoverable proto error, make proto, go mod vendor
* zombify leases if unrecoverable error, tests
* test fix: somehow pointer in pointer rx is null after pointer rx called
* tweaks based on roy feedback
* improve zombie errors
* update which errors are unrecoverable
* combine zombie logic
* keep subset of zombie lease in memory
* Create helpers which integrate with OpenTelemetry for diagnose collection
* Go mod vendor
* consul tls checks
* draft for storage end to end check
* Comments
* Update vault/diagnose/helpers.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Add unit test/example
* tweak output
* More comments
* add spot check concept
* Get unit tests working on Result structs
* Fix unit test
* Get unit tests working, and make diagnose sessions local rather than global
* Comments
* Last comments
* No need for init
* :|
* Fix helpers_test
* cleaned up chan logic. Tests next.
* fix tests
* remove a comment
* tests
* remove a comment
* run direct access checks in diagnose command
* review comments
Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
* Create helpers which integrate with OpenTelemetry for diagnose collection
* Go mod vendor
* consul tls checks
* draft for storage end to end check
* Comments
* Update vault/diagnose/helpers.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Add unit test/example
* tweak output
* More comments
* add spot check concept
* Get unit tests working on Result structs
* Fix unit test
* Get unit tests working, and make diagnose sessions local rather than global
* Comments
* Last comments
* No need for init
* :|
* Fix helpers_test
* cleaned up chan logic. Tests next.
* fix tests
* remove a comment
* tests
* remove a comment
* cosmetic changes
Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
* Create helpers which integrate with OpenTelemetry for diagnose collection
* Go mod vendor
* Comments
* Update vault/diagnose/helpers.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Add unit test/example
* tweak output
* More comments
* add spot check concept
* Get unit tests working on Result structs
* Fix unit test
* Get unit tests working, and make diagnose sessions local rather than global
* Comments
* Last comments
* No need for init
* :|
* Fix helpers_test
Co-authored-by: swayne275 <swayne275@gmail.com>
* build out zombie lease system
* add typo for CI
* undo test CI commit
* time equality test isn't working on CI, so let's see what this does...
* protect against nil pointer receiver calls
* new font and add as font-family to be used in masked-input
* clean up logic
* refactor for displayOnly
* start cert masking
* work on certificates
* upload cert work
* fix global styling
* fix styling for class no longer used
* make mask by default and remove option
* glimmerize start and certificate on LDAP a file field
* glimmerize actions
* first part of glimmerizing text-file still need to do some clean up
* not doing awesome over here
* getting ready to un-glimmer
* unglimmerize
* remove placeholder based on conversations with design
* clean up text-file
* cleanup
* fix class bindings
* handle class binding
* set up for test
* fix elementId
* track down index
* update masked-input test
* add more to the masked-input test
* test-file test
* fix broken test
* clear old style
* clean up
* remove pgp key masked font, this really needs to be refactored to text-file component
* changelog
* cover other certificate view
* add allowCopy
* address some pr styling comments
* improve test coverage
* fix some issues
* add attr.options.masked
