* use automtls for v5 secrets/auth plugins
* add automtls env guard
* start backend without metadata mode
* use PluginClientConfig for backend's NewPluginClient param
refactor
* - fix pluginutil test
- do not expect plugin to be unloaded in UT
- fix pluginutil tests --need new env var
- use require in UT
- fix lazy load test
* add changelog
* prioritize automtls; improve comments
* user multierror; refactor pluginSet for v4 unit test
* add test cases for v4 and v5 plugin versions
* remove unnecessary call to AutoMTLSSupported
* update comment on pluginSets
* use runconfig directly in sdk newpluginclient
* use automtls without metadatamode for v5 backend plugin registration
* use multierror for plugin runconfig calls
* remove some unnecessary code
- Make the dev quick start link readily available on the client library documentation page
- Move the full code samples to the top of the dev quickstart page so that they're easily accessible.
- Update the api/readme to have a link to the dev quickstart
* Removed redundant checks for same env var in ReadEnvironment, extracted Unix domain socket logic to function, and made use of this logic in SetAddress. Adjusted unit tests to verify proper Unix domain socket handling.
* Adding case to revert from Unix domain socket dial function back to TCP
* Adding changelog file
* Only adjust DialContext if RoundTripper is an http.Transport
* Switching from read lock to normal lock
* only reset transport DialContext when setting different address type
* made ParseAddress a method on Config
* Adding additional tests to cover transitions to/from TCP to Unix
* Moved Config type method ParseAddress closer to type's other methods.
* make release note more end-user focused
* adopt review feedback to add comment about holding a lock
- If the timing is correct, a delay in the test's select might see the
doneCh signal before the renew channels signal. If that happens, the
test fails as it assumes we will receive signals across different
channels in order.
- Rework the test to make sure that we read from the renew channel if expected
and the done channel so that any errors might not be escaping from detection
on a renew.
* Add PutMetadata and PatchMetadata KV helpers
* Refactor tests, stop needlessly converting zero values to nil
* Merge fix for delete-version-after zero value bug
* Cast nils to zero value on Put
* Use 2 different structs for put/patch metadata input, clarity fixes
* Use local constants and simplify stringification
Note that this only really applies to the SSH engine, nothing else uses
this helper. Other go.mod's updated for consistency.
See also: https://github.com/hashicorp/go-secure-stdlib/pull/40
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Update AWS auth method certificates
Add tests that the `rsa2048` document can also be verified using the
`pkcs7` field for AWS auth.
Due to the use of SHA-1-based signatures for the `identity` and `pkcs7`
methods, we want to encourage moving toward using the RSA 2048 workflow,
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/verify-rsa2048.html
This doesn't require code changes for Vault necessarily, but adding in
the (many) certificates will help end users.
Also adds `rsa2048` option to API to fetch the RSA 2048 signature.
I will make a PR to update to the AWS auth docs to document the RSA 2048
flow soon after this.
* Add Read methods for KVClient
* KV write helper
* Add changelog
* Add Delete method
* Use extractVersionMetadata inside extractDataAndVersionMetadata
* Return nil, nil for v1 writes
* Add test for extracting version metadata
* Split kv client into v1 and v2-specific clients
* Add ability to set options on Put
* Add test for KV helpers
* Add custom metadata to top level and allow for getting versions as sorted slice
* Update tests
* Separate KV v1 and v2 into different files
* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity
* Move structs and godoc comments to more appropriate files
* Add more tests for extract methods
* Rework custom metadata helper to be more consistent with other helpers
* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method
* Add Patch KV helper
* Add godoc comment and use WithOption ourselves in other KVOption functions
* Clean up options-handling and resp parsing logic; add more tests
* Add constants and more patch tests
* Add Read methods for KVClient
* KV write helper
* Add changelog
* Add Delete method
* Use extractVersionMetadata inside extractDataAndVersionMetadata
* Return nil, nil for v1 writes
* Add test for extracting version metadata
* Split kv client into v1 and v2-specific clients
* Add ability to set options on Put
* Add test for KV helpers
* Add custom metadata to top level and allow for getting versions as sorted slice
* Update tests
* Separate KV v1 and v2 into different files
* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity
* Move structs and godoc comments to more appropriate files
* Add more tests for extract methods
* Rework custom metadata helper to be more consistent with other helpers
* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method
* Return early for readability and make test value name less confusing
Updated documentation to describe the behavior when supplying `VAULT_HTTP_PROXY`. Also added support for `VAULT_PROXY_ADDR` as a 'better name' for `VAULT_HTTP_PROXY`.
* Use new parseutil helper: Safe variants
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update parseutil to v0.1.5
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix additional integer overflow in command/server
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* WIP: output policy
* Outputs example policy HCL for given request
* Simplify conditional
* Add PATCH capability
* Use OpenAPI spec and regex patterns to determine if path is sudo
* Add test for isSudoPath
* Add changelog
* Fix broken CLI tests
* Add output-policy to client cloning code
* Smaller fixes from PR comments
* Clone client instead of saving and restoring custom values
* Fix test
* Address comments
* Don't unset output-policy flag on KV requests otherwise the preflight request will fail and not populate LastOutputPolicyError
* Print errors saved in buffer from preflight KV requests
* Unescape characters in request URL
* Rename methods and properties to improve readability
* Put KV-specificness at front of KV-specific error
* Simplify logic by doing more direct returns of strings and errors
* Use precompiled regexes and move OpenAPI call to tests
* Remove commented out code
* Remove legacy MFA paths
* Remove unnecessary use of client
* Move sudo paths map to plugin helper
* Remove unused error return
* Add explanatory comment
* Remove need to pass in address
* Make {name} regex less greedy
* Use method and path instead of info from retryablerequest
* Add test for IsSudoPaths, use more idiomatic naming
* Use precompiled regexes and move OpenAPI call to tests (#15170)
* Use precompiled regexes and move OpenAPI call to tests
* Remove commented out code
* Remove legacy MFA paths
* Remove unnecessary use of client
* Move sudo paths map to plugin helper
* Remove unused error return
* Add explanatory comment
* Remove need to pass in address
* Make {name} regex less greedy
* Use method and path instead of info from retryablerequest
* Add test for IsSudoPaths, use more idiomatic naming
* Make stderr writing more obvious, fix nil pointer deref
* add BuildDate to version base
* populate BuildDate with ldflags
* include BuildDate in FullVersionNumber
* add BuildDate to seal-status and associated status cmd
* extend core/versions entries to include BuildDate
* include BuildDate in version-history API and CLI
* fix version history tests
* fix sys status tests
* fix TestStatusFormat
* remove extraneous LD_FLAGS from build.sh
* add BuildDate to build.bat
* fix TestSysUnseal_Reset
* attempt to add build-date to release builds
* add branch to github build workflow
* add get-build-date to build-* job needs
* fix release build command vars
* add missing quote in release build command
* Revert "add branch to github build workflow"
This reverts commit b835699ecb7c2c632757fa5fe64b3d5f60d2a886.
* add changelog entry
* go-secure-stdlib/parseutil to v0.1.4
* add TypeCommaStringSlice for json.Number
* add changelog entry
* upgrade go-secure-stdlib/parseutil to v0.1.4 in sdk
* move json number TypeCommaStringSlice test
* go mod download for api; go mod tidy
This patch adds a new /agent/v1/metrics that will return metrics on the
running Vault agent. Configuration is done using the same telemetry
stanza as the Vault server. For now default runtime metrics are
returned with a few additional ones specific to the agent:
- `vault.agent.auth.failure` and `vault.agent.auth.success` to monitor
the correct behavior of the auto auth mechanism
- `vault.agent.proxy.success`, `vault.agent.proxy.client_error` and
`vault.agent.proxy.error` to check the connection with the Vault server
- `vault.agent.cache.hit` and `vault.agent.cache.miss` to monitor the
cache
Closes https://github.com/hashicorp/vault/issues/8649
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Login MFA
* ENT OSS segragation (#14088)
* Delete method id if not used in an MFA enforcement config (#14063)
* Delete an MFA methodID only if it is not used by an MFA enforcement config
* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path
* adding use_passcode field to DUO config (#14059)
* add changelog
* preventing replay attack on MFA passcodes (#14056)
* preventing replay attack on MFA passcodes
* using %w instead of %s for error
* Improve CLI command for login mfa (#14106)
CLI prints a warning message indicating the login request needs to get validated
* adding the validity period of a passcode to error messages (#14115)
* PR feedback
* duo to handle preventing passcode reuse
Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
* adds development workflow to mirage config
* adds mirage handler and factory for mfa workflow
* adds mfa handling to auth service and cluster adapter
* moves auth success logic from form to controller
* adds mfa form component
* shows delayed auth message for all methods
* adds new code delay to mfa form
* adds error views
* fixes merge conflict
* adds integration tests for mfa-form component
* fixes auth tests
* updates mfa response handling to align with backend
* updates mfa-form to handle multiple methods and constraints
* adds noDefault arg to Select component
* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios
* adds tests
* flaky test fix attempt
* reverts test fix attempt
* adds changelog entry
* updates comments for todo items
* removes faker from mfa mirage factory and handler
* adds number to word helper
* fixes tests
* Revert "Merge branch 'main' into ui/mfa"
This reverts commit 8ee6a6aaa1b6c9ec16b985c10d91c3806819ec40, reversing
changes made to 2428dd6cca07bb41cda3f453619646ca3a88bfd0.
* format-ttl helper fix from main
* [api] Add LDAP auth method
This commit adds LDAP to the available Vault API auth methods.
* Add changelog entry for PR 13841
* Obtain password for LDAPAuth from File/EnvVar
* Fix name of package in error message
This function call was previously used to generate mappings from
potential subjects (or SANs) to certificates within the TLS client
object. However, newer Go versions have deprecated this method, instead
building the mapping automatically based on present certificates at
request time. Because the corresponding client configuration field is
not used in Vault (NameToCertificate), it is safe to remove this call
and leave it nil.
See also: 67d894ee65
See also: https://pkg.go.dev/crypto/tls#Config.BuildNameToCertificate
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for client certificates to -output-curl-string
I did not write tests for this feature as -output-curl-string was not
already tested and this is a simple change. Because the name of the
certificates would be lost once loaded I added fields to Config to keep
track of them. I did not add a public method for the user to set them
explicitely as I don't think anyone would need this functionnality
outside of the Vault CLI.
Closes https://github.com/hashicorp/vault/issues/13376
* Add changelog
* Add lock in ConfigureTLS
* Fixed null token panic from 'v1/auth/token/' endpoints and returned proper error response
* added changelog entry for PR #13233
* changed error message from 'bad token' to 'null token'
* rebased off of main
* Revert "changed error message from 'bad token' to 'null token'"
This reverts commit 381ed9b32c5ddd5e47adb1643ef7e46fb768bc76.
* changed 'bad token' error message to 'invalid token' after revert
* remove unnecessary vault-data folder
* Add native Login method for GCP auth backend
* Add native Login method for Azure auth backend
* Add changelog entry
* Use official azure library Environment struct rather than passing string, add timeouts
* Use v1.3.0 which now has interface definition
* Don't throw away error and close resp body
* Back to WithResource so we can support non-Azure URLs for aud
* Native Login method, userpass and approle interfaces to implement it
* Add AWS auth interface for Login, unexported struct fields for now
* Add Kubernetes client login
* Add changelog
* Add a test for approle client login
* Return errors from LoginOptions, use limited reader for secret ID
* Fix auth comment length
* Return actual type not interface, check for client token in tests
* Require specification of secret ID location using SecretID struct as AppRole arg
* Allow password from env, file, or plaintext
* Add flexibility in how to fetch k8s service token, but still with default
* Avoid passing strings that need to be validated by just having different login options
* Try a couple real tests with approle and userpass login
* Fix method name in comment
* Add context to Login methods, remove comments about certain sources being inherently insecure
* Perform read of secret ID at login time
* Read password from file at login time
* Pass context in integ tests
* Read env var values in at login time, add extra tests
* Update api version
* Revert "Update api version"
This reverts commit 1ef3949497dcf878c47e0e5ffcbc8cac1c3c1679.
* Update api version in all go.mod files
- add new configuration option, ReadYourWrites, which enables a Client
to provide cluster replication states to every request. A curated set
of cluster replication states are stored in the replicationStateStore,
and is shared across clones.
* handle HTTP PATCH requests as logical.PatchOperation
* update go.mod, go.sum
* a nil response for logical.PatchOperation should result in 404
* respond with 415 for incorrect MIME type in PATCH Content-Type header
* add abstraction to handle PatchOperation requests
* add ACLs for patch
* Adding JSON Merge support to the API client
* add HTTP PATCH tests to check high level response logic
* add permission-based 'kv patch' tests in prep to add HTTP PATCH
* adding more 'kv patch' CLI command tests
* fix TestHandler_Patch_NotFound
* Fix TestKvPatchCommand_StdinValue
* add audit log test for HTTP PATCH
* patch CLI changes
* add patch CLI tests
* change JSONMergePatch func to accept a ctx
* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag
* go fmt
* add a test to verify patching works by default with the root token
* add changelog entry
* get vault-plugin-secrets-kv@add-patch-support
* PR feedback
* reorder some imports; go fmt
* add doc comment for HandlePatchOperation
* add json-patch@v5.5.0 to go.mod
* remove unnecessary cancelFunc for WriteBytes
* remove default for -method
* use stable version of json-patch; go mod tidy
* more PR feedback
* temp go get vault-plugin-secrets-kv@master until official release
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
* move merge and compare states to vault core
* move MergeState, CompareStates and ParseRequiredStates to api package
* fix merge state reference in API Proxy
* move mergeStates test to api package
* add changelog
* ghost commit to trigger CI
* rename CompareStates to CompareReplicationStates
* rename MergeStates and make compareStates and parseStates private methods
* improved error messaging in parseReplicationState
* export ParseReplicationState for enterprise files
* patch to support VAULT_HTTP_PROXY variable
* simplify the proxy replacement
* internal code review
* rename to VAULT_HTTP_PROXY, apply within ReadEnvironment
* clean up some unintended whitespace changes
* add docs for the new env variable and a changelog entry
Co-authored-by: Dave Du Cros <davidducros@gmail.com>
* Check api and sdk dirs in go_test
* Update typo in script
* Append package names if non empty
* Don't fail command if no test packages found
* Add comments, clean up echoes
* Use pushd/popd, misc review fixes
* VAULT-1303 when a request to vault fails, show namespace if set
* Adding changelog
* Fix Changelog file name
* Set namespace in ResponseWriter headers if it is set
* Using consts.NamespaceHeaderName instead of the literal string
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
* hghaf099-VAULT-1303-Adding namespace in error when it is set
* casting ResponseWriter in handleMonitor to logical.NamespaceResponseWriter
* Casting ResponseWriter conditionally for http.Flusher
Adding changelog
* Improving changlog message
* tls tests and root verification
* make the certificate verification check correct for non root CA case
* add expiry test
* addressed comments but struggling with the bug in parsing Cas and inters from single file:
* final checks on tls and listener
* cleanup
* snapshot
* basic test
* update command and add documentation
* update help text
* typo
* add changelog for lease lookup command
* run go mod vendor
* remove tabs from help output
* k8s doc: update for 0.9.1 and 0.8.0 releases (#10825)
* k8s doc: update for 0.9.1 and 0.8.0 releases
* Update website/content/docs/platform/k8s/helm/configuration.mdx
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
* Autopilot initial commit
* Move autopilot related backend implementations to its own file
* Abstract promoter creation
* Add nil check for health
* Add server state oss no-ops
* Config ext stub for oss
* Make way for non-voters
* s/health/state
* s/ReadReplica/NonVoter
* Add synopsis and description
* Remove struct tags from AutopilotConfig
* Use var for config storage path
* Handle nin-config when reading
* Enable testing autopilot by using inmem cluster
* First passing test
* Only report the server as known if it is present in raft config
* Autopilot defaults to on for all existing and new clusters
* Add locking to some functions
* Persist initial config
* Clarify the command usage doc
* Add health metric for each node
* Fix audit logging issue
* Don't set DisablePerformanceStandby to true in test
* Use node id label for health metric
* Log updates to autopilot config
* Less aggressively consume config loading failures
* Return a mutable config
* Return early from known servers if raft config is unable to be pulled
* Update metrics name
* Reduce log level for potentially noisy log
* Add knob to disable autopilot
* Don't persist if default config is in use
* Autopilot: Dead server cleanup (#10857)
* Dead server cleanup
* Initialize channel in any case
* Fix a bunch of tests
* Fix panic
* Add follower locking in heartbeat tracker
* Add LastContactFailureThreshold to config
* Add log when marking node as dead
* Update follower state locking in heartbeat tracker
* Avoid follower states being nil
* Pull test to its own file
* Add execution status to state response
* Optionally enable autopilot in some tests
* Updates
* Added API function to fetch autopilot configuration
* Add test for default autopilot configuration
* Configuration tests
* Add State API test
* Update test
* Added TestClusterOptions.PhysicalFactoryConfig
* Update locking
* Adjust locking in heartbeat tracker
* s/last_contact_failure_threshold/left_server_last_contact_threshold
* Add disabling autopilot as a core config option
* Disable autopilot in some tests
* s/left_server_last_contact_threshold/dead_server_last_contact_threshold
* Set the lastheartbeat of followers to now when setting up active node
* Don't use config defaults from CLI command
* Remove config file support
* Remove HCL test as well
* Persist only supplied config; merge supplied config with default to operate
* Use pointer to structs for storing follower information
* Test update
* Retrieve non voter status from configbucket and set it up when a node comes up
* Manage desired suffrage
* Consider bucket being created already
* Move desired suffrage to its own entry
* s/DesiredSuffrageKey/LocalNodeConfigKey
* s/witnessSuffrage/recordSuffrage
* Fix test compilation
* Handle local node config post a snapshot install
* Commit to storage first; then record suffrage in fsm
* No need of local node config being nili case, post snapshot restore
* Reconcile autopilot config when a new leader takes over duty
* Grab fsm lock when recording suffrage
* s/Suffrage/DesiredSuffrage in FollowerState
* Instantiate autopilot only in leader
* Default to old ways in more scenarios
* Make API gracefully handle 404
* Address some feedback
* Make IsDead an atomic.Value
* Simplify follower hearbeat tracking
* Use uber.atomic
* Don't have multiple causes for having autopilot disabled
* Don't remove node from follower states if we fail to remove the dead server
* Autopilot server removals map (#11019)
* Don't remove node from follower states if we fail to remove the dead server
* Use map to track dead server removals
* Use lock and map
* Use delegate lock
* Adjust when to remove entry from map
* Only hold the lock while accessing map
* Fix race
* Don't set default min_quorum
* Fix test
* Ensure follower states is not nil before starting autopilot
* Fix race
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
