* Expose raw request from client.Logical()
Not all Vault API endpoints return well-formatted JSON objects.
Sometimes, in the case of the PKI secrets engine, they're not even
printable (/pki/ca returns a binary (DER-encoded) certificate). While
this endpoint isn't authenticated, in general the API caller would
either need to use Client.RawRequestWithContext(...) directly (which
the docs advise against), or setup their own net/http client and
re-create much of Client and/or Client.Logical.
Instead, exposing the raw Request (via the new ReadRawWithData(...))
allows callers to directly consume these non-JSON endpoints like they
would nearly any other endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add raw formatter for direct []byte data
As mentioned in the previous commit, some API endpoints return non-JSON
data. We get as far as fetching this data (via client.Logical().Read),
but parsing it as an api.Secret fails (as in this case, it is non-JSON).
Given that we intend to update `vault read` to support such endpoints,
we'll need a "raw" formatter that accepts []byte-encoded data and simply
writes it to the UI.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for reading raw API endpoints
Some endpoints, such as `pki/ca` and `pki/ca/pem` return non-JSON
objects. When calling `vault read` on these endpoints, an error
is returned because they cannot be parsed as api.Secret instances:
> Error reading pki/ca/pem: invalid character '-' in numeric literal
Indeed, we go to all the trouble of (successfully) fetching this value,
only to be unable to Unmarshal into a Secrets value. Instead, add
support for a new -format=raw option, allowing these endpoints to be
consumed by callers of `vault read` directly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove panic
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
[`govulncheck`](https://go.dev/blog/vuln) reports that the `api` package
has a call chain that includes
`golang.org/x/net/http/httpguts.HeaderValuesContainsToken`, a vulnerable
function.
* Added flag and env var which will disable client redirection
* Added changelog
* Docs fix for unsaved file, and test single request made
* Updated test for case when redirect is enabled, updated docs based on suggestions
* Update tlsutil in sdk
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update tlsutil in API
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update tlsutil to v0.1.2 for ecdsa algo naming
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add -plugin-version flag to vault auth/secrets tune
* CLI tests for auth/secrets tune
* CLI test for plugin register
* Plugin catalog listing bug where plugins of different type with the same name could be double counted
* Use constant for -plugin-version flag name
* HCP link integration
* update configure-git.yml
* more OSS stuff
* removing internal repos
* adding a nil check
* removing config test to be included in ENT only
* updating hcp-sdk-go to v0.22.0
* remove Hostname and AuthURL link config params
Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
* auth: Add Deprecation Status to auth list -detailed
* secrets: Add Deprecation Status to secrets list -detailed
* Add changelog entry for deprecation status list
* enable registering backend muxed plugins in plugin catalog
* set the sysview on the pluginconfig to allow enabling secrets/auth plugins
* store backend instances in map
* store single implementations in the instances map
cleanup instance map and ensure we don't deadlock
* fix system backend unit tests
move GetMultiplexIDFromContext to pluginutil package
fix pluginutil test
fix dbplugin ut
* return error(s) if we can't get the plugin client
update comments
* refactor/move GetMultiplexIDFromContext test
* add changelog
* remove unnecessary field on pluginClient
* add unit tests to PluginCatalog for secrets/auth plugins
* fix comment
* return pluginClient from TestRunTestPlugin
* add multiplexed backend test
* honor metadatamode value in newbackend pluginconfig
* check that connection exists on cleanup
* add automtls to secrets/auth plugins
* don't remove apiclientmeta parsing
* use formatting directive for fmt.Errorf
* fix ut: remove tls provider func
* remove tlsproviderfunc from backend plugin tests
* use env var to prevent test plugin from running as a unit test
* WIP: remove lazy loading
* move non lazy loaded backend to new package
* use version wrapper for backend plugin factory
* remove backendVersionWrapper type
* implement getBackendPluginType for plugin catalog
* handle backend plugin v4 registration
* add plugin automtls env guard
* modify plugin factory to determine the backend to use
* remove old pluginsets from v5 and log pid in plugin catalog
* add reload mechanism via context
* readd v3 and v4 to pluginset
* call cleanup from reload if non-muxed
* move v5 backend code to new package
* use context reload for for ErrPluginShutdown case
* add wrapper on v5 backend
* fix run config UTs
* fix unit tests
- use v4/v5 mapping for plugin versions
- fix test build err
- add reload method on fakePluginClient
- add multiplexed cases for integration tests
* remove comment and update AutoMTLS field in test
* remove comment
* remove errwrap and unused context
* only support metadatamode false for v5 backend plugins
* update plugin catalog errors
* use const for env variables
* rename locks and remove unused
* remove unneeded nil check
* improvements based on staticcheck recommendations
* use const for single implementation string
* use const for context key
* use info default log level
* move pid to pluginClient struct
* remove v3 and v4 from multiplexed plugin set
* return from reload when non-multiplexed
* update automtls env string
* combine getBackend and getBrokeredClient
* update comments for plugin reload, Backend return val and log
* revert Backend return type
* allow non-muxed plugins to serve v5
* move v5 code to existing sdk plugin package
* do next export sdk fields now that we have removed extra plugin pkg
* set TLSProvider in ServeMultiplex for backwards compat
* use bool to flag multiplexing support on grpc backend server
* revert userpass main.go
* refactor plugin sdk
- update comments
- make use of multiplexing boolean and single implementation ID const
* update comment and use multierr
* attempt v4 if dispense fails on getPluginTypeForUnknown
* update comments on sdk plugin backend
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* use automtls for v5 secrets/auth plugins
* add automtls env guard
* start backend without metadata mode
* use PluginClientConfig for backend's NewPluginClient param
refactor
* - fix pluginutil test
- do not expect plugin to be unloaded in UT
- fix pluginutil tests --need new env var
- use require in UT
- fix lazy load test
* add changelog
* prioritize automtls; improve comments
* user multierror; refactor pluginSet for v4 unit test
* add test cases for v4 and v5 plugin versions
* remove unnecessary call to AutoMTLSSupported
* update comment on pluginSets
* use runconfig directly in sdk newpluginclient
* use automtls without metadatamode for v5 backend plugin registration
* use multierror for plugin runconfig calls
* remove some unnecessary code
- Make the dev quick start link readily available on the client library documentation page
- Move the full code samples to the top of the dev quickstart page so that they're easily accessible.
- Update the api/readme to have a link to the dev quickstart
* Removed redundant checks for same env var in ReadEnvironment, extracted Unix domain socket logic to function, and made use of this logic in SetAddress. Adjusted unit tests to verify proper Unix domain socket handling.
* Adding case to revert from Unix domain socket dial function back to TCP
* Adding changelog file
* Only adjust DialContext if RoundTripper is an http.Transport
* Switching from read lock to normal lock
* only reset transport DialContext when setting different address type
* made ParseAddress a method on Config
* Adding additional tests to cover transitions to/from TCP to Unix
* Moved Config type method ParseAddress closer to type's other methods.
* make release note more end-user focused
* adopt review feedback to add comment about holding a lock
- If the timing is correct, a delay in the test's select might see the
doneCh signal before the renew channels signal. If that happens, the
test fails as it assumes we will receive signals across different
channels in order.
- Rework the test to make sure that we read from the renew channel if expected
and the done channel so that any errors might not be escaping from detection
on a renew.
* Add PutMetadata and PatchMetadata KV helpers
* Refactor tests, stop needlessly converting zero values to nil
* Merge fix for delete-version-after zero value bug
* Cast nils to zero value on Put
* Use 2 different structs for put/patch metadata input, clarity fixes
* Use local constants and simplify stringification
Note that this only really applies to the SSH engine, nothing else uses
this helper. Other go.mod's updated for consistency.
See also: https://github.com/hashicorp/go-secure-stdlib/pull/40
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Update AWS auth method certificates
Add tests that the `rsa2048` document can also be verified using the
`pkcs7` field for AWS auth.
Due to the use of SHA-1-based signatures for the `identity` and `pkcs7`
methods, we want to encourage moving toward using the RSA 2048 workflow,
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/verify-rsa2048.html
This doesn't require code changes for Vault necessarily, but adding in
the (many) certificates will help end users.
Also adds `rsa2048` option to API to fetch the RSA 2048 signature.
I will make a PR to update to the AWS auth docs to document the RSA 2048
flow soon after this.
* Add Read methods for KVClient
* KV write helper
* Add changelog
* Add Delete method
* Use extractVersionMetadata inside extractDataAndVersionMetadata
* Return nil, nil for v1 writes
* Add test for extracting version metadata
* Split kv client into v1 and v2-specific clients
* Add ability to set options on Put
* Add test for KV helpers
* Add custom metadata to top level and allow for getting versions as sorted slice
* Update tests
* Separate KV v1 and v2 into different files
* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity
* Move structs and godoc comments to more appropriate files
* Add more tests for extract methods
* Rework custom metadata helper to be more consistent with other helpers
* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method
* Add Patch KV helper
* Add godoc comment and use WithOption ourselves in other KVOption functions
* Clean up options-handling and resp parsing logic; add more tests
* Add constants and more patch tests
* Add Read methods for KVClient
* KV write helper
* Add changelog
* Add Delete method
* Use extractVersionMetadata inside extractDataAndVersionMetadata
* Return nil, nil for v1 writes
* Add test for extracting version metadata
* Split kv client into v1 and v2-specific clients
* Add ability to set options on Put
* Add test for KV helpers
* Add custom metadata to top level and allow for getting versions as sorted slice
* Update tests
* Separate KV v1 and v2 into different files
* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity
* Move structs and godoc comments to more appropriate files
* Add more tests for extract methods
* Rework custom metadata helper to be more consistent with other helpers
* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method
* Return early for readability and make test value name less confusing
Updated documentation to describe the behavior when supplying `VAULT_HTTP_PROXY`. Also added support for `VAULT_PROXY_ADDR` as a 'better name' for `VAULT_HTTP_PROXY`.
* Use new parseutil helper: Safe variants
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update parseutil to v0.1.5
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix additional integer overflow in command/server
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
