Removing the timeout logic from raw-response functions and adding documentation comments. The following functions are affected:
- `ReadRaw`
- `ReadRawWithContext` (newly added)
- `ReadRawWithData`
- `ReadRawWithDataWithContext`
The previous logic of using `ctx, _ = c.c.withConfiguredTimeout(ctx)` could cause a potential [context leak](https://pkg.go.dev/context):
> Failing to call the CancelFunc leaks the child and its children until the parent is canceled or the timer fires. The go vet tool checks that CancelFuncs are used on all control-flow paths.
Cancelling the context would have caused more issues since the context would be cancelled before the request body is closed.
Resolves: #18658
* Stub out initial health check command
This command will be used to generate health check results for the PKI
engine.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Start common health check implementation
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add common health check utilities
These utilities will collect helpers not specific to PKI health checks,
such as formatting longer durations more legibly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add PKI health check common utils
Many health checks will need issuer and/or CRL information in order to
execute. We've centrally located these helpers to avoid particular
health checks from needing to reimplement them each time.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Adding ca_validity_period health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Begin using health-checks in PKI command
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow parsing raw requests afterwards
This shifts the last of the logic difference between Read(...) and
ReadRaw(...) to a new helper, allowing ReadRaw(...) requests to be
parsed into the same response structure afterwards as Read(...); this
allows API callers to fetch the raw secret and inspect the raw response
object in case something went wrong (error code &c) -- and when the
request succeeds, they can still get the api.Secret out.
This will be used with the PKI health check functionality, making both
LIST and READ operations use ReadRaw, and optionally parsing the secret
afterwards.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add crl_validity_period health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for PKI health check
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix bug in raw reading with contexts
When reading raw objects, don't manually call the context cancellation:
this causes timeouts and/or EOF errors when attempting to read or parse
the response body. See message in client.RawRequestWithContext(...) for
more information.
This was causing the test suite to randomly fail, due to the context
cancelling. The test suite's client usually had a default timeout,
whereas the CLI didn't, and thus didn't exhibit the same issue.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Fix typo in permissions message
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move %v->%w for errs
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Expose raw request from client.Logical()
Not all Vault API endpoints return well-formatted JSON objects.
Sometimes, in the case of the PKI secrets engine, they're not even
printable (/pki/ca returns a binary (DER-encoded) certificate). While
this endpoint isn't authenticated, in general the API caller would
either need to use Client.RawRequestWithContext(...) directly (which
the docs advise against), or setup their own net/http client and
re-create much of Client and/or Client.Logical.
Instead, exposing the raw Request (via the new ReadRawWithData(...))
allows callers to directly consume these non-JSON endpoints like they
would nearly any other endpoint.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add raw formatter for direct []byte data
As mentioned in the previous commit, some API endpoints return non-JSON
data. We get as far as fetching this data (via client.Logical().Read),
but parsing it as an api.Secret fails (as in this case, it is non-JSON).
Given that we intend to update `vault read` to support such endpoints,
we'll need a "raw" formatter that accepts []byte-encoded data and simply
writes it to the UI.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add support for reading raw API endpoints
Some endpoints, such as `pki/ca` and `pki/ca/pem` return non-JSON
objects. When calling `vault read` on these endpoints, an error
is returned because they cannot be parsed as api.Secret instances:
> Error reading pki/ca/pem: invalid character '-' in numeric literal
Indeed, we go to all the trouble of (successfully) fetching this value,
only to be unable to Unmarshal into a Secrets value. Instead, add
support for a new -format=raw option, allowing these endpoints to be
consumed by callers of `vault read` directly.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog entry
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Remove panic
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* handle HTTP PATCH requests as logical.PatchOperation
* update go.mod, go.sum
* a nil response for logical.PatchOperation should result in 404
* respond with 415 for incorrect MIME type in PATCH Content-Type header
* add abstraction to handle PatchOperation requests
* add ACLs for patch
* Adding JSON Merge support to the API client
* add HTTP PATCH tests to check high level response logic
* add permission-based 'kv patch' tests in prep to add HTTP PATCH
* adding more 'kv patch' CLI command tests
* fix TestHandler_Patch_NotFound
* Fix TestKvPatchCommand_StdinValue
* add audit log test for HTTP PATCH
* patch CLI changes
* add patch CLI tests
* change JSONMergePatch func to accept a ctx
* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag
* go fmt
* add a test to verify patching works by default with the root token
* add changelog entry
* get vault-plugin-secrets-kv@add-patch-support
* PR feedback
* reorder some imports; go fmt
* add doc comment for HandlePatchOperation
* add json-patch@v5.5.0 to go.mod
* remove unnecessary cancelFunc for WriteBytes
* remove default for -method
* use stable version of json-patch; go mod tidy
* more PR feedback
* temp go get vault-plugin-secrets-kv@master until official release
Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
We support this in the API as of 0.10.2 so read should support it too.
Trivially tested with some log info:
`core: data: data="map[string]interface {}{"zip":[]string{"zap", "zap2"}}"`
This takes place in two parts, since working on this exposed an issue
with response wrapping when there is a raw body set. The changes are (in
diff order):
* A CurrentWrappingLookupFunc has been added to return the current
value. This is necessary for the lookahead call since we don't want the
lookahead call to be wrapped.
* Support for unwrapping < 0.6.2 tokens via the API/CLI has been
removed, because we now have backends returning 404s with data and can't
rely on the 404 trick. These can still be read manually via
cubbyhole/response.
* KV preflight version request now ensures that its calls is not
wrapped, and restores any given function after.
* When responding with a raw body, instead of always base64-decoding a
string value and erroring on failure, on failure we assume that it
simply wasn't a base64-encoded value and use it as is.
* A test that fails on master and works now that ensures that raw body
responses that are wrapped and then unwrapped return the expected
values.
* A flag for response data that indicates to the wrapping handling that
the data contained therein is already JSON decoded (more later).
* RespondWithStatusCode now defaults to a string so that the value is
HMAC'd during audit. The function always JSON encodes the body, so
before now it was always returning []byte which would skip HMACing. We
don't know what's in the data, so this is a "better safe than sorry"
issue. If different behavior is needed, backends can always manually
populate the data instead of relying on the helper function.
* We now check unwrapped data after unwrapping to see if there were raw
flags. If so, we try to detect whether the value can be unbase64'd. The
reason is that if it can it was probably originally a []byte and
shouldn't be audit HMAC'd; if not, it was probably originally a string
and should be. In either case, we then set the value as the raw body and
hit the flag indicating that it's already been JSON decoded so not to
try again before auditing. Doing it this way ensures the right typing.
* There is now a check to see if the data coming from unwrapping is
already JSON decoded and if so the decoding is skipped before setting
the audit response.
* Allow returning list information and other data in 404s.
On read it'll output data and/or warnings on a 404 if they exist. On
list, the same behavior; the actual 'vault list' command doesn't change
behavior though in terms of output unless there are no actual keys (so
it doesn't just magically show other data).
This corrects some assumptions in response_util and wrapping.go; it also
corrects a few places in the latter where it could leak a (useless)
token in some error cases.
* Use same 404 logic in delete/put too
* Add the same secret parsing logic to the KV request functions
This PR fixes the error handling in the api packages logical.Unwrap
method. Previously if there was an error making the request to Vault,
the error was only returned if there was an HTTP response and the status
code was not a 404.
The new code returns all errors but does special case handling if the
response code is a 404.
backends for the moment. This is pretty simple; it just adds the actual
capability to make a list call into both the CLI and the HTTP handler.
The real meat was already in those backends.
Hamid Ghaf