* implement SSRF protection header
* add test for SSRF protection header
* cleanup
* refactor
* implement SSRF header on a per-listener basis
* cleanup
* cleanup
* creat unit test for agent SSRF
* improve unit test for agent SSRF
* add VaultRequest SSRF header to CLI
* fix unit test
* cleanup
* improve test suite
* simplify check for Vault-Request header
* add constant for Vault-Request header
* improve test suite
* change 'config' to 'agentConfig'
* Revert "change 'config' to 'agentConfig'"
This reverts commit 14ee72d21fff8027966ee3c89dd3ac41d849206f.
* do not remove header from request
* change header name to X-Vault-Request
* simplify http.Handler logic
* cleanup
* simplify http.Handler logic
* use stdlib errors package
* vault-agent-cache: squashed 250+ commits
* Add proper token revocation validations to the tests
* Add more test cases
* Avoid leaking by not closing request/response bodies; add comments
* Fix revoke orphan use case; update tests
* Add CLI test for making request over unix socket
* agent/cache: remove namespace-related tests
* Strip-off the auto-auth token from the lookup response
* Output listener details along with configuration
* Add scheme to API address output
* leasecache: use IndexNameLease for prefix lease revocations
* Make CLI accept the fully qualified unix address
* export VAULT_AGENT_ADDR=unix://path/to/socket
* unix:/ to unix://
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
As the CLI client is initialized with a specific Vault address, is makes
sense to use the pooled HTTP client here. This will prevent setting up
new TCP sessions for each API call that the client needs to make.
* Redo API client locking
This assigns local values when in critical paths, allowing a single API
client to much more quickly and safely pipeline requests.
Additionally, in order to take that paradigm all the way it changes how
timeouts are set. It now uses a context value set on the request instead
of configuring the timeout in the http client per request, which was
also potentially quite racy.
Trivially tested with
VAULT_CLIENT_TIMEOUT=2 vault write pki/root/generate/internal key_type=rsa key_bits=8192
This takes place in two parts, since working on this exposed an issue
with response wrapping when there is a raw body set. The changes are (in
diff order):
* A CurrentWrappingLookupFunc has been added to return the current
value. This is necessary for the lookahead call since we don't want the
lookahead call to be wrapped.
* Support for unwrapping < 0.6.2 tokens via the API/CLI has been
removed, because we now have backends returning 404s with data and can't
rely on the 404 trick. These can still be read manually via
cubbyhole/response.
* KV preflight version request now ensures that its calls is not
wrapped, and restores any given function after.
* When responding with a raw body, instead of always base64-decoding a
string value and erroring on failure, on failure we assume that it
simply wasn't a base64-encoded value and use it as is.
* A test that fails on master and works now that ensures that raw body
responses that are wrapped and then unwrapped return the expected
values.
* A flag for response data that indicates to the wrapping handling that
the data contained therein is already JSON decoded (more later).
* RespondWithStatusCode now defaults to a string so that the value is
HMAC'd during audit. The function always JSON encodes the body, so
before now it was always returning []byte which would skip HMACing. We
don't know what's in the data, so this is a "better safe than sorry"
issue. If different behavior is needed, backends can always manually
populate the data instead of relying on the helper function.
* We now check unwrapped data after unwrapping to see if there were raw
flags. If so, we try to detect whether the value can be unbase64'd. The
reason is that if it can it was probably originally a []byte and
shouldn't be audit HMAC'd; if not, it was probably originally a string
and should be. In either case, we then set the value as the raw body and
hit the flag indicating that it's already been JSON decoded so not to
try again before auditing. Doing it this way ensures the right typing.
* There is now a check to see if the data coming from unwrapping is
already JSON decoded and if so the decoding is skipped before setting
the audit response.
* Redo the API client quite a bit to make the behavior of NewClient more
predictable and add locking to make it safer to use with Clone() and if
multiple goroutines for some reason decide to change things.
Along the way I discovered that currently, the x/net/http2 package is
broke with the built-in h2 support in released Go. For those using
DefaultConfig (the vast majority of cases) this will be a non-event.
Others can manually call http2.ConfigureTransport as needed. We should
keep an eye on commits on that repo and consider more updates before
release. Alternately we could go back revisions but miss out on bug
fixes; my theory is that this is not a purposeful break and I'll be
following up on this in the Go issue tracker.
In a few tests that don't use NewTestCluster, either for legacy or other
reasons, ensure that http2.ConfigureTransport is called.
* Use tls config cloning
* Don't http2.ConfigureServer anymore as current Go seems to work properly without requiring the http2 package
* Address feedback
* Set number of pester retries to zero by default and make seal command return 403 if unauthorized instead of 500
* Fix build
* Use 403 instead and update test
* Change another 500 to 403
This should help with transient issues. Full control over min/max delays
and number of retries (and ability to turn off) is provided in the API
and via env vars.
Fix tests.
This allows the same environment variables to be read, parsed, and used
from any API client as was previously handled in the CLI. The CLI now
uses the API environment variable reading capability, then overrides any
values from command line flags, if necessary.
Fixes#618
This strips out http.DefaultClient everywhere I could immediately find
it. Too many things use it and then modify it in incompatible ways.
Fixes#700, I believe.
Vault doesn't generate these, but in some cases Go's internal HTTP
handler does. For instance, during a mount-tune command, finishing the
mount path with / (as in secret/) would cause the final URL path to
contain .../mounts/secret//tune. The double slash would trigger this
behavior in Go's handler and generate a 301. Since Vault generates 307s,
this would cause the client to think that everything was okay when in
fact nothing had happened.
create our own. This avoids some potential client race conditions when
they are setting values on the Vault API client while the default client
is being used elsewhere in other goroutines, as was seen in
consul-template.
ncabatoff