Commit graph

58 commits

Author SHA1 Message Date
Tom Proctor 0180ba2984
agent: tolerate partial restore failure from persistent cache (#12718)
* agent: tolerate partial restore failure from persistent cache

* Review comments: improved consistency, test robustness, comments, assertions
2021-10-08 11:30:04 +01:00
vinay-gopalan 458927c2ed
[VAULT-3157] Move mergeStates utils from Agent to api module (#12731)
* move merge and compare states to vault core

* move MergeState, CompareStates and ParseRequiredStates to api package

* fix merge state reference in API Proxy

* move mergeStates test to api package

* add changelog

* ghost commit to trigger CI

* rename CompareStates to CompareReplicationStates

* rename MergeStates and make compareStates and parseStates private methods

* improved error messaging in parseReplicationState

* export ParseReplicationState for enterprise files
2021-10-06 10:57:06 -07:00
Theron Voran ae0bda77b3
vault-agent: copy values retrieved from bolt (#12534)
Byte slices returned from Bolt are only valid during a transaction, so
this makes a copy.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2021-09-13 11:06:08 -07:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Calvin Leung Huang c1a2a939f9
agent: restart template runner on retry for unlimited retries (#11775)
* agent: restart template runner on retry for unlimited retries

* template: log error message early

* template: delegate retries back to template if param is set to true

* agent: add and use the new template config stanza

* agent: fix panic, fix existing tests

* changelog: add changelog entry

* agent: add tests for exit_on_retry_failure

* agent: properly check on agent exit cases, add separate tests for missing key vs missing secrets

* agent: add note on difference between missing key vs missing secret

* docs: add docs for template_config

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* docs: fix exit_on_retry_failure, fix Functionality section

* docs: update interaction title

* template: add internal note on behavior for persist case

* docs: update agent, template, and template-config docs

* docs: update agent docs on retry stanza

* Apply suggestions from code review

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update changelog/11775.txt

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* agent/test: rename expectExit to expectExitFromError

* agent/test: add check on early exits on the happy path

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-21 16:10:15 -07:00
Lars Lehtonen 42759b5a5f
command: deprecate errwrap.Wrapf() (#11744) 2021-06-02 09:22:31 -04:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Brian Kassouf de0253056c
Fix a few static analysis findings (#11307) 2021-04-07 16:48:40 -07:00
Theron Voran 6ea82f6d1a
agent: drop dockertest dep in cacheboltdb test (#11145)
ioutils --> ioutil
2021-03-18 17:46:40 -07:00
Nick Cabatoff 66e10f64bc
Fix a bug in Agent's handling of consistency headers: they should be ignored when computing cache ID. (#11099) 2021-03-12 12:42:06 -05:00
Theron Voran 1fdf08b149
agent: persistent caching support (#10938)
Adds the option of a write-through cache, backed by boltdb

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-03-03 14:01:33 -08:00
Nick Cabatoff c1ddfbb538
OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
Jason O'Donnell 5502d43f6e
agent: add caching encryption package (#10986)
* agent: add caching encryption package

* Fix documentation

* Add GetKey, GetPersistentKey

* Remove chan from interface

* Add error to interface

* Fix tests
2021-02-23 19:48:39 -05:00
Theron Voran c62ce48b5b
Set TokenParent in the Index to be cached (#10833)
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-02-03 18:30:41 -08:00
Calvin Leung Huang 0df09e356d
agent: add an inflight cache better concurrent request handling (#10705)
* agent: do not grap idLock writelock until caching entry

* agent: inflight cache using sync.Map

* agent: implement an inflight caching mechanism

* agent/lease: add lock for inflight cache to prevent simultaneous Set calls

* agent/lease: lock on a per-ID basis so unique requests can be processed independently

* agent/lease: add some concurrency tests

* test: use lease_id for uniqueness

* agent: remove env flags, add comments around locks

* agent: clean up test comment

* agent: clean up test comment

* agent: remove commented debug code

* agent/lease: word-smithing

* Update command/agent/cache/lease_cache.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* agent/lease: return the context error if the Done ch got closed

* agent/lease: fix data race in concurrency tests

* agent/lease: mockDelayProxier: return ctx.Err() if context got canceled

* agent/lease: remove unused inflightCacheLock

* agent/lease: test: bump context timeout to 3s

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-01-26 12:09:37 -08:00
Brian Kassouf 84dbca38a1
Revert "Migrate internalshared out (#9727)" (#10141)
This reverts commit ee6391b691ac12ab6ca13c3912404f1d3a842bd6.
2020-10-13 16:38:21 -07:00
Jeff Mitchell e6881c8147
Migrate internalshared out (#9727)
* Migrate internalshared out

* fix merge issue

* fix merge issue

* go mod vendor

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2020-10-12 11:56:24 -07:00
Jeff Mitchell 1d3d89e2aa
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Calvin Leung Huang 0509985408
http: ensure return after writing response by respondError (#8796) 2020-04-21 15:30:36 -07:00
Jeff Mitchell 844b2c3a5d Bump API/SDK and adapt to move from SDK stuff 2020-02-15 14:58:05 -05:00
Jeff Mitchell 86327b8010 Bump api/sdk and fix imports 2020-02-13 10:41:16 -05:00
Alex Antonov 3457d383ba
Added flag to disable X-Vault-Token header proxy if client passes the token (#8101)
* Added flag to disable X-Vault-Token header proxy if client passes the token

* Reveresed the flag value to better match the name intent

* Introduced UseAutoAuthTokenRaw for Cache to support triplicate value of true/false/force

Co-authored-by: Clint <catsby@users.noreply.github.com>
2020-01-30 09:08:42 -06:00
Jeff Mitchell 5c3649defe Sync up Agent and API's renewers. (#7733)
* Sync up Agent and API's renewers.

This introduces a new type, LifetimeWatcher, which can handle both
renewable and non-renewable secrets, modeled after the version in Agent.
It allows the user to select behavior, with the new style being the
default when calling Start(), and old style if using the legacy Renew()
call.

No tests have been modified (except for reflect issues) and no other
code has been modified to make sure the changes are backwards
compatible.

Once this is accepted I'll pull the Agent version out.

* Move compat flags to NewRenewer

* Port agent to shared lifetime watcher lib
2019-10-28 17:28:59 -07:00
Jim Kalafut 3621179560 Fix Agent handling of gzipped responses (#7470)
* Fix Agent handling of gzipped responses

Fixes #6606

* Only remove "gzip" member, if present

* Simplify to just removing Accept-Encoding altogether
2019-09-18 14:24:41 -07:00
Jeff Mitchell 09a9587523 Move renewer to internal package so tests don't barf 2019-09-09 16:09:53 -04:00
Jeff Mitchell a965378b17
Support batch tokens in renewer and caching proxy for transit auto-unseal (#7441)
Currently this uses a fork of the api.Renewer code, which we should consolidate in 1.3.
2019-09-09 12:58:28 -07:00
Christian Muehlhaeuser e6febc5839 Fixed a bunch of typos (#7146) 2019-07-18 21:10:15 -04:00
Jeff Mitchell 068b38faa0
Make flushing the cache race safe (#6828)
* Make flushing the cache race safe

* Remove noop aborts
2019-06-05 02:52:18 -04:00
Calvin Leung Huang 679f7415ea
agent/caching: support proxying request query parameters (#6772)
* agent/caching: support proxying request query parameters

* update comment

* rejig other agent log messages to output method followed by path
2019-05-22 09:21:47 -07:00
Jeff Mitchell 213b9fd1cf Update to api 1.0.1 and sdk 0.1.8 2019-04-15 14:10:07 -04:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Calvin Leung Huang c336059812 agent/cache: add comment around APIProxy.Send return case 2019-04-05 13:58:12 -07:00
ncabatoff ef1926b5e4 Agent auto auth wrapping new config checks (#6479)
* Simplify Run(): the function that was being sent over a channel doesn't
need to close over anything except latestToken, and we don't need to
create a new one each iteration.  Instead just pass the relevant items,
namely the token and sink to work on.

* Disallow the following config combinations:
1. auto_auth.method.wrap_ttl > 0 and multiple file sinks
2. auto_auth.method.wrap_ttl > 0 and single file sink with wrap_ttl > 0
3. auto_auth.method.wrap_ttl > 0 and cache.use_auto_auth_token = true

* Expose errors that occur when APIProxy is forwarding request to Vault.

* Fix merge issues.
2019-04-05 16:12:54 -04:00
Vishal Nayak f7907c2809 Agent: Listener refactoring and socket file system permissions (#6397)
* Listener refactoring and file system permissions

* added listenerutil and move some common code there

* Added test for verifying socket file permissions

* Change default port of agent to 8200

* address review feedback

* Address review feedback

* Read socket options from listener config
2019-03-14 11:53:14 -07:00
Calvin Leung Huang b7dcef399d
agent/caching: add X-Cache and Age headers (#6394)
* agent/caching: add X-Cache and Age headers, update Date header on cached resp

* Update command/agent/cache/lease_cache.go

Co-Authored-By: calvn <cleung2010@gmail.com>

* Update command/agent/cache/proxy.go

Co-Authored-By: calvn <cleung2010@gmail.com>
2019-03-12 13:21:02 -07:00
Calvin Leung Huang 86df7ee2d5
agent/caching: use locks on LeaseCache.Send (#6374)
* agent/caching: use locks on LeaseCache.Send to prevent running multiple renewer goroutines

* update comment
2019-03-07 15:50:12 -08:00
Calvin Leung Huang e6798347a7
Refactor handleCacheClear input params (#6350)
* Refactor handleCacheClear input params

* address review feedback

* fix handleCacheClear's token_accessor case
2019-03-06 18:08:19 -08:00
Calvin Leung Huang a74b0deac0
agent/caching: proxy redirect and non-json responses; update tests (#6353)
* agent/caching: proxy redirect and non-json responses; update tests

* agent/caching: do not wrap error responses as internal errors, simply proxy them back as-is

* minor refactoring of APIProxy.Send logic

* add test case to ensure error response is not wrapped
2019-03-06 17:23:20 -08:00
Vishal Nayak d8f39d54c9
Change agent's port to 8007 (#6348) 2019-03-05 12:57:17 -05:00
Vishal Nayak 59e8632d2d
Refactor respond error function (#6343) 2019-03-05 11:43:30 -05:00
Calvin Leung Huang d73216b5c3
Fix TestLeaseCache_SendCacheable (#6333) 2019-03-04 15:18:52 -08:00
Jeff Mitchell 718ae5a010 Minor syncs 2019-03-04 13:35:22 -05:00
Calvin Leung Huang c5aaf8dcb0
agent/caching: simplify orphan check; add orphan token creation tests (#6322) 2019-03-01 17:57:25 -08:00
Vishal Nayak ce42e9ea1f
Add locking around base context (#6321)
Got offline 👍 from Calvin.
2019-03-01 20:30:14 -05:00
Calvin Leung Huang 23395891d3
agent/caching: refactor ContextInfo (#6311)
* agent/caching: refactor ContextInfo

* use NewContextInfo in NewLeaseCache
2019-03-01 11:20:36 -08:00
Calvin Leung Huang 5eb50a261e agent/caching: test renaming; add api-level cache-clear test (#6305)
* test renaming; add api-level cache-clear test

* fix import ordering
2019-02-28 10:05:55 -08:00
Calvin Leung Huang c07253999c
agent/caching: enable caching of leases generated by autoauth token (#6293)
* agent/caching: enable caching of leases generated by autoauth token

* add test for auth/token/create path

* update error message log

* Some minor updates

* add sleep timer for renewal logic to process
2019-02-27 13:14:58 -08:00
Vishal Nayak ffabeda5d9
Agent Caching: Remove response updates using renewal output (#6303)
* Agent Caching: Remove response updates using renewal output

* Use renewal output channel to log the fact
2019-02-27 14:33:45 -05:00
vishalnayak f04b4d1668 Change agent's default port number to 8100 2019-02-26 10:02:12 -05:00