Commit Graph

50 Commits

Author SHA1 Message Date
Theron Voran 6ea82f6d1a
agent: drop dockertest dep in cacheboltdb test (#11145)
ioutils --> ioutil
2021-03-18 17:46:40 -07:00
Nick Cabatoff 66e10f64bc
Fix a bug in Agent's handling of consistency headers: they should be ignored when computing cache ID. (#11099) 2021-03-12 12:42:06 -05:00
Theron Voran 1fdf08b149
agent: persistent caching support (#10938)
Adds the option of a write-through cache, backed by boltdb

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-03-03 14:01:33 -08:00
Nick Cabatoff c1ddfbb538
OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
Jason O'Donnell 5502d43f6e
agent: add caching encryption package (#10986)
* agent: add caching encryption package

* Fix documentation

* Add GetKey, GetPersistentKey

* Remove chan from interface

* Add error to interface

* Fix tests
2021-02-23 19:48:39 -05:00
Theron Voran c62ce48b5b
Set TokenParent in the Index to be cached (#10833)
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-02-03 18:30:41 -08:00
Calvin Leung Huang 0df09e356d
agent: add an inflight cache better concurrent request handling (#10705)
* agent: do not grap idLock writelock until caching entry

* agent: inflight cache using sync.Map

* agent: implement an inflight caching mechanism

* agent/lease: add lock for inflight cache to prevent simultaneous Set calls

* agent/lease: lock on a per-ID basis so unique requests can be processed independently

* agent/lease: add some concurrency tests

* test: use lease_id for uniqueness

* agent: remove env flags, add comments around locks

* agent: clean up test comment

* agent: clean up test comment

* agent: remove commented debug code

* agent/lease: word-smithing

* Update command/agent/cache/lease_cache.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* agent/lease: return the context error if the Done ch got closed

* agent/lease: fix data race in concurrency tests

* agent/lease: mockDelayProxier: return ctx.Err() if context got canceled

* agent/lease: remove unused inflightCacheLock

* agent/lease: test: bump context timeout to 3s

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-01-26 12:09:37 -08:00
Brian Kassouf 84dbca38a1
Revert "Migrate internalshared out (#9727)" (#10141)
This reverts commit ee6391b691ac12ab6ca13c3912404f1d3a842bd6.
2020-10-13 16:38:21 -07:00
Jeff Mitchell e6881c8147
Migrate internalshared out (#9727)
* Migrate internalshared out

* fix merge issue

* fix merge issue

* go mod vendor

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2020-10-12 11:56:24 -07:00
Jeff Mitchell 1d3d89e2aa
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Calvin Leung Huang 0509985408
http: ensure return after writing response by respondError (#8796) 2020-04-21 15:30:36 -07:00
Jeff Mitchell 844b2c3a5d Bump API/SDK and adapt to move from SDK stuff 2020-02-15 14:58:05 -05:00
Jeff Mitchell 86327b8010 Bump api/sdk and fix imports 2020-02-13 10:41:16 -05:00
Alex Antonov 3457d383ba
Added flag to disable X-Vault-Token header proxy if client passes the token (#8101)
* Added flag to disable X-Vault-Token header proxy if client passes the token

* Reveresed the flag value to better match the name intent

* Introduced UseAutoAuthTokenRaw for Cache to support triplicate value of true/false/force

Co-authored-by: Clint <catsby@users.noreply.github.com>
2020-01-30 09:08:42 -06:00
Jeff Mitchell 5c3649defe Sync up Agent and API's renewers. (#7733)
* Sync up Agent and API's renewers.

This introduces a new type, LifetimeWatcher, which can handle both
renewable and non-renewable secrets, modeled after the version in Agent.
It allows the user to select behavior, with the new style being the
default when calling Start(), and old style if using the legacy Renew()
call.

No tests have been modified (except for reflect issues) and no other
code has been modified to make sure the changes are backwards
compatible.

Once this is accepted I'll pull the Agent version out.

* Move compat flags to NewRenewer

* Port agent to shared lifetime watcher lib
2019-10-28 17:28:59 -07:00
Jim Kalafut 3621179560 Fix Agent handling of gzipped responses (#7470)
* Fix Agent handling of gzipped responses

Fixes #6606

* Only remove "gzip" member, if present

* Simplify to just removing Accept-Encoding altogether
2019-09-18 14:24:41 -07:00
Jeff Mitchell 09a9587523 Move renewer to internal package so tests don't barf 2019-09-09 16:09:53 -04:00
Jeff Mitchell a965378b17
Support batch tokens in renewer and caching proxy for transit auto-unseal (#7441)
Currently this uses a fork of the api.Renewer code, which we should consolidate in 1.3.
2019-09-09 12:58:28 -07:00
Christian Muehlhaeuser e6febc5839 Fixed a bunch of typos (#7146) 2019-07-18 21:10:15 -04:00
Jeff Mitchell 068b38faa0
Make flushing the cache race safe (#6828)
* Make flushing the cache race safe

* Remove noop aborts
2019-06-05 02:52:18 -04:00
Calvin Leung Huang 679f7415ea
agent/caching: support proxying request query parameters (#6772)
* agent/caching: support proxying request query parameters

* update comment

* rejig other agent log messages to output method followed by path
2019-05-22 09:21:47 -07:00
Jeff Mitchell 213b9fd1cf Update to api 1.0.1 and sdk 0.1.8 2019-04-15 14:10:07 -04:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Calvin Leung Huang c336059812 agent/cache: add comment around APIProxy.Send return case 2019-04-05 13:58:12 -07:00
ncabatoff ef1926b5e4 Agent auto auth wrapping new config checks (#6479)
* Simplify Run(): the function that was being sent over a channel doesn't
need to close over anything except latestToken, and we don't need to
create a new one each iteration.  Instead just pass the relevant items,
namely the token and sink to work on.

* Disallow the following config combinations:
1. auto_auth.method.wrap_ttl > 0 and multiple file sinks
2. auto_auth.method.wrap_ttl > 0 and single file sink with wrap_ttl > 0
3. auto_auth.method.wrap_ttl > 0 and cache.use_auto_auth_token = true

* Expose errors that occur when APIProxy is forwarding request to Vault.

* Fix merge issues.
2019-04-05 16:12:54 -04:00
Vishal Nayak f7907c2809 Agent: Listener refactoring and socket file system permissions (#6397)
* Listener refactoring and file system permissions

* added listenerutil and move some common code there

* Added test for verifying socket file permissions

* Change default port of agent to 8200

* address review feedback

* Address review feedback

* Read socket options from listener config
2019-03-14 11:53:14 -07:00
Calvin Leung Huang b7dcef399d
agent/caching: add X-Cache and Age headers (#6394)
* agent/caching: add X-Cache and Age headers, update Date header on cached resp

* Update command/agent/cache/lease_cache.go

Co-Authored-By: calvn <cleung2010@gmail.com>

* Update command/agent/cache/proxy.go

Co-Authored-By: calvn <cleung2010@gmail.com>
2019-03-12 13:21:02 -07:00
Calvin Leung Huang 86df7ee2d5
agent/caching: use locks on LeaseCache.Send (#6374)
* agent/caching: use locks on LeaseCache.Send to prevent running multiple renewer goroutines

* update comment
2019-03-07 15:50:12 -08:00
Calvin Leung Huang e6798347a7
Refactor handleCacheClear input params (#6350)
* Refactor handleCacheClear input params

* address review feedback

* fix handleCacheClear's token_accessor case
2019-03-06 18:08:19 -08:00
Calvin Leung Huang a74b0deac0
agent/caching: proxy redirect and non-json responses; update tests (#6353)
* agent/caching: proxy redirect and non-json responses; update tests

* agent/caching: do not wrap error responses as internal errors, simply proxy them back as-is

* minor refactoring of APIProxy.Send logic

* add test case to ensure error response is not wrapped
2019-03-06 17:23:20 -08:00
Vishal Nayak d8f39d54c9
Change agent's port to 8007 (#6348) 2019-03-05 12:57:17 -05:00
Vishal Nayak 59e8632d2d
Refactor respond error function (#6343) 2019-03-05 11:43:30 -05:00
Calvin Leung Huang d73216b5c3
Fix TestLeaseCache_SendCacheable (#6333) 2019-03-04 15:18:52 -08:00
Jeff Mitchell 718ae5a010 Minor syncs 2019-03-04 13:35:22 -05:00
Calvin Leung Huang c5aaf8dcb0
agent/caching: simplify orphan check; add orphan token creation tests (#6322) 2019-03-01 17:57:25 -08:00
Vishal Nayak ce42e9ea1f
Add locking around base context (#6321)
Got offline 👍 from Calvin.
2019-03-01 20:30:14 -05:00
Calvin Leung Huang 23395891d3
agent/caching: refactor ContextInfo (#6311)
* agent/caching: refactor ContextInfo

* use NewContextInfo in NewLeaseCache
2019-03-01 11:20:36 -08:00
Calvin Leung Huang 5eb50a261e agent/caching: test renaming; add api-level cache-clear test (#6305)
* test renaming; add api-level cache-clear test

* fix import ordering
2019-02-28 10:05:55 -08:00
Calvin Leung Huang c07253999c
agent/caching: enable caching of leases generated by autoauth token (#6293)
* agent/caching: enable caching of leases generated by autoauth token

* add test for auth/token/create path

* update error message log

* Some minor updates

* add sleep timer for renewal logic to process
2019-02-27 13:14:58 -08:00
Vishal Nayak ffabeda5d9
Agent Caching: Remove response updates using renewal output (#6303)
* Agent Caching: Remove response updates using renewal output

* Use renewal output channel to log the fact
2019-02-27 14:33:45 -05:00
vishalnayak f04b4d1668 Change agent's default port number to 8100 2019-02-26 10:02:12 -05:00
Calvin Leung Huang fda590f538
agent/caching: use const for cache-clear path (#6271) 2019-02-21 14:53:34 -08:00
Calvin Leung Huang 8fc9138fe4
agent/caching: remove EvictByPrefix and EvictAll (#6269) 2019-02-21 14:52:10 -08:00
Vishal Nayak 779d49bab5
AgentCache: Properly compute auto-auth token (#6264)
* Properly compute auto-auth token

* Use inmem sink to track auto-auth token

* update debug statement

* Only add inmem sink if using auto-auth token is allowed
2019-02-20 02:11:05 -05:00
Vishal Nayak 5037185d5d
Agent Cache: Some review feedback (#6257)
* Revamp agent cache client

* Update command/agent.go

Co-Authored-By: vishalnayak <vishalnayak@users.noreply.github.com>

* Agent cache auto auth token lookup case (#6258)

* agent cache auto auth token lookup case

* Use Blake2b256Hash instead of SHA256

* agent/cache: update cache-clear endpoint; use bytes.NewReader instead (#6259)

* agent/cache: update cache-clear endpoint; use bytes.NewReader instead

* agent/cache: Fix TestCache_ComputeIndexID after switching to blake2b

* agent/cache: Only parse response body if it's non-nil (#6260)

* Differently disable agent address in the API client

* Remove DisableAgent
2019-02-19 16:53:29 -05:00
Vishal Nayak 614ec67b05
address some review back (#6237) 2019-02-15 13:40:03 -05:00
Calvin Leung Huang 4e31f955cb cacememdb: add LeaseToken and TokenParent tests for GetByPrefix 2019-02-15 07:55:09 -08:00
vishalnayak cdaac4a715 Fix cachememdb test 2019-02-15 10:34:19 -05:00
Vishal Nayak feb235d5f8
Vault Agent Cache (#6220)
* vault-agent-cache: squashed 250+ commits

* Add proper token revocation validations to the tests

* Add more test cases

* Avoid leaking by not closing request/response bodies; add comments

* Fix revoke orphan use case; update tests

* Add CLI test for making request over unix socket

* agent/cache: remove namespace-related tests

* Strip-off the auto-auth token from the lookup response

* Output listener details along with configuration

* Add scheme to API address output

* leasecache: use IndexNameLease for prefix lease revocations

* Make CLI accept the fully qualified unix address

* export VAULT_AGENT_ADDR=unix://path/to/socket

* unix:/ to unix://
2019-02-14 20:10:36 -05:00