Commit graph

146 commits

Author SHA1 Message Date
Jeff Mitchell 33ff878946
Move awsutil over to the go-secure-stdlib version (#12128)
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
2021-07-20 20:42:00 -04:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Jason O'Donnell b2b25be0ce
agent/template: add static_secret_render_interval configurable (#11934)
* agent/template: add default_lease_duration config

* go mod tidy

* Add changelog

* Fix panic

* Add documentation

* Change to static_secret_render_interval

* Update doc

* Update command/agent/template/template.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update changelog/11934.txt

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-06-24 15:40:31 -04:00
Calvin Leung Huang c1a2a939f9
agent: restart template runner on retry for unlimited retries (#11775)
* agent: restart template runner on retry for unlimited retries

* template: log error message early

* template: delegate retries back to template if param is set to true

* agent: add and use the new template config stanza

* agent: fix panic, fix existing tests

* changelog: add changelog entry

* agent: add tests for exit_on_retry_failure

* agent: properly check on agent exit cases, add separate tests for missing key vs missing secrets

* agent: add note on difference between missing key vs missing secret

* docs: add docs for template_config

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* docs: fix exit_on_retry_failure, fix Functionality section

* docs: update interaction title

* template: add internal note on behavior for persist case

* docs: update agent, template, and template-config docs

* docs: update agent docs on retry stanza

* Apply suggestions from code review

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update changelog/11775.txt

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* agent/test: rename expectExit to expectExitFromError

* agent/test: add check on early exits on the happy path

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-21 16:10:15 -07:00
Scott Miller ee0d6603f3
Wire configuration checks into diagnose and fix resulting bugs. (#11854)
* Actually call config.Validate in diagnose

* Wire configuration checks into diagnose and fix resulting bugs.

* go mod vendor

* Merge to vendorless version

* Remove sentinel section to allow diagnose_ok to pass

* Fix unit tests
2021-06-17 13:09:37 -05:00
Lars Lehtonen 42759b5a5f
command: deprecate errwrap.Wrapf() (#11744) 2021-06-02 09:22:31 -04:00
Jason O'Donnell 5ed63d4ce1
logical/aw: move sts signing request to awsutil (#11704) 2021-05-26 13:30:46 -04:00
Pierce Bartine e56982f782
Add ServerName to Vault Agent template config (#11288)
* Add ServerName to Vault Agent template config

* Remove newline

* Add changelog for 11288

* Update changelog/11288.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2021-05-13 07:18:15 -07:00
Calvin Leung Huang 8b7cd1bff7
agent/cert: properly return the cached client on AuthClient (#11576)
* agent/cert: properly return the cached client on AuthClient

* test: pass in nil client config, check on pointer values directly

* test: pass in nil client config

* changelog: add changelog entry
2021-05-12 14:59:07 -07:00
Tom Proctor 3ebe16d7c1
Allow Agent auto auth to read symlinked JWT files (#11502) 2021-05-06 14:11:57 +01:00
Scott Miller 7d9524be2f
Expose unknown fields and duplicate sections as diagnose warnings (#11455)
* Expose unknown fields and duplicate sections as diagnose warnings

* section counts not needed, already handled

* Address PR feedback

* Prune more of the new fields before tests call deep.Equals

* Update go.mod
2021-05-04 14:47:56 -05:00
Clint 59870ee0d3
Update Agent Auth with GCP to use new SignJWT endpoint (#11473)
* Update Agent Auth with GCP to use new SignJWT endpoint

* use iamcredentials name instead of renaming the package on import

* add changelog

* Update changelog/11473.txt

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-04-30 15:45:06 -05:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Brian Kassouf de0253056c
Fix a few static analysis findings (#11307) 2021-04-07 16:48:40 -07:00
Theron Voran 6ea82f6d1a
agent: drop dockertest dep in cacheboltdb test (#11145)
ioutils --> ioutil
2021-03-18 17:46:40 -07:00
Nick Cabatoff 9c5f018938
Rework agent retry config, extend it to cover proxy cache as well (#11113)
Remove template_retry config section.  Add new vault.retry section which only has num_retries field; if num_retries is 0 or absent, default it to 12 for backwards compat with pre-1.7 template retrying.  Setting num_retries=-1 disables retries.

Configured retries are used for both templating and api proxy, though if template requests go through proxy (currently requires persistence enabled) we'll only configure retries for the latter to avoid duplicate retrying.  Though there is some duplicate retrying already because whenever the template server does a retry when not going through the proxy, the Vault client it uses allows for 2 behind-the-scenes retries for some 400/500 http error codes.
2021-03-18 14:14:09 -04:00
Nick Cabatoff 66e10f64bc
Fix a bug in Agent's handling of consistency headers: they should be ignored when computing cache ID. (#11099) 2021-03-12 12:42:06 -05:00
Calvin Leung Huang 0a6e7ab94b
mod: update plugin versions for 1.7.0 (#11046)
* mod: update plugin versions for 1.7.0

* command/agent: fix TestCFEndToEnd test
2021-03-04 18:32:51 -08:00
Jim Kalafut 4ef8d66318
Reset agent backoff on successful auth (#11033)
The existing code would retain the previous backoff value even after the
system had recovered. This PR fixes that issue and improves the
structure of the backoff code.
2021-03-03 14:15:18 -08:00
Theron Voran 1fdf08b149
agent: persistent caching support (#10938)
Adds the option of a write-through cache, backed by boltdb

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-03-03 14:01:33 -08:00
Nick Cabatoff c1ddfbb538
OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
Jason O'Donnell 5502d43f6e
agent: add caching encryption package (#10986)
* agent: add caching encryption package

* Fix documentation

* Add GetKey, GetPersistentKey

* Remove chan from interface

* Add error to interface

* Fix tests
2021-02-23 19:48:39 -05:00
Jim Kalafut e60cc11f33
Add configurable exponential backoff to Agent auto-auth (#10964) 2021-02-23 12:04:21 -08:00
Jason O'Donnell 458061d43b
agent: route templating server through cache (#10927)
* agent: route templating server through cache

* Remove TemplateRetry, fix unix path

* Remove mtls comment, remove redundant tls enable

* Fix test

* Refactor vault address logic

* Fix cert/key for mtls

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Reject mtls listeners

* changelog

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-02-23 09:36:11 -05:00
Austin Gebauer a7531a11ea
Updates the JWT/OIDC auth plugin (#10919) 2021-02-16 17:21:35 -08:00
Jason O'Donnell ba9b3318d8
agent: allow auto-auth to use an existing token (#10850)
* agent/auto-auth: add use_existing_token

* Add better logging for lookup errors

* Fix test

* changelog

* Remove preload config, add token var

* Update filename

* Update changelog

* Revert test name

* Remove unused function

* Remove redundant error message

* Short circuit authenticate for preloaded token

* Add comment for auto-auth login
2021-02-11 09:36:03 -05:00
Theron Voran c62ce48b5b
Set TokenParent in the Index to be cached (#10833)
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-02-03 18:30:41 -08:00
Calvin Leung Huang 0df09e356d
agent: add an inflight cache better concurrent request handling (#10705)
* agent: do not grap idLock writelock until caching entry

* agent: inflight cache using sync.Map

* agent: implement an inflight caching mechanism

* agent/lease: add lock for inflight cache to prevent simultaneous Set calls

* agent/lease: lock on a per-ID basis so unique requests can be processed independently

* agent/lease: add some concurrency tests

* test: use lease_id for uniqueness

* agent: remove env flags, add comments around locks

* agent: clean up test comment

* agent: clean up test comment

* agent: remove commented debug code

* agent/lease: word-smithing

* Update command/agent/cache/lease_cache.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* agent/lease: return the context error if the Done ch got closed

* agent/lease: fix data race in concurrency tests

* agent/lease: mockDelayProxier: return ctx.Err() if context got canceled

* agent/lease: remove unused inflightCacheLock

* agent/lease: test: bump context timeout to 3s

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-01-26 12:09:37 -08:00
Ricardo Cardenas 049301f70b
feat(agent): add retry configuration for vault agent (#10644)
* feat(agent): add retry configuration for vault agent

* feat(agent): add test fixtures for retry

* fix(retry): move retry stanza to top level as template_retry

* fix(retry): add retry config to ServerConfig struct

* fix(retry): point config parser to parse template_retry instead of retry

* remove netlify config (#10711)

* Fix build (#10749)

* Move the declaration to a OSS build tag file to not have it collide w… (#10750)

* Move the declaration to a OSS build tag file to not have it collide with ent declarations

* Add comment

* Remove comment to trigger ci

* Unconditionally use the root namespace when calling sys/seal-status. (#10742)

* feat(agent): add retry configuration for vault agent

* feat(agent): add test fixtures for retry

* fix(retry): move retry stanza to top level as template_retry

* fix(retry): add retry config to ServerConfig struct

* fix(retry): point config parser to parse template_retry instead of retry

Co-authored-by: Hridoy Roy <roy@hashicorp.com>
Co-authored-by: Jeff Escalante <jescalan@users.noreply.github.com>
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
Co-authored-by: Mark Gritter <mgritter@hashicorp.com>
2021-01-25 11:00:17 -08:00
Josh Black b8ba047b35
Allow Vault Agent to run as a Windows service (#10231) 2020-11-23 14:24:32 -08:00
Brian Kassouf 84dbca38a1
Revert "Migrate internalshared out (#9727)" (#10141)
This reverts commit ee6391b691ac12ab6ca13c3912404f1d3a842bd6.
2020-10-13 16:38:21 -07:00
Jeff Mitchell e6881c8147
Migrate internalshared out (#9727)
* Migrate internalshared out

* fix merge issue

* fix merge issue

* go mod vendor

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2020-10-12 11:56:24 -07:00
Hridoy Roy 27d68e7df2
Flaky Test Fix: TestSink [VAULT-720] (#10086)
* fixed

* test sink fix

* remove print statements used for debugging

* remove print statements used for debugging

Co-authored-by: HridoyRoy <hridoyroy@Hridoys-MacBook-Pro.local>
Co-authored-by: HridoyRoy <hridoyroy@Hridoys-MBP.hitronhub.home>
2020-10-05 10:29:08 -07:00
Calvin Leung Huang 90a3f32771
agent: return a non-zero exit code on error (#9670)
* agent: return a non-zero exit code on error

* agent/template: always return on template server error, add case for error_on_missing_key

* agent: fix tests by updating Run params to use an errCh

* agent/template: add permission denied test case, clean up test var

* agent: use unbuffered errCh, emit fatal errors directly to the UI output

* agent: use oklog's run.Group to schedule subsystem runners (#9761)

* agent: use oklog's run.Group to schedule subsystem runners

* agent: clean up unused DoneCh, clean up agent's main Run func

* agent/template: use ts.stopped.CAS to atomically swap value

* fix tests

* fix tests

* agent/template: add timeout on TestRunServer

* agent: output error via logs and return a generic error on non-zero exit

* fix TestAgent_ExitAfterAuth

* agent/template: do not restart ct runner on new incoming token if exit_after_auth is set to true

* agent: drain ah.OutputCh after sink exits to avoid blocking on the channel

* use context.WithTimeout, expand comments around ordering of defer cancel()
2020-09-29 18:03:09 -07:00
Theron Voran 52581cd472
Add logging during awskms auto-unseal (#9794)
Adds debug and warn logging around AWS credential chain generation,
specifically to help users debugging auto-unseal problems on AWS, by
logging which role is being used in the case of a webidentity token.

Adds a deferred call to flush the log output as well, to ensure logs
are output in the event of an initialization failure.
2020-09-28 14:06:49 -07:00
ncabatoff b615da43d7
Run CI tests in docker instead of a machine. (#8948) 2020-09-15 10:01:26 -04:00
Jason O'Donnell 469555ef1a
agent/auth/kerberos: add disable_fast_negotiation (#9892)
* agent/auth/kerberos: add disable_fast_negotiation

* simplify test

* Update command/agent/auth/kerberos/kerberos_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* simplify tests

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-09-04 16:46:18 -04:00
Calvin Leung Huang 8b7a3812dc
agent: support providing certificate information in cert's config map (#9819)
* agent: support providing certificate information in cert's config map

* update TestCertEndToEnd

* remove URL reference on warning message
2020-08-25 14:26:06 -07:00
Scott Miller 5b003b06f8
Trail of bits 018 (#9674)
* TOB-018 remediation

* Make key derivation an optional config flag, off by default, for backwards compatibility

* Fix unit tests

* Address some feedback

* Set config on unit test

* Fix another test failure

* One more conf fail

* Switch one of the test cases to not use a derive dkey

* wip

* comments
2020-08-17 11:36:16 -05:00
ncabatoff c8833c24d8
Restart template server if it shuts down (#9200) 2020-06-15 15:25:45 -04:00
Thomas L. Kula 3ce9615992
Allow auto_auth with templates without specifying a sink (#8812)
For situations where you want the Vault agent to handle one or more templates but do not require the acquired credentials elsewhere.

Modify the logic in SyncServer so that if there are no sinks, ignore any new credentials. Since SyncServer is responsible for shutting down the agent, make sure it still properly shuts down in this new situation.

Solves #7988
2020-05-26 13:52:14 -04:00
Josh Black 6e92c8cbd2
Add a new "vault monitor" command (#8477)
Add a new "vault monitor" command

Co-authored-by: ncabatoff <ncabatoff@hashicorp.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
2020-05-21 13:07:50 -07:00
Jason O'Donnell dd254b08d5
agent/template: update consul-template dep (#9044) 2020-05-20 13:03:33 -04:00
Jeff Mitchell 1d3d89e2aa
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Calvin Leung Huang 0509985408
http: ensure return after writing response by respondError (#8796) 2020-04-21 15:30:36 -07:00
Jeff Mitchell 844b2c3a5d Bump API/SDK and adapt to move from SDK stuff 2020-02-15 14:58:05 -05:00
Clint 39f1d26902
Docs enforce autoauth token (#8270)
* rename UseAutoAuthForce to ForceAutoAuth, because I think it reads better

* Document 'ForceAuthAuthToken' option for Agent Cache

* Update website/pages/docs/agent/caching/index.mdx

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* Add additional tests around use_auto_auth=force and add documentation

* remove note, it's no longer correct

Co-authored-by: Jim Kalafut <jim@kalafut.net>
2020-02-14 15:48:12 -06:00
Jeff Mitchell 86327b8010 Bump api/sdk and fix imports 2020-02-13 10:41:16 -05:00
Becca Petrin 1459544630
update from github.com/hashicorp/gokrb5 to github.com/jcmturner/gokrb5/v8 (#8296) 2020-02-05 14:23:22 -08:00
Alex Antonov 3457d383ba
Added flag to disable X-Vault-Token header proxy if client passes the token (#8101)
* Added flag to disable X-Vault-Token header proxy if client passes the token

* Reveresed the flag value to better match the name intent

* Introduced UseAutoAuthTokenRaw for Cache to support triplicate value of true/false/force

Co-authored-by: Clint <catsby@users.noreply.github.com>
2020-01-30 09:08:42 -06:00