Commit graph

90 commits

Author SHA1 Message Date
Violet Hynes 92dc054bb3
VAULT-15547 Agent/proxy decoupling, take two (#20634)
* VAULT-15547 Additional tests, refactoring, for proxy split

* VAULT-15547 Additional tests, refactoring, for proxy split

* VAULT-15547 Import reorganization

* VAULT-15547 Some missed updates for PersistConfig

* VAULT-15547 address comments

* VAULT-15547 address comments
2023-05-19 13:17:48 -04:00
Violet Hynes b2468d3481
VAULT-15547 First pass at agent/proxy decoupling (#20548)
* VAULT-15547 First pass at agent/proxy decoupling

* VAULT-15547 Fix some imports

* VAULT-15547 cases instead of string.Title

* VAULT-15547 changelog

* VAULT-15547 Fix some imports

* VAULT-15547 some more dependency updates

* VAULT-15547 More dependency paths

* VAULT-15547 godocs for tests

* VAULT-15547 godocs for tests

* VAULT-15547 test package updates

* VAULT-15547 test packages

* VAULT-15547 add proxy to test packages

* VAULT-15547 gitignore

* VAULT-15547 address comments

* VAULT-15547 Some typos and small fixes
2023-05-17 09:38:34 -04:00
Violet Hynes a2f457e10c
VAULT-12940 Vault Agent uses Vault Agent specific User-Agent header when issuing requests (#19776)
* VAULT-12940 test for templating user agent

* VAULT-12940 User agent work so far

* VAULT-12940 Vault Agent uses Vault Agent specific User-Agent header when issuing requests

* VAULT-12940 Clean-up and godocs

* VAULT-12940 changelog

* VAULT-12940 Fix test checking headers

* VAULT-12940 Fix test checking headers

* VAULT-12940 Fix test checking headers

* VAULT-12940 Fix test checking headers

* VAULT-12940 copy/paste typos

* VAULT-12940 improve comments, use make(http.Header)

* VAULT-12940 small typos and clean-up
2023-04-03 14:14:47 -04:00
Hamid Ghaf 27bb03bbc0
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Francis Chuang 74c3697144
Add Oracle Cloud auth to the Vault Agent (#19260)
* Add Oracle Cloud auth to the Vault Agent

* Use ParseDurationSecond to parse credential_poll_interval

* Use os.UserHomeDir()
2023-03-15 09:08:52 -04:00
Violet Hynes 5da90d563b
VAULT-14215 Fix panic for non-TLS listeners during SIGHUP (#19483)
* VAULT-14215 Fix panic for non-TLS listeners during SIGHUP

* VAULT-14215 Changelog

* VAULT-14215 Godoc for test
2023-03-09 10:09:16 -05:00
Peter Wilson 15302d9fe2
Restore 'server' and 'agent' base loggers to use their original names (#19304) 2023-02-23 14:56:21 +00:00
Violet Hynes 0bb5cdbe75
VAULT-13056 fix leasecache usage, add test coverage (#18922)
* VAULT-13056 fix leasecache usage, add test coverage

* VAULT-13056 remove deprecated ioutil functions

* VAULT-13056 some test clean-up

* VAULT-13056 re-add environment variable thing

* VAULT-13056 add comment for clarity
2023-02-01 11:40:20 -05:00
Violet Hynes 6ec669bb07
VAULT-12564 Add new token_file auto-auth method (#18740)
* VAULT-12564 Work so far on token file auto-auth

* VAULT-12564 remove lifetime watcher struct modifications

* VAULT-12564 add other config items, and clean up

* VAULT-12564 clean-up and more tests

* VAULT-12564 clean-up

* VAULT-12564 lookup-self and some clean-up

* VAULT-12564 safer client usage

* VAULT-12564 some clean-up

* VAULT-12564 changelog

* VAULT-12564 some clean-ups

* VAULT-12564 batch token warning

* VAULT-12564 remove follow_symlink reference

* VAULT-12564 Remove redundant stat, change temp file creation

* VAULT-12564 Remove ability to delete token after auth
2023-01-24 16:09:32 -05:00
Violet Hynes e8aa9c6429
VAULT-12542 Add info encouraging users to upgrade if agent version is different to server (#18684)
* VAULT-12542 Add info encouraging users to upgrade if agent version is different to server

* VAULT-12542 Changelog

* VAULT-12542 Language update
2023-01-13 09:49:36 -05:00
Peter Wilson 5f5cad736a
VAULT-12264: Fix log rotation params which require an integer (#18666)
* integer values for some log flags
* Adjusted `log_flags` to expect `int` for max files and max bytes
* Updated `server` and `agent`
 Renamed updateConfig (and updateLogConfig)
* Added int log params to test
* Adjust config/params so we can identify when they're not present
* Removed pointer confusion
2023-01-11 20:04:57 +00:00
Peter Wilson e4685c10ef
VAULT-9883: Agent Reloadable Config (#18638)
* Update command/agent.go
* Attempt to only reload log level and certs
* Mimicked 'server' test for cert reload in 'agent'

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Left out the `c.config` tweak that meant changes to lots of lines of code within the `Run` function of Agent command. :)
2023-01-10 17:45:34 +00:00
Violet Hynes 0b15ad18a2
VAULT-12095 Support multiple config files for Vault Agent (#18403)
* VAULT-12095 Code changes for multi-config

* VAULT-12095 typo

* VAULT-12095 make vault non-nil during update

* VAULT-12095 docs

* VAULT-12095 small refactor

* VAULT-12095 typos
2023-01-03 12:50:19 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
Violet Hynes 398cf38e1e
VAULT-11510 Vault Agent can start listeners without caching (#18137)
* VAULT-11510 Vault Agent can start listeners without caching

* VAULT-11510 fix order of imports

* VAULT-11510 changelog

* VAULT-11510 typo and better switch

* VAULT-11510 update name

* VAULT-11510 New api_proxy stanza to configure API proxy

* VAULT-11510 First pass at API Proxy docs

* VAULT-11510 nav data

* VAULT-11510 typo

* VAULT-11510 docs update
2022-12-05 10:51:03 -05:00
Peter Wilson 33e6a3a87c
VAULT-9900: Log rotation for 'agent' and 'server' commands (#18031)
* Work to unify log-file for agent/server and add rotation
* Updates to rotation code, tried to centralise the log config setup
* logging + tests
* Move LogFile to ShareConfig in test
* Docs
2022-11-29 14:07:04 +00:00
Violet Hynes 3d7f9a402f
VAULT-6368 Metrics-only listener for Agent (#18101)
* VAULT-6368 Metrics-only listener for Agent

* VAULT-6368 changelog

* VAULT-6368 Update config to use string instead of bool

* VAULT-6368 Fix leftover code

* VAULT-6368 Fix changelog

* VAULT-6368 fix typo

* VAULT-6368 recommended doc update

* VAULT-6368 use != over !(==)
2022-11-25 16:00:56 -05:00
Peter Wilson 0fad0c3864
VAULT-8732: Add log-file to Vault Agent (#17841)
* Started work on adding log-file support to Agent
* Allow log file to be picked up and appended
* Use NewLogFile everywhere
* Tried to pull out the config aggregation from Agent.Run

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-11-11 10:59:16 +00:00
Jason O'Donnell 87350f927f
agent/auto-auth: add exit_on_err configurable (#17091)
* agent/auto-auth: add exit_on_err configurable

* changelog

* Update backoff function to quit

* Clarify doc

* Fix test
2022-09-15 11:00:31 -07:00
aphorise 35650c9b96
Add systemd notify support to Agent. Resolves: #7028 (#9802)
Agent systemd notify added similar to Consul. Resolves: #7028
2022-08-29 08:18:47 -04:00
Scott Miller 3bd38fd5dc
OSS portion of wrapper-v2 (#16811)
* OSS portion of wrapper-v2

* Prefetch barrier type to avoid encountering an error in the simple BarrierType() getter

* Rename the OveriddenType to WrapperType and use it for the barrier type prefetch

* Fix unit test
2022-08-23 15:37:16 -04:00
swayne275 4632a26a09
Use %q for quoted strings where appropriate (#15216)
* change '%s' to %q where single vs double quotes shouldn't matter

* replace double quotes with %q in logs and errors
2022-08-03 12:32:45 -06:00
Jason O'Donnell e3f942f51c
agent: add disable_keep_alives configurable (#16479)
agent: add disable_keep_alives config

Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
2022-07-28 12:59:49 -07:00
Jason O'Donnell dd2ced661b
agent: add disable_idle_connections configurable (#15986)
* agent: add disable_keep_alives configurable

* Add empty test

* Add website doc

* Change to disable_idle_connections

* Update tests and doc

* Add note about env

* Changelog

* Change to slice

* Remove unused disable keep alive methods

* Add invalid value test
2022-06-16 18:06:22 -04:00
Peter Wilson 2939159a4b
Added optional -log-level flag to 'operator migrate' command (#15405) 2022-05-12 15:56:25 -04:00
Chris Capurso 5f3d63a03a
clone vault agent client with headers (#15274) 2022-05-03 16:27:53 -04:00
Jason O'Donnell e9535bda2f
agent/auto-auth: Add min_backoff to set first backoff value (#15204)
* Add initial_backoff to auto-auth method

* Disable retries in client

* Fix bug

* Thread initial backoff to CT

* Add comment

* Change to min_backoff

* changelog

* remove initial references, review

* fix test

* Thread max_backoff through

* Add doc note for max_backoff/templating
2022-04-29 12:31:32 -04:00
hghaf099 9ae2a85700
Fixing excessive unix file permissions (#14791)
* Fixing excessive unix file permissions

* CL

* reduce the permission from 750 to 700
2022-04-01 12:57:38 -04:00
Tom Proctor 3668275903
Quit agent endpoint with config (#14223)
* Add agent/v1/quit endpoint
  * Closes https://github.com/hashicorp/vault/issues/11089
* Agent quit API behind config setting
* Normalise test config whitespace
* Document config option

Co-authored-by: Rémi Lapeyre <remi.lapeyre@lenstra.fr>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
2022-02-25 10:29:05 +00:00
Rémi Lapeyre 98b18ee08e
Add telemetry to Vault agent (#13675)
This patch adds a new /agent/v1/metrics that will return metrics on the
running Vault agent. Configuration is done using the same telemetry
stanza as the Vault server. For now default runtime metrics are
returned with a few additional ones specific to the agent:
  - `vault.agent.auth.failure` and `vault.agent.auth.success` to monitor
  the correct behavior of the auto auth mechanism
  - `vault.agent.proxy.success`, `vault.agent.proxy.client_error` and
  `vault.agent.proxy.error` to check the connection with the Vault server
  - `vault.agent.cache.hit` and `vault.agent.cache.miss` to monitor the
  cache

Closes https://github.com/hashicorp/vault/issues/8649

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2022-02-17 17:10:26 -08:00
Pratyoy Mukhopadhyay 85725ba3ec
OSS changes for ent pr (#13696) 2022-01-19 09:43:12 -08:00
Theron Voran bfa008d78f
agent/cache: differentiate open log messages (#13362)
Changes the error output for the second open of the persistent cache
file, to differentiate it from the c.UI.Error message for the initial
open of the cache file, just to make it easier to tell where a problem
occurred.
2021-12-08 10:32:49 -08:00
Theron Voran ae79afdd26
agent: Use an in-process listener with cache (#12762)
Uses a bufconn listener between consul-template and vault-agent when
caching is enabled and either templates or a listener is defined. This
means no listeners need to be defined in vault-agent for just
templating. Always routes consul-template through the vault-agent
cache (instead of only when persistent cache is enabled).

Uses a local transportDialer interface in config.Cache{}. 

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
2021-10-15 17:22:19 -07:00
Mayo 0bd0339c0b
cleanup unused code and fix t.Fatal usage in goroutine in testing (#11694) 2021-09-30 07:33:14 -04:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Lars Lehtonen 42759b5a5f
command: deprecate errwrap.Wrapf() (#11744) 2021-06-02 09:22:31 -04:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Nick Cabatoff 9c5f018938
Rework agent retry config, extend it to cover proxy cache as well (#11113)
Remove template_retry config section.  Add new vault.retry section which only has num_retries field; if num_retries is 0 or absent, default it to 12 for backwards compat with pre-1.7 template retrying.  Setting num_retries=-1 disables retries.

Configured retries are used for both templating and api proxy, though if template requests go through proxy (currently requires persistence enabled) we'll only configure retries for the latter to avoid duplicate retrying.  Though there is some duplicate retrying already because whenever the template server does a retry when not going through the proxy, the Vault client it uses allows for 2 behind-the-scenes retries for some 400/500 http error codes.
2021-03-18 14:14:09 -04:00
Theron Voran 1fdf08b149
agent: persistent caching support (#10938)
Adds the option of a write-through cache, backed by boltdb

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-03-03 14:01:33 -08:00
Nick Cabatoff c1ddfbb538
OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
Jim Kalafut e60cc11f33
Add configurable exponential backoff to Agent auto-auth (#10964) 2021-02-23 12:04:21 -08:00
Jason O'Donnell 458061d43b
agent: route templating server through cache (#10927)
* agent: route templating server through cache

* Remove TemplateRetry, fix unix path

* Remove mtls comment, remove redundant tls enable

* Fix test

* Refactor vault address logic

* Fix cert/key for mtls

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Reject mtls listeners

* changelog

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-02-23 09:36:11 -05:00
Vishal Nayak 2602675402
Set namespace for template server in agent (#10757)
* Set namespace for template server in agent

* cl++
2021-01-25 17:37:01 -05:00
Ricardo Cardenas 049301f70b
feat(agent): add retry configuration for vault agent (#10644)
* feat(agent): add retry configuration for vault agent

* feat(agent): add test fixtures for retry

* fix(retry): move retry stanza to top level as template_retry

* fix(retry): add retry config to ServerConfig struct

* fix(retry): point config parser to parse template_retry instead of retry

* remove netlify config (#10711)

* Fix build (#10749)

* Move the declaration to a OSS build tag file to not have it collide w… (#10750)

* Move the declaration to a OSS build tag file to not have it collide with ent declarations

* Add comment

* Remove comment to trigger ci

* Unconditionally use the root namespace when calling sys/seal-status. (#10742)

* feat(agent): add retry configuration for vault agent

* feat(agent): add test fixtures for retry

* fix(retry): move retry stanza to top level as template_retry

* fix(retry): add retry config to ServerConfig struct

* fix(retry): point config parser to parse template_retry instead of retry

Co-authored-by: Hridoy Roy <roy@hashicorp.com>
Co-authored-by: Jeff Escalante <jescalan@users.noreply.github.com>
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
Co-authored-by: Mark Gritter <mgritter@hashicorp.com>
2021-01-25 11:00:17 -08:00
Josh Black a7aac342bd
Only set the namespace if the env var isn't present (#1519) (#10556) 2020-12-14 11:40:48 -08:00
Josh Black b8ba047b35
Allow Vault Agent to run as a Windows service (#10231) 2020-11-23 14:24:32 -08:00
Brian Kassouf 84dbca38a1
Revert "Migrate internalshared out (#9727)" (#10141)
This reverts commit ee6391b691ac12ab6ca13c3912404f1d3a842bd6.
2020-10-13 16:38:21 -07:00
Jeff Mitchell e6881c8147
Migrate internalshared out (#9727)
* Migrate internalshared out

* fix merge issue

* fix merge issue

* go mod vendor

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2020-10-12 11:56:24 -07:00
Calvin Leung Huang 90a3f32771
agent: return a non-zero exit code on error (#9670)
* agent: return a non-zero exit code on error

* agent/template: always return on template server error, add case for error_on_missing_key

* agent: fix tests by updating Run params to use an errCh

* agent/template: add permission denied test case, clean up test var

* agent: use unbuffered errCh, emit fatal errors directly to the UI output

* agent: use oklog's run.Group to schedule subsystem runners (#9761)

* agent: use oklog's run.Group to schedule subsystem runners

* agent: clean up unused DoneCh, clean up agent's main Run func

* agent/template: use ts.stopped.CAS to atomically swap value

* fix tests

* fix tests

* agent/template: add timeout on TestRunServer

* agent: output error via logs and return a generic error on non-zero exit

* fix TestAgent_ExitAfterAuth

* agent/template: do not restart ct runner on new incoming token if exit_after_auth is set to true

* agent: drain ah.OutputCh after sink exits to avoid blocking on the channel

* use context.WithTimeout, expand comments around ordering of defer cancel()
2020-09-29 18:03:09 -07:00
Calvin Leung Huang 8b7a3812dc
agent: support providing certificate information in cert's config map (#9819)
* agent: support providing certificate information in cert's config map

* update TestCertEndToEnd

* remove URL reference on warning message
2020-08-25 14:26:06 -07:00