Commit Graph

1694 Commits

Author SHA1 Message Date
Theron Voran 4585bdda6b
Update TestServer error string (#12954)
Update the error string expected from parsing unknown units in a
duration.
2021-10-28 11:01:18 -07:00
Yannick Lorenzati e8f14b451b
CLI: add new -header option to be able to add headers to all cli requests #8754 (#12508) 2021-10-27 11:04:04 -04:00
Tom Proctor 119474e7a2
agent/cache: Store leases in-order in persistent cache so that restore respects dependencies (#12843) 2021-10-27 11:36:48 +01:00
VAL a44505dd06
Native Login method for Go client (#12796)
* Native Login method, userpass and approle interfaces to implement it

* Add AWS auth interface for Login, unexported struct fields for now

* Add Kubernetes client login

* Add changelog

* Add a test for approle client login

* Return errors from LoginOptions, use limited reader for secret ID

* Fix auth comment length

* Return actual type not interface, check for client token in tests

* Require specification of secret ID location using SecretID struct as AppRole arg

* Allow password from env, file, or plaintext

* Add flexibility in how to fetch k8s service token, but still with default

* Avoid passing strings that need to be validated by just having different login options

* Try a couple real tests with approle and userpass login

* Fix method name in comment

* Add context to Login methods, remove comments about certain sources being inherently insecure

* Perform read of secret ID at login time

* Read password from file at login time

* Pass context in integ tests

* Read env var values in at login time, add extra tests

* Update api version

* Revert "Update api version"

This reverts commit 1ef3949497dcf878c47e0e5ffcbc8cac1c3c1679.

* Update api version in all go.mod files
2021-10-26 16:48:48 -07:00
Chris Capurso a6b1cbad12
Extend kv metadata to get, put, and patch (#12907)
* go get vault-plugin-secrets-kv@extend-kv-metadata-to-get-and-put

* test for custom_metadata in kv get, put, patch command output

* remove flagFormat-specific check from TestKVMetadataGetCommand

* rewrite custom metadata changelog entry

* go get vault-plugin-secrets-kv@master

* go mod tidy
2021-10-26 15:38:56 -04:00
swayne275 559816254b
API Lock CLI OSS port (#12925)
* api lock in oss

* add namespace lock/unlock help
2021-10-26 10:52:29 -06:00
hghaf099 c79cd1c744
fixing a bug for cli when namespace is in both arg and path (#12911)
* fixing a bug for cli when namespace is in both arg and path

* Add a changelog
2021-10-22 17:47:16 -04:00
swayne275 fe9da20d67
oss components of vault-3372 (#12898) 2021-10-22 14:22:49 -06:00
Nick Cabatoff de0a56bd44
Fix TestParseSockaddrTemplate to take into account CustomResponseHeaders. (#12902) 2021-10-22 08:50:12 -04:00
hghaf099 b472d7ed3f
CLI request when namespace is in argument and part of the path (#12720)
* CLI makes request to incorrect URL when namespace is both provided as argument and part of the path
fixes #12675

* adding change log

* removing a switch and addressing a possibility of out of bound index
2021-10-21 22:35:13 -04:00
Nick Cabatoff d66fd98d4a
Add support for go-sockaddr templated addresses in config. (#9109) 2021-10-21 10:10:48 -04:00
Scott Miller 9f62768cc7
Diagnose partial/missing telemetry configuration (#12802)
* Diagnose partial/missing telemetry configuration

* changelog

* fixup

* not sure which component?
2021-10-20 16:47:59 -05:00
Dave Du Cros ceac6e913d
operator generate-root -decode: allow token from stdin (#12881)
* operator generate-root -decode: allow token from stdin

Allow passing "-" as the value for -decode, causing the encoded token to
be read from stdin. This is intended to prevent leaking the encoded
token + otp into process logs in enterprise environments.

* add changelog entry for PR12881

* add check/test for empty decode value passed via stdin
2021-10-20 12:29:17 -04:00
Chris Capurso eb6df00992
add retry logic when kv is upgrading in handler test (#12864)
* add retry logic when kv is upgrading in handler test

* make retry func for kv cli test more generic

* use ticker for kv retry logic in tests
2021-10-20 08:44:56 -04:00
Brian Kassouf c705adc79c
Fix some linting errors (#12860) 2021-10-18 17:29:47 -07:00
Theron Voran ae79afdd26
agent: Use an in-process listener with cache (#12762)
Uses a bufconn listener between consul-template and vault-agent when
caching is enabled and either templates or a listener is defined. This
means no listeners need to be defined in vault-agent for just
templating. Always routes consul-template through the vault-agent
cache (instead of only when persistent cache is enabled).

Uses a local transportDialer interface in config.Cache{}. 

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Ben Ash <32777270+benashz@users.noreply.github.com>
2021-10-15 17:22:19 -07:00
Hridoy Roy 1c427d3286
Port: add client ID to TWEs in activity log [vault-3136] (#12820)
* port for tracking twes as clients

* comment clean up

* changelog

* change changelog entry phrasing
2021-10-14 09:10:59 -07:00
Chris Capurso bbb4ab4a41
Add HTTP PATCH support to KV (#12687)
* handle HTTP PATCH requests as logical.PatchOperation

* update go.mod, go.sum

* a nil response for logical.PatchOperation should result in 404

* respond with 415 for incorrect MIME type in PATCH Content-Type header

* add abstraction to handle PatchOperation requests

* add ACLs for patch

* Adding JSON Merge support to the API client

* add HTTP PATCH tests to check high level response logic

* add permission-based 'kv patch' tests in prep to add HTTP PATCH

* adding more 'kv patch' CLI command tests

* fix TestHandler_Patch_NotFound

* Fix TestKvPatchCommand_StdinValue

* add audit log test for HTTP PATCH

* patch CLI changes

* add patch CLI tests

* change JSONMergePatch func to accept a ctx

* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag

* go fmt

* add a test to verify patching works by default with the root token

* add changelog entry

* get vault-plugin-secrets-kv@add-patch-support

* PR feedback

* reorder some imports; go fmt

* add doc comment for HandlePatchOperation

* add json-patch@v5.5.0 to go.mod

* remove unnecessary cancelFunc for WriteBytes

* remove default for -method

* use stable version of json-patch; go mod tidy

* more PR feedback

* temp go get vault-plugin-secrets-kv@master until official release

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
2021-10-13 15:24:31 -04:00
hghaf099 ad2ef412cc
Customizing HTTP headers in the config file (#12485)
* Customizing HTTP headers in the config file

* Add changelog, fix bad imports

* fixing some bugs

* fixing interaction of custom headers and /ui

* Defining a member in core to set custom response headers

* missing additional file

* Some refactoring

* Adding automated tests for the feature

* Changing some error messages based on some recommendations

* Incorporating custom response headers struct into the request context

* removing some unused references

* fixing a test

* changing some error messages, removing a default header value from /ui

* fixing a test

* wrapping ResponseWriter to set the custom headers

* adding a new test

* some cleanup

* removing some extra lines

* Addressing comments

* fixing some agent tests

* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before

* Removing default custom headers, and renaming some function varibles

* some refacotring

* Refactoring and addressing comments

* removing a function and fixing comments
2021-10-13 11:06:33 -04:00
contrun e292bc1b5d
remove trailing comma in json file (#12640)
Signed-off-by: YI <uuuuuu@protonmail.com>
2021-10-12 12:03:01 -04:00
Tom Proctor 0180ba2984
agent: tolerate partial restore failure from persistent cache (#12718)
* agent: tolerate partial restore failure from persistent cache

* Review comments: improved consistency, test robustness, comments, assertions
2021-10-08 11:30:04 +01:00
vinay-gopalan 458927c2ed
[VAULT-3157] Move `mergeStates` utils from Agent to api module (#12731)
* move merge and compare states to vault core

* move MergeState, CompareStates and ParseRequiredStates to api package

* fix merge state reference in API Proxy

* move mergeStates test to api package

* add changelog

* ghost commit to trigger CI

* rename CompareStates to CompareReplicationStates

* rename MergeStates and make compareStates and parseStates private methods

* improved error messaging in parseReplicationState

* export ParseReplicationState for enterprise files
2021-10-06 10:57:06 -07:00
Mayo 0bd0339c0b
cleanup unused code and fix t.Fatal usage in goroutine in testing (#11694) 2021-09-30 07:33:14 -04:00
hghaf099 8b29493e76
fix writing back the Filename into unusedKeys in ParseConfig (#12615) 2021-09-23 09:30:44 -04:00
Rachel Culpepper 9ff3fd39a2
fix build tag (#12588) 2021-09-20 17:10:54 -04:00
Pratyoy Mukhopadhyay 0819eac6a8
Update token renew docs (#12572)
* Update docs for token renew api and cli

* Clarify api docs for renew/renew-self

* Update wording around periodic tokens
2021-09-16 16:54:46 -07:00
Theron Voran ae0bda77b3
vault-agent: copy values retrieved from bolt (#12534)
Byte slices returned from Bolt are only valid during a transaction, so
this makes a copy.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2021-09-13 11:06:08 -07:00
Rachel Culpepper c0ada31a99
Vault-2840 Add no-op check for HTTP listener validity (#12421)
* fips tag and no-op code

* tls check

* change to error

* add tests

* add test file

* cleanup

* add changelog

* fix file difference

* remove changelog
2021-09-07 18:36:58 -04:00
Lars Lehtonen fe1cf762fd
command: fix dropped test error (#12474) 2021-09-01 16:05:14 -07:00
Chris Capurso 3f4a381f1b
Add kv custom key metadata (#12218)
* add custom-metdata flag to "kv metadata put" command

* add kv metadata put command test for custom-metadata flag

* add custom_metadata to kv-v2 api docs

* add custom_metadata to kv-v2 cli docs

* update go.mod

* Add custom metadata limits to docs

* add changelog entry

* update vault-plugin-secrets-kv to @master
2021-08-23 15:49:09 -04:00
hghaf099 f885d97774
VAULT-2285 adding capability to accept comma separated entries for au… (#12126)
* VAULT-2285 adding capability to accept comma separated entries for auth enable/tune

* Adding changelog

* Adding logic to detect invalid input parameter for auth enable config

* Updating tune.mdx

* Updating secret enable/tune for comma separated parameters

* Adding further parameter checks for auth/secret tests
Fixing changelog
using builtin type for a switch statement
Fixing a possible panic scenario

* Changing a function name, using deep.Equal instead of what reflect package provides

* Fixing auth/secret enable/tune mdx files

* One more mdx file fix

* Only when users provide a single comma separated string in a curl command, split the entries by commas

* Fixing API docs for auth/mount enable/tune for comma separated entries

* updating docs, removing an unnecessary switch case
2021-08-09 15:37:03 -04:00
Hridoy Roy a3fefdca35
oss part of license diagnose test fix (#12234)
* oss part of license diagnose test fix

* cl
2021-08-02 10:50:49 -07:00
Nick Cabatoff 6016e86115
Fix `vault debug` so that captured logs include newlines. (#12175) 2021-07-27 09:15:24 -04:00
Jeff Mitchell 33ff878946
Move awsutil over to the go-secure-stdlib version (#12128)
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
2021-07-20 20:42:00 -04:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Scott Miller 555eb2ae0b
Add advice as a trace option to spot checks (#12105)
* Add advice as a trace option to spot checks

* typo

* Collect advice when forming the tree
2021-07-15 18:01:51 -05:00
Scott Miller 3f01d92d3b
Suppress logging during a diagnose run (#12101)
* Suppress logging during a diagnose run

* remove debugging
2021-07-15 16:54:59 -05:00
Scott Miller 1aa98e81f7
Make any section skippable via string contains skip checking. (#12055)
* Make any section skippable via string contains skip checking

* Fix unit test
2021-07-13 18:25:04 -05:00
Scott Miller 5ec576a810
Wire up remaining sections for config validation (#12048)
* wip

* Add validation to storage, serviceregistration

* Wire up remaining sections except Storage which is generic
2021-07-12 16:33:17 -05:00
Hridoy Roy 3c3555d6b4
skip seal check for manual sealing (#12046)
* skip seal check for manual sealing

* language
2021-07-12 13:22:56 -07:00
Hridoy Roy d3034f2851
Diagnose Language Pass (#11909)
* save

* save

* save

* first round of the diagnose language pass

* capitalization

* first round of feedback

* fix bug in advise

* a few more nouns to verbs
2021-07-11 15:44:19 -07:00
Pratyoy Mukhopadhyay 9b5e89bd34
[VAULT-2776] Add prefix_filter option to Vault (#12025)
* [VAULT-2776] Add prefix_filter support to vault

* [VAULT-2776] Add filter_default config, update docs

* [VAULT-2776] Add changelog file

* [VAULT-2776] Update telemetry tests and error handling

* [VAULT-2776] Add test fixtures, update test

* [VAULT-2776] Update gitignore hcl filter
2021-07-09 14:49:53 -05:00
Nick Cabatoff 518944c599
Make the `list` and `kv list` commands work with wrapping, e.g. for controlgroups (#12031) 2021-07-09 12:08:58 -04:00
hghaf099 fedc31e905
Diagnose warns if HTTPS is not used for ha-storage-tls-consul [VAULT-2773] (#11967)
* Diagnose warns if HTTPS is not used for ha-storage-tls-consul

* Skipping TLS verification if https is not used in ha storage tls consul

* Adding diagnose skip message for consul service registration
2021-07-07 14:35:25 -04:00
Nick Cabatoff a2dcb131ee
`vault delete` should allow the same output options as `vault write`,… (#11992)
* `vault delete` and `vault kv delete` should allow the same output options as `vault write`, as delete operations can similarly return data.  This is needed if you want to use control groups with deletion.
2021-07-06 10:36:07 -04:00
Hridoy Roy f3895f7cce
oss part of license diagnose (#11939) 2021-06-25 14:18:34 -07:00
hghaf099 6d7dbc85b5
diagnose: Add seal transit tls check (#11922)
* diagnose: Add seal transit tls check

* Fixing the path to the config file and the path to the cert files

* Addressing comment

* Addressing seal transit tls check comments
2021-06-24 18:30:42 -04:00
Jason O'Donnell b2b25be0ce
agent/template: add static_secret_render_interval configurable (#11934)
* agent/template: add default_lease_duration config

* go mod tidy

* Add changelog

* Fix panic

* Add documentation

* Change to static_secret_render_interval

* Update doc

* Update command/agent/template/template.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update changelog/11934.txt

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-06-24 15:40:31 -04:00
Hridoy Roy a8cc5e4557
TLS Verification Bugfixes (#11910)
* tls verification bugfix

* tls verification bugfix

* allow diagnose fail to report status when there are also warnings

* allow diagnose fail to report status when there are also warnings

* Update vault/diagnose/helpers_test.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* comments

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-06-24 10:43:49 -07:00
Hridoy Roy e2614979f7
Diagnose Storage Panic Bugfixes (#11923)
* partial

* fix raft panics and ensure checks are skipped if storage isnt initialized

* cleanup directories

* newline

* typo in nil check

* another nil check
2021-06-24 09:56:38 -07:00
hghaf099 4495b932d8
Fix diagnose panic when configuration file does not exist (#11932)
* Fix diagnose panic when configuration file does not exist

* Addressing comments

* Update command/operator_diagnose.go

Co-authored-by: Hridoy Roy <roy@hashicorp.com>

Co-authored-by: Hridoy Roy <roy@hashicorp.com>
2021-06-23 18:35:52 -04:00
Calvin Leung Huang c1a2a939f9
agent: restart template runner on retry for unlimited retries (#11775)
* agent: restart template runner on retry for unlimited retries

* template: log error message early

* template: delegate retries back to template if param is set to true

* agent: add and use the new template config stanza

* agent: fix panic, fix existing tests

* changelog: add changelog entry

* agent: add tests for exit_on_retry_failure

* agent: properly check on agent exit cases, add separate tests for missing key vs missing secrets

* agent: add note on difference between missing key vs missing secret

* docs: add docs for template_config

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* docs: fix exit_on_retry_failure, fix Functionality section

* docs: update interaction title

* template: add internal note on behavior for persist case

* docs: update agent, template, and template-config docs

* docs: update agent docs on retry stanza

* Apply suggestions from code review

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update changelog/11775.txt

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* agent/test: rename expectExit to expectExitFromError

* agent/test: add check on early exits on the happy path

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-21 16:10:15 -07:00
hghaf099 38b753046b
Tls check client certs [VAULT-2117] (#11860)
* add expiration warning to certificate checking for diagnose

* Adding TLS Client CA checks to diagnose
Adding checks for tls_disable_client_certs and tls_require_and_verify_client_cert flags

* Check validity of TLSClientCAFile
Adding related tests

* Addressing comments

* Fixing some tls tests

* Addressing comments

Co-authored-by: HridoyRoy <roy@hashicorp.com>
2021-06-18 19:35:38 -04:00
Brian Kassouf 9dbdc4050d
Diagnose: Skip test-access-storage on raft storage (#11889) 2021-06-17 12:15:19 -07:00
Scott Miller ee0d6603f3
Wire configuration checks into diagnose and fix resulting bugs. (#11854)
* Actually call config.Validate in diagnose

* Wire configuration checks into diagnose and fix resulting bugs.

* go mod vendor

* Merge to vendorless version

* Remove sentinel section to allow diagnose_ok to pass

* Fix unit tests
2021-06-17 13:09:37 -05:00
Hridoy Roy e38f991054
Diagnose checks for raft quorum status and file backend permissions (#11771)
* raft file and quorum checks

* raft checks

* backup

* raft file checks test

* address comments and add more raft and file and process checks

* syntax issues

* modularize functions to compile differently on different os

* compile raft checks everywhere

* more build tag issues

* raft-diagnose

* correct file permission checks

* upgrade tests and add a getConfigOffline test that currently does not work

* comment

* update file checks method signature on windows

* Update physical/raft/raft_test.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* raft tests

* add todo comment for windows root ownership

* voter count message

* raft checks test fixes

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-17 10:04:21 -07:00
Hridoy Roy e58c39033b
Add expiration warning to certificate checking for diagnose [VAULT-1018] (#11850)
* add expiration warning to certificate checking for diagnose

* Update serviceregistration/consul/consul_service_registration.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* review comments

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-06-15 09:53:29 -07:00
Josh Black 3189526cc4
OSS parts of sighup license reload test (#11816) 2021-06-11 09:08:19 -07:00
Hridoy Roy 65e3489c45
Diagnose resource creation checks (#11627)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* merge main branch

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* backup while I fix something else

* fix storage latency test errors

* runtime checks

* diagnose with timeout on seal
2021-06-10 12:29:32 -07:00
Shanker Naik 35593978ed
fix typo in vault operator init cmd (#11792) 2021-06-10 10:39:53 -04:00
Nick Cabatoff bfae4e610b
Pull out license commands, and make the OSS changes needed for the license inspect PR in ent. (#11783) 2021-06-07 14:44:20 -04:00
Scott Miller 69d0242db9
Add Advice and Advise functions for adding an advice section to any span. (#11760)
* wip

* wip

* Finish implementing advice handling and word wrapping

* Properly word wrap messages and warnings

* Remove debugging

* Remove debugging

* Remove unnecessary test

* unit test bug

* go vendor
2021-06-07 11:29:36 -05:00
Josh Black c8cfcd9514
OSS parts of sighup license reload (#11767) 2021-06-04 10:24:35 -07:00
Scott Miller 5e55f1b3fe
Return different exit codes for different diagnose results (#11758)
This allows operators to run diagnose in scripts and detect the difference between success, warning, and failure.

Exit codes are now:
0: Success (no warnings)
1: Failure (some test failed)
2: Warning (some test warned)
3: User input failure such as a bad flag
4: Other error
2021-06-04 11:52:49 -05:00
Hridoy Roy 991f9eba56
remove operating system stanza from tests on account of disk usage being different across ci for different runs (#11764) 2021-06-03 14:11:31 -07:00
Scott Miller 9db384e0f7
Edge cases in the skipping gathering and output logic (#11752)
* Edge cases in the skipping gathering and output logic

* Fix unit test
2021-06-03 11:01:14 -05:00
Scott Miller dc0774731d
Remove duplicate children test (#11751) 2021-06-02 15:10:43 -05:00
Scott Miller d151b0b55b
Segment out disk checks to disable on openbsd/arm (#11749)
* Segment out disk checks to disable on openbsd/arm

Also add a spot skipped helper.

* Expected results may be fewer than actual because of variable length tests like disk usage

* Move to os_common and build on windows
2021-06-02 12:17:52 -05:00
Lars Lehtonen 42759b5a5f
command: deprecate errwrap.Wrapf() (#11744) 2021-06-02 09:22:31 -04:00
Scott Miller 59bea5f6fd
Add ulimit check, and tidy unit test cases (#11678)
* Add ulimit check, and tidy unit test cases to avoid needing to have all results and perfect ordering

* Make order independent check recursive

* Fix unit tests

* Try a 5s request timeout
2021-06-01 12:43:51 -05:00
Nick Cabatoff 62ba5c39ca
Restore server license config lost in #11573 (#11730) 2021-06-01 08:42:30 -04:00
Nick Cabatoff 56c0a67a53
OSS parts of #1903: we'd disabled these tests on enterprise temporarily, but now we're ready to make them run there again. (#11729)
Vault cares about the VAULT_LICENSE env var, but we don't want to set that in CI because it would change behaviour of tests that don't intend for it to be set.  Instead, we use VAULT_LICENSE_CI so that only packages/tests that opt-in will use it.
2021-05-31 12:08:10 -04:00
Scott Miller 46da8ebae6
Diagnose Disk usage checks (#11673)
* Disk usage checks

* Move disk free earlier

* Move logic to helpers

* Bring over test logic from the ulimit PR

* imports

* Report error

* Get unit tests working
2021-05-27 10:17:52 -07:00
Jason O'Donnell 5ed63d4ce1
logical/aw: move sts signing request to awsutil (#11704) 2021-05-26 13:30:46 -04:00
Hridoy Roy b6e1d62d33
Modularize Run Command (#11573)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* fix storage latency test errors

* review comments

* use random uuid for latency checks instead of static id
2021-05-25 15:23:20 -07:00
Scott Miller 941d01eee3
Initial Diagnose CLI output (#11583)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* wip

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* wip

* wip

* wip

* Revendor otel

* Fix merge related problems

* imports

* Fix unit tests

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-21 19:21:11 -07:00
Nick Cabatoff e212ec5d8e
OSS parts of ent PR #1857: license autoloading init changes. (#11623) 2021-05-17 14:10:26 -04:00
Pierce Bartine e56982f782
Add ServerName to Vault Agent template config (#11288)
* Add ServerName to Vault Agent template config

* Remove newline

* Add changelog for 11288

* Update changelog/11288.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2021-05-13 07:18:15 -07:00
Calvin Leung Huang 8b7cd1bff7
agent/cert: properly return the cached client on AuthClient (#11576)
* agent/cert: properly return the cached client on AuthClient

* test: pass in nil client config, check on pointer values directly

* test: pass in nil client config

* changelog: add changelog entry
2021-05-12 14:59:07 -07:00
Josh Black 641a81b74b
Provide a new API endpoint for retrieving signed licenses (#11543) 2021-05-12 12:19:25 -07:00
Scott Miller 9dbf1a7dba
Add infrastructure and helpers for skipping diagnose checks (#11593)
* Add infrastructure for skipping tests

* Add infrastructure for skipping tests

* Set it

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Implement type alias for test functions

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-12 12:54:40 -05:00
Scott Miller 4b87b7257e
Move config validation code to it's own, non ENT specific file (#11579)
* Move config validation code to it's own, non ENT specific file

* Fix imports

* import order
2021-05-11 11:59:08 -05:00
Nick Cabatoff 53c7d1de7d
config for autoloading license (oss parts) 2021-05-07 08:55:41 -04:00
Tom Proctor 3ebe16d7c1
Allow Agent auto auth to read symlinked JWT files (#11502) 2021-05-06 14:11:57 +01:00
Josh Black ab1710a9ea
Introduce a "vault license get" command (#11526) 2021-05-04 12:55:48 -07:00
Scott Miller 7d9524be2f
Expose unknown fields and duplicate sections as diagnose warnings (#11455)
* Expose unknown fields and duplicate sections as diagnose warnings

* section counts not needed, already handled

* Address PR feedback

* Prune more of the new fields before tests call deep.Equals

* Update go.mod
2021-05-04 14:47:56 -05:00
Scott Miller 52930c5614
When running under systemd, send notifications about server startup, shutdown, and config reload (#11517) 2021-05-04 14:47:16 -05:00
Hridoy Roy 28aa9b9175
Diagnoses Consul Direct Access Check (#11505)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* run direct access checks in diagnose command

* review comments

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 16:21:06 -07:00
Hridoy Roy 5b63066c2c
Diagnose V0: Storage End to End Checks (#11468)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* cosmetic changes

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 13:33:13 -07:00
Clint 59870ee0d3
Update Agent Auth with GCP to use new SignJWT endpoint (#11473)
* Update Agent Auth with GCP to use new SignJWT endpoint

* use iamcredentials name instead of renaming the package on import

* add changelog

* Update changelog/11473.txt

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-04-30 15:45:06 -05:00
Scott Miller 85fbd45e1c
Create helpers which integrate with OpenTelemetry for diagnose collection (#11454)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-04-29 13:32:41 -05:00
Hridoy Roy 36ae22c07a
Consul TLS Checks for Diagnose [draft] (#11467)
* consul tls checks

* fix some tests

* complete physical and service registration tls checks
2021-04-28 08:55:18 -07:00
Nick Cabatoff a21e7139ff
Update tests to reflect new HC public pgp keys. (#11446) 2021-04-23 12:32:46 -04:00
Josh Black 06809930a3
Add HTTP response headers for hostname and raft node ID (if applicable) (#11289) 2021-04-20 15:25:04 -07:00
Nick Cabatoff a8023e0fdb
Add support for unauthenticated pprof access on a per-listener basis,… (#11324)
* Add support for unauthenticated pprof access on a per-listener basis, as we do for metrics.

* Add missing pprof sub-targets like 'allocs' and 'block'.  Capture the goroutine subtarget a second time in text form.  This is mostly a convenience, but also I think the pprof format might be a bit lossy?
2021-04-19 14:30:59 -04:00
Nick Cabatoff 684ebf0928
Don't cut off stack traces at 32MB. (#11364) 2021-04-16 15:55:05 -04:00
Brian Kassouf 49489da596
Fix flakey TestAgent_Template_Retry test (#11332) 2021-04-09 12:11:01 -07:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Brian Kassouf de0253056c
Fix a few static analysis findings (#11307) 2021-04-07 16:48:40 -07:00
Hridoy Roy 049f2513e6
Initial Diagnose Command for TLS and Listener [VAULT-1896, VAULT-1899] (#11249)
* sanity checks for tls config in diagnose

* backup

* backup

* backup

* added necessary tests

* remove comment

* remove parallels causing test flakiness

* comments

* small fix

* separate out config hcl test case into new hcl file

* newline

* addressed comments

* addressed comments

* addressed comments

* addressed comments

* addressed comments

* reload funcs should be allowed to be nil
2021-04-06 16:40:43 -07:00
Hridoy Roy 4c569d0fc6
Diagnose Stub Command Tests (#11180)
* a few tests to the operator diagnose stub command

* a few tests to the operator diagnose stub command

* a few tests to the operator diagnose stub command

* empty commit to fix circle ci permissions issue

* empty commit to fix circle ci permissions issue
2021-03-29 14:22:32 -07:00
Nick Cabatoff 72a172bce9
Add support for tls_max_version in listener config. (#11226) 2021-03-29 14:39:14 -04:00
Nick Cabatoff 62c1f25140
Use a 15s instead of 10s timeout on this test. The `default` subtest takes 9s on my laptop, so pad that for CI. (#11221) 2021-03-29 09:51:19 -04:00
Theron Voran 6ea82f6d1a
agent: drop dockertest dep in cacheboltdb test (#11145)
ioutils --> ioutil
2021-03-18 17:46:40 -07:00
Nick Cabatoff 9c5f018938
Rework agent retry config, extend it to cover proxy cache as well (#11113)
Remove template_retry config section.  Add new vault.retry section which only has num_retries field; if num_retries is 0 or absent, default it to 12 for backwards compat with pre-1.7 template retrying.  Setting num_retries=-1 disables retries.

Configured retries are used for both templating and api proxy, though if template requests go through proxy (currently requires persistence enabled) we'll only configure retries for the latter to avoid duplicate retrying.  Though there is some duplicate retrying already because whenever the template server does a retry when not going through the proxy, the Vault client it uses allows for 2 behind-the-scenes retries for some 400/500 http error codes.
2021-03-18 14:14:09 -04:00
Clint 5353279e75
Add command to look up a lease by ID (#11129)
* snapshot

* basic test

* update command and add documentation

* update help text

* typo

* add changelog for lease lookup command

* run go mod vendor

* remove tabs from help output
2021-03-18 11:11:09 -05:00
Nick Cabatoff 66e10f64bc
Fix a bug in Agent's handling of consistency headers: they should be ignored when computing cache ID. (#11099) 2021-03-12 12:42:06 -05:00
Vishal Nayak 9839e76192
Remove unneeded fields from state output (#11073) 2021-03-10 12:08:12 -05:00
Calvin Leung Huang 0a6e7ab94b
mod: update plugin versions for 1.7.0 (#11046)
* mod: update plugin versions for 1.7.0

* command/agent: fix TestCFEndToEnd test
2021-03-04 18:32:51 -08:00
Jim Kalafut 4ef8d66318
Reset agent backoff on successful auth (#11033)
The existing code would retain the previous backoff value even after the
system had recovered. This PR fixes that issue and improves the
structure of the backoff code.
2021-03-03 14:15:18 -08:00
Theron Voran 1fdf08b149
agent: persistent caching support (#10938)
Adds the option of a write-through cache, backed by boltdb

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-03-03 14:01:33 -08:00
Vishal Nayak 3e55e79a3f
Autopilot: Server Stabilization, State and Dead Server Cleanup (#10856)
* k8s doc: update for 0.9.1 and 0.8.0 releases (#10825)

* k8s doc: update for 0.9.1 and 0.8.0 releases

* Update website/content/docs/platform/k8s/helm/configuration.mdx

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Autopilot initial commit

* Move autopilot related backend implementations to its own file

* Abstract promoter creation

* Add nil check for health

* Add server state oss no-ops

* Config ext stub for oss

* Make way for non-voters

* s/health/state

* s/ReadReplica/NonVoter

* Add synopsis and description

* Remove struct tags from AutopilotConfig

* Use var for config storage path

* Handle nin-config when reading

* Enable testing autopilot by using inmem cluster

* First passing test

* Only report the server as known if it is present in raft config

* Autopilot defaults to on for all existing and new clusters

* Add locking to some functions

* Persist initial config

* Clarify the command usage doc

* Add health metric for each node

* Fix audit logging issue

* Don't set DisablePerformanceStandby to true in test

* Use node id label for health metric

* Log updates to autopilot config

* Less aggressively consume config loading failures

* Return a mutable config

* Return early from known servers if raft config is unable to be pulled

* Update metrics name

* Reduce log level for potentially noisy log

* Add knob to disable autopilot

* Don't persist if default config is in use

* Autopilot: Dead server cleanup (#10857)

* Dead server cleanup

* Initialize channel in any case

* Fix a bunch of tests

* Fix panic

* Add follower locking in heartbeat tracker

* Add LastContactFailureThreshold to config

* Add log when marking node as dead

* Update follower state locking in heartbeat tracker

* Avoid follower states being nil

* Pull test to its own file

* Add execution status to state response

* Optionally enable autopilot in some tests

* Updates

* Added API function to fetch autopilot configuration

* Add test for default autopilot configuration

* Configuration tests

* Add State API test

* Update test

* Added TestClusterOptions.PhysicalFactoryConfig

* Update locking

* Adjust locking in heartbeat tracker

* s/last_contact_failure_threshold/left_server_last_contact_threshold

* Add disabling autopilot as a core config option

* Disable autopilot in some tests

* s/left_server_last_contact_threshold/dead_server_last_contact_threshold

* Set the lastheartbeat of followers to now when setting up active node

* Don't use config defaults from CLI command

* Remove config file support

* Remove HCL test as well

* Persist only supplied config; merge supplied config with default to operate

* Use pointer to structs for storing follower information

* Test update

* Retrieve non voter status from configbucket and set it up when a node comes up

* Manage desired suffrage

* Consider bucket being created already

* Move desired suffrage to its own entry

* s/DesiredSuffrageKey/LocalNodeConfigKey

* s/witnessSuffrage/recordSuffrage

* Fix test compilation

* Handle local node config post a snapshot install

* Commit to storage first; then record suffrage in fsm

* No need of local node config being nili case, post snapshot restore

* Reconcile autopilot config when a new leader takes over duty

* Grab fsm lock when recording suffrage

* s/Suffrage/DesiredSuffrage in FollowerState

* Instantiate autopilot only in leader

* Default to old ways in more scenarios

* Make API gracefully handle 404

* Address some feedback

* Make IsDead an atomic.Value

* Simplify follower hearbeat tracking

* Use uber.atomic

* Don't have multiple causes for having autopilot disabled

* Don't remove node from follower states if we fail to remove the dead server

* Autopilot server removals map (#11019)

* Don't remove node from follower states if we fail to remove the dead server

* Use map to track dead server removals

* Use lock and map

* Use delegate lock

* Adjust when to remove entry from map

* Only hold the lock while accessing map

* Fix race

* Don't set default min_quorum

* Fix test

* Ensure follower states is not nil before starting autopilot

* Fix race

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-03-03 13:59:50 -05:00
Scott Miller a7b372b447
Two minor changes not reflected OSS side (#11020) 2021-02-26 14:23:56 -06:00
Nick Cabatoff c9ae15bd95
Make metrics access unauthenticated when in dev mode. (#10992) 2021-02-24 14:04:23 -05:00
Vishal Nayak 565568f7fa
Bool pointer flag (#10988)
* Bool pointer flag

* Add default handling

* Support Hidden and EnvVar

* Adjust error handling

* Add test steps; Fix hidden
2021-02-24 10:45:03 -05:00
Nick Cabatoff c1ddfbb538
OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
Jason O'Donnell 5502d43f6e
agent: add caching encryption package (#10986)
* agent: add caching encryption package

* Fix documentation

* Add GetKey, GetPersistentKey

* Remove chan from interface

* Add error to interface

* Fix tests
2021-02-23 19:48:39 -05:00
Jim Kalafut e60cc11f33
Add configurable exponential backoff to Agent auto-auth (#10964) 2021-02-23 12:04:21 -08:00
Jason O'Donnell 458061d43b
agent: route templating server through cache (#10927)
* agent: route templating server through cache

* Remove TemplateRetry, fix unix path

* Remove mtls comment, remove redundant tls enable

* Fix test

* Refactor vault address logic

* Fix cert/key for mtls

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update command/agent/template/template_test.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Reject mtls listeners

* changelog

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-02-23 09:36:11 -05:00
Clint 2aff402279
Bundle new Vault plugin: Terraform secrets (#10931)
* Bundle Terraform secrets engine

* update go.mod/sum

* vendor update

* add changelog entry

* add secrets terraform
2021-02-19 16:38:56 -06:00
Austin Gebauer a7531a11ea
Updates the JWT/OIDC auth plugin (#10919) 2021-02-16 17:21:35 -08:00
swayne275 6e1b183f79
Shutdown Test Cores when Tests Complete (#10912)
* Shutdown Test Cores when Tests Complete

* go mod vendor
2021-02-12 13:04:48 -07:00
Jason O'Donnell ba9b3318d8
agent: allow auto-auth to use an existing token (#10850)
* agent/auto-auth: add use_existing_token

* Add better logging for lookup errors

* Fix test

* changelog

* Remove preload config, add token var

* Update filename

* Update changelog

* Revert test name

* Remove unused function

* Remove redundant error message

* Short circuit authenticate for preloaded token

* Add comment for auto-auth login
2021-02-11 09:36:03 -05:00
Vishal Nayak 53cb1deb38
Revert "Read-replica instead of non-voter (#10875)" (#10890)
This reverts commit fc745670cf34821f5834357d9caebc3351dbc1e7.
2021-02-10 16:41:58 -05:00
Vishal Nayak a2394e7353
Read-replica instead of non-voter (#10875) 2021-02-10 09:58:18 -05:00
Theron Voran c62ce48b5b
Set TokenParent in the Index to be cached (#10833)
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2021-02-03 18:30:41 -08:00
Mark Gritter 595b0445be
`vault operator diagnose` stub command (#10819)
* Stub "operator diagnose" command.
* Parse configuration files.
* Refactor storage setup to call from diagnose.
* Add the ability to run Diagnose as a prequel to server start.
2021-02-02 14:15:10 -06:00
Calvin Leung Huang 0df09e356d
agent: add an inflight cache better concurrent request handling (#10705)
* agent: do not grap idLock writelock until caching entry

* agent: inflight cache using sync.Map

* agent: implement an inflight caching mechanism

* agent/lease: add lock for inflight cache to prevent simultaneous Set calls

* agent/lease: lock on a per-ID basis so unique requests can be processed independently

* agent/lease: add some concurrency tests

* test: use lease_id for uniqueness

* agent: remove env flags, add comments around locks

* agent: clean up test comment

* agent: clean up test comment

* agent: remove commented debug code

* agent/lease: word-smithing

* Update command/agent/cache/lease_cache.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* agent/lease: return the context error if the Done ch got closed

* agent/lease: fix data race in concurrency tests

* agent/lease: mockDelayProxier: return ctx.Err() if context got canceled

* agent/lease: remove unused inflightCacheLock

* agent/lease: test: bump context timeout to 3s

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-01-26 12:09:37 -08:00
Vishal Nayak 2602675402
Set namespace for template server in agent (#10757)
* Set namespace for template server in agent

* cl++
2021-01-25 17:37:01 -05:00
Ricardo Cardenas 049301f70b
feat(agent): add retry configuration for vault agent (#10644)
* feat(agent): add retry configuration for vault agent

* feat(agent): add test fixtures for retry

* fix(retry): move retry stanza to top level as template_retry

* fix(retry): add retry config to ServerConfig struct

* fix(retry): point config parser to parse template_retry instead of retry

* remove netlify config (#10711)

* Fix build (#10749)

* Move the declaration to a OSS build tag file to not have it collide w… (#10750)

* Move the declaration to a OSS build tag file to not have it collide with ent declarations

* Add comment

* Remove comment to trigger ci

* Unconditionally use the root namespace when calling sys/seal-status. (#10742)

* feat(agent): add retry configuration for vault agent

* feat(agent): add test fixtures for retry

* fix(retry): move retry stanza to top level as template_retry

* fix(retry): add retry config to ServerConfig struct

* fix(retry): point config parser to parse template_retry instead of retry

Co-authored-by: Hridoy Roy <roy@hashicorp.com>
Co-authored-by: Jeff Escalante <jescalan@users.noreply.github.com>
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
Co-authored-by: Mark Gritter <mgritter@hashicorp.com>
2021-01-25 11:00:17 -08:00
Mark Gritter bd6d25499f
Unconditionally use the root namespace when calling sys/seal-status. (#10742) 2021-01-25 11:25:54 -06:00
Vishal Nayak 5d270db1df
Add list peers to DR secondaries (#10746) 2021-01-22 11:50:59 -05:00
Eugene R 331529fc94
Aerospike storage backend (#10131)
* add an Aerospike storage backend

* go mod vendor

* add Aerospike storage configuration docs

* review fixes

* bump aerospike client to v3.1.1

* rename the defaultHostname variable

* relocate the docs page
2021-01-12 15:26:07 -08:00
Lauren Voswinkel 7189a67a33
Adding snowflake as a bundled database secrets plugin (#10603)
* Adding snowflake as a bundled database secrets plugin

* Add snowflake-database-plugin to expected bundled plugins

* Add snowflake plugin name to the mockBuiltinRegistry
2021-01-07 09:30:24 -08:00
Jace Tan 74cadeb8e5
Fix typo in usage description of operator init -status flag (#10618)
This commit fixes a typo in the operator init -status flag's usage
description and aligns it with that found on https://www.vaultproject.io/docs/commands/operator/init#status.
2020-12-23 13:12:44 -07:00
Nick Cabatoff d2096b251d
Add log gathering to debug command. (#10609) 2020-12-22 15:15:24 -05:00
Mark Gritter 8c67bed7ae
Send a test message before committing a new audit device. (#10520)
* Send a test message before committing a new audit device.
Also, lower timeout on connection attempts in socket device.
* added changelog
* go mod vendor (picked up some unrelated changes.)
* Skip audit device check in integration test.
Co-authored-by: swayne275 <swayne@hashicorp.com>
2020-12-16 16:00:32 -06:00
Josh Black a7aac342bd
Only set the namespace if the env var isn't present (#1519) (#10556) 2020-12-14 11:40:48 -08:00
Brian Kassouf 275ca323e8
core: Record the time a node became active (#10489)
* core: Record the time a node became active

* Update vault/core.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Add omitempty field

* Update vendor

* Added CL entry and fixed test

* Fix test

* Fix command package tests

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2020-12-11 16:50:19 -08:00
Nick Cabatoff 84d566db9e
Be consistent with how we report init status. (#10498)
Also make half-joined raft peers consider storage to be initialized, whether or not they're sealed.
2020-12-08 13:55:34 -05:00
Caius Durling c1fa14e00c
Fix typo in print token synopsis text (#10320)
Co-authored-by: swayne275 <swayne@hashicorp.com>
2020-12-03 10:59:09 -07:00
Josh Black b8ba047b35
Allow Vault Agent to run as a Windows service (#10231) 2020-11-23 14:24:32 -08:00
Mark Gritter ab2e28bf55
"vault operator usage" CLI for client count reporting (#10365)
* Working draft of CLI command.
* Sort order, robustness checking.
* Text edits and check of queries_available.
* Added changelog.
2020-11-23 14:57:35 -06:00
Michel Vocks 22d55c00e6
Fix KV list command with whitespaces (#8017)
* Fix KV list command with whitespaces

* Fix kv list whitespace

* Fix list whitespace

* Fix failing test

Co-authored-by: swayne275 <swayne@hashicorp.com>
2020-11-23 12:38:14 -07:00
Hridoy Roy 6261afb343
Port: Telemetry For Lease Expiration Times (#10375)
* port lease metrics

* go mod vendor

* caught a bug
2020-11-13 10:26:58 -08:00
Brian Kassouf 10668331e4
Update go version to 1.15.3 (#10279)
* Update go version to 1.15.3

* Fix OU ordering for go1.15.x testing

* Fix CI version

* Update docker image

* Fix test

* packagespec upgrade -version 0.1.8

Co-authored-by: Sam Salisbury <samsalisbury@gmail.com>
2020-10-30 16:44:06 -04:00
Brian Kassouf 81a86f48e8
Backport some OSS changes (#10267)
* Backport some OSS changes

* go mod vendor
2020-10-29 16:47:34 -07:00
Scott Miller dd0ea9a389
Wrap the dev logs in a sync.Once and deregister first, to eliminate the possibility of emitting the dev output twice in a race. (#10258) 2020-10-28 10:23:18 -05:00
Aleksandr Bezobchuk 95bbd8d920
Merge PR #10192: Auto-Join: Configurable Scheme & Port (and add k8s provider) 2020-10-23 16:13:09 -04:00
Byungjin Park (BJ) ca63903321
Fix outdated enable-versioning command usage in kv (#10212) 2020-10-23 12:46:20 -07:00
Nick Cabatoff 0d6a929a4c
Same seal migration oss (#10224)
* Refactoring and test improvements.

* Support migrating from a given type of autoseal to that same type but with different parameters.
2020-10-23 14:16:04 -04:00
Michael Golowka bd79fbafb3
Add couchbase, elasticsearch, and mongodbatlas back (#10222)
Updated the `Serve` function so these can be added back into Vault
2020-10-22 17:20:17 -06:00
Michael Golowka e6c8ee24ea
DBPW - Enables AutoMTLS for DB plugins (#10220)
This also temporarily disables couchbase, elasticsearch, and
mongodbatlas because the `Serve` function needs to change signatures
and those plugins are vendored in from external repos, causing problems
when building.
2020-10-22 15:43:19 -06:00
Brian Kassouf 84dbca38a1
Revert "Migrate internalshared out (#9727)" (#10141)
This reverts commit ee6391b691ac12ab6ca13c3912404f1d3a842bd6.
2020-10-13 16:38:21 -07:00
Aleksandr Bezobchuk d37be9af6e
Merge PR #10095: Integrated Storage Cloud Auto-Join 2020-10-13 16:26:39 -04:00
Jeff Mitchell e6881c8147
Migrate internalshared out (#9727)
* Migrate internalshared out

* fix merge issue

* fix merge issue

* go mod vendor

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2020-10-12 11:56:24 -07:00
Hridoy Roy 27d68e7df2
Flaky Test Fix: TestSink [VAULT-720] (#10086)
* fixed

* test sink fix

* remove print statements used for debugging

* remove print statements used for debugging

Co-authored-by: HridoyRoy <hridoyroy@Hridoys-MacBook-Pro.local>
Co-authored-by: HridoyRoy <hridoyroy@Hridoys-MBP.hitronhub.home>
2020-10-05 10:29:08 -07:00
Aleksandr Bezobchuk a3cfa7c447
Merge PR #10059: Port OSS changes from #1497 2020-10-01 15:15:20 -04:00
Calvin Leung Huang 90a3f32771
agent: return a non-zero exit code on error (#9670)
* agent: return a non-zero exit code on error

* agent/template: always return on template server error, add case for error_on_missing_key

* agent: fix tests by updating Run params to use an errCh

* agent/template: add permission denied test case, clean up test var

* agent: use unbuffered errCh, emit fatal errors directly to the UI output

* agent: use oklog's run.Group to schedule subsystem runners (#9761)

* agent: use oklog's run.Group to schedule subsystem runners

* agent: clean up unused DoneCh, clean up agent's main Run func

* agent/template: use ts.stopped.CAS to atomically swap value

* fix tests

* fix tests

* agent/template: add timeout on TestRunServer

* agent: output error via logs and return a generic error on non-zero exit

* fix TestAgent_ExitAfterAuth

* agent/template: do not restart ct runner on new incoming token if exit_after_auth is set to true

* agent: drain ah.OutputCh after sink exits to avoid blocking on the channel

* use context.WithTimeout, expand comments around ordering of defer cancel()
2020-09-29 18:03:09 -07:00
Theron Voran 52581cd472
Add logging during awskms auto-unseal (#9794)
Adds debug and warn logging around AWS credential chain generation,
specifically to help users debugging auto-unseal problems on AWS, by
logging which role is being used in the case of a webidentity token.

Adds a deferred call to flush the log output as well, to ensure logs
are output in the event of an initialization failure.
2020-09-28 14:06:49 -07:00
Hridoy Roy c595244482
Normalize format output for vault status [VAULT-508] (#9976)
* normalize format output for vault status

* interim commit

* interim commit

* make formatting idiomatic

* clean up comments

* added formatting test

* updated comments in format test to match godocs

Co-authored-by: HridoyRoy <hridoyroy@Hridoys-MBP.hitronhub.home>
Co-authored-by: HridoyRoy <hridoyroy@Hridoys-MacBook-Pro.local>
2020-09-23 10:30:01 -07:00
Lauren Voswinkel 5740e1ff9e
5844 AWS Root Credential Rotation (#9921)
* strip redundant field type declarations

* root credential rotation for aws creds plugin

* Change location of mocks awsutil and update methods that no longer exist

* Update website/pages/docs/auth/aws.mdx

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update sdk version to get the awsutil mock file

* Re-vendor modules to pass CI

* Use write lock for the entirety of AWS root cred rotation

* Update docs for AWS root cred rotation for clarity

Co-authored-by: Becca Petrin <beccapetrin@gmail.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-09-15 15:26:56 -07:00
ncabatoff b615da43d7
Run CI tests in docker instead of a machine. (#8948) 2020-09-15 10:01:26 -04:00
Scott Miller ed131225a2
Don't open a raft snapshot file until we have a successful snapshot response. (#9894)
* Don't open the snapshot file until we have a successful response

* Check the success of Close if nothing else errors
2020-09-14 12:22:45 -05:00
Jason O'Donnell 6f552506ea
Add disable_mlock env support (#9931) 2020-09-11 12:56:30 -04:00
Jason O'Donnell 469555ef1a
agent/auth/kerberos: add disable_fast_negotiation (#9892)
* agent/auth/kerberos: add disable_fast_negotiation

* simplify test

* Update command/agent/auth/kerberos/kerberos_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* simplify tests

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-09-04 16:46:18 -04:00
Mark Gritter c4ee595e23
Retry a KV put command if the message indicates kv-v2 upgrade. (#9873) 2020-09-02 10:53:36 -05:00
Scott Miller 24794d8457
Dev Message after log quiescence (#9702)
* Register a log sink that delays the printing of the big dev warning until logs have settled down

* Since this is always an intercept logger, just be explicit about the type

* changelog++
2020-08-31 10:45:50 -05:00
Mark Gritter 6d1b71962f
Add date/time argument type. (#9817)
* Add date/time argument type.
* Add an argument to select which time formats are valid.
* Increase minimum date for epoch timestamps to avoid ambiguity.
2020-08-26 14:40:23 -05:00
Calvin Leung Huang 8b7a3812dc
agent: support providing certificate information in cert's config map (#9819)
* agent: support providing certificate information in cert's config map

* update TestCertEndToEnd

* remove URL reference on warning message
2020-08-25 14:26:06 -07:00
Scott Miller 5b003b06f8
Trail of bits 018 (#9674)
* TOB-018 remediation

* Make key derivation an optional config flag, off by default, for backwards compatibility

* Fix unit tests

* Address some feedback

* Set config on unit test

* Fix another test failure

* One more conf fail

* Switch one of the test cases to not use a derive dkey

* wip

* comments
2020-08-17 11:36:16 -05:00
Rodrigo D. L d0df8bfa21
adding new config flag disable_sentinel_trace (#9696) 2020-08-10 06:23:44 -04:00
ncabatoff b0a3972892
Use printf field widths instead of strings.Repeat. (#9681) 2020-08-07 08:54:23 -04:00
Tom Proctor 4ca978598f
Bundle couchbase database plugin with vault (#9664) 2020-08-07 11:01:04 +01:00
Alexander Bezobchuk 479777fcfe
Merge PR #9615: tests: attempt build tag fix 2020-07-28 21:43:14 -04:00
ncabatoff b491c6d72a
Fix parsing of seal stanzas that have an array for `purpose` (#9589)
Hexadecimal integers will be converted to decimal, which is unfortunate but shouldn't have any negative effects other than perhaps confusion in the `vault debug` output.
2020-07-27 16:28:52 -04:00
aphorise 7e2f80b8c4
Always provide version & storage type in status output (#9598) 2020-07-27 14:33:57 -04:00
ncabatoff bbd991a008
Fix 1.5 regression that meant non-string values in the seal stanza would fail config parsing, preventing startup. (#9555) 2020-07-23 13:53:00 -04:00
ncabatoff b02652058b
Prevent panic due to upgrading existing `kv` mount to v2. (#9536) 2020-07-20 16:35:26 -04:00
Brian Kassouf 90ed4d7eac
Fix UI in dev mode 2020-07-02 15:18:57 -07:00
Scott Miller 2aa90105d3
Address feedback on Plugin Reload: OSS Side (#9350)
* just use an error string

* Switch command to use new struct
2020-06-30 16:26:38 -05:00
Scott Miller 001ee861bd
Global Plugin Reload: OSS Changes Take II (#9347)
* Carefully move changes from the plugin-cluster-reload branch into this clean branch off master.

* Don't test this at this level, adequately covered in the api level tests

* Change PR link

* go.mod

* Vendoring

* Vendor api/sys_plugins.go
2020-06-30 10:26:52 -05:00
Scott Miller e92f8f5a81
Revert global plugin reload commits (#9344)
* Revert "Some of the OSS changes were clobbered when merging with quotas out of, master (#9343)"

This reverts commit 8719a9b7c4d6ca7afb2e0a85e7c570cc17081f41.

* Revert "OSS side of Global Plugin Reload (#9340)"

This reverts commit f98afb998ae50346849050e882b6be50807983ad.
2020-06-29 17:36:22 -05:00
Scott Miller cc51427584
Some of the OSS changes were clobbered when merging with quotas out of, master (#9343)
* OSS side of Global Plugin Reload
2020-06-29 16:58:51 -05:00
Scott Miller a83fe0fc6d
OSS side of Global Plugin Reload (#9340)
* OSS side of Global Plugin Reload

* changelog++
2020-06-29 16:23:28 -05:00
Calvin Leung Huang c45bdca0b3
raft: add support for using backend for ha_storage (#9193)
* raft: initial work on raft ha storage support

* add note on join

* add todo note

* raft: add support for bootstrapping and joining existing nodes

* raft: gate bootstrap join by reading leader api address from storage

* raft: properly check for raft-only for certain conditionals

* raft: add bootstrap to api and cli

* raft: fix bootstrap cli command

* raft: add test for setting up new cluster with raft HA

* raft: extend TestRaft_HA_NewCluster to include inmem and consul backends

* raft: add test for updating an existing cluster to use raft HA

* raft: remove debug log lines, clean up verifyRaftPeers

* raft: minor cleanup

* raft: minor cleanup

* Update physical/raft/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/ha.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/ha.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/logical_system_raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* address feedback comments

* address feedback comments

* raft: refactor tls keyring logic

* address feedback comments

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* address feedback comments

* testing: fix import ordering

* raft: rename var, cleanup comment line

* docs: remove ha_storage restriction note on raft

* docs: more raft HA interaction updates with migration and recovery mode

* docs: update the raft join command

* raft: update comments

* raft: add missing isRaftHAOnly check for clearing out state set earlier

* raft: update a few ha_storage config checks

* Update command/operator_raft_bootstrap.go

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* raft: address feedback comments

* raft: fix panic when checking for config.HAStorage.Type

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update website/pages/docs/commands/operator/raft.mdx

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* raft: remove bootstrap cli command

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* raft: address review feedback

* raft: revert vendored sdk

* raft: don't send applied index and node ID info if we're HA-only

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2020-06-23 12:04:13 -07:00
Scott Miller 632c86ecc1
Add new Telemetry config options (#9238)
* Add new Telemetry config options

Add cluster_name, maximum_gauge_cardinality, and usage_gauge_period
configuration options to the config stanza.

Update unit tests.

Document.

Co-authored-by: Mark Gritter <mgritter@hashicorp.com>
2020-06-17 10:07:33 -05:00
ncabatoff c8833c24d8
Restart template server if it shuts down (#9200) 2020-06-15 15:25:45 -04:00
Mike Jarmy 4303790aae
Test pre-1.4 seal migration (#9085)
* enable seal wrap in all seal migration tests

* move adjustForSealMigration to vault package

* fix adjustForSealMigration

* begin working on new seal migration test

* create shamir seal migration test

* refactor testhelpers

* add VerifyRaftConfiguration to testhelpers

* stub out TestTransit

* Revert "refactor testhelpers"

This reverts commit 39593defd0d4c6fd79aedfd37df6298391abb9db.

* get shamir test working again

* stub out transit join

* work on transit join

* Revert "move resuable storage test to avoid creating import cycle"

This reverts commit b3ff2317381a5af12a53117f87d1c6fbb093af6b.

* remove debug code

* initTransit now works with raft join

* runTransit works with inmem

* work on runTransit with raft

* runTransit works with raft

* get rid of dis-used test

* cleanup tests

* TestSealMigration_TransitToShamir_Pre14

* TestSealMigration_ShamirToTransit_Pre14

* split for pre-1.4 testing

* add simple tests for transit and shamir

* fix typo in test suite

* debug wrapper type

* test debug

* test-debug

* refactor core migration

* Revert "refactor core migration"

This reverts commit a776452d32a9dca7a51e3df4a76b9234d8c0c7ce.

* begin refactor of adjustForSealMigration

* fix bug in adjustForSealMigration

* clean up tests

* clean up core refactoring

* fix bug in shamir->transit migration

* remove unnecessary lock from setSealsForMigration()

* rename sealmigration test package

* use ephemeral ports below 30000

* simplify use of numTestCores
2020-06-11 15:07:59 -04:00
Yoan Blanc 77dfab2b62
operator: init -status to return JSON (#8773) 2020-06-08 14:35:39 -04:00
Jason O'Donnell ab0bbc595b
agent/raft: fix typo in help strings (#9114) 2020-06-02 10:17:08 -04:00
ncabatoff 4481521c0e
Extend agent template tests to also validate that updated templates get re-rendered. (#9097) 2020-05-29 13:36:59 -04:00
Thomas L. Kula 3ce9615992
Allow auto_auth with templates without specifying a sink (#8812)
For situations where you want the Vault agent to handle one or more templates but do not require the acquired credentials elsewhere.

Modify the logic in SyncServer so that if there are no sinks, ignore any new credentials. Since SyncServer is responsible for shutting down the agent, make sure it still properly shuts down in this new situation.

Solves #7988
2020-05-26 13:52:14 -04:00
Alexander Bezobchuk 1dd2113755
Merge PR #9078: Add go version to server message output 2020-05-26 12:28:51 -04:00
Alexander Bezobchuk 378ec869db
Merge PR #9060: Support reading Raft TLS flags from file 2020-05-23 11:09:55 -04:00
Josh Black 6e92c8cbd2
Add a new "vault monitor" command (#8477)
Add a new "vault monitor" command

Co-authored-by: ncabatoff <ncabatoff@hashicorp.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
2020-05-21 13:07:50 -07:00
Jason O'Donnell dd254b08d5
agent/template: update consul-template dep (#9044) 2020-05-20 13:03:33 -04:00
Josh Black e53556130c
Fix JSON encoding adding newlines. (#8928)
Fix JSON encoding adding newlines.

This manifested itself when encoding config values, which all map to
strings. An extra new line would get added by json.Encode, which caused
other things to break with confusing error messagges. Switching to
json.Marshal seems to solve the problem.
2020-05-19 16:13:05 -07:00
ncabatoff 6291b40e34
Don't include username or password of proxy env vars when logging them. (#9022) 2020-05-19 10:07:46 -04:00
Brian Kassouf 1bb0bd489d
storage/raft: Add committed and applied indexes to the status output (#9011)
* storage/raft: Add committed and applied indexes to the status output

* Update api vendor

* changelog++

* Update http/sys_leader.go

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-05-18 16:07:27 -07:00
Calvin Leung Huang 8cefbca1c9
Refactor service registration (#8976)
* serivceregistration: refactor service registration logic to run later

* move state check to the internal func

* sr/kubernetes: update setInitialStateInternal godoc

* sr/kubernetes: remove return in setInitialState

* core/test: fix mockServiceRegistration

* address review feedback
2020-05-15 11:06:58 -07:00
Jeff Mitchell 1d3d89e2aa
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Mark Gritter bd766d7bae
Metrics wrapper that adds the cluster name as a label. (#8961) 2020-05-12 21:00:59 -05:00
Vladimir Dimitrov 34f01920e9
Added plugin reload function to api (#8777)
* Added plugin reload function to api

* Apply suggestions from code review

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* go mod vendor

* addressing comments

* addressing comments

* add docs

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-05-04 11:14:23 -07:00
ncabatoff 55609f1d38
Ensure that the .vault-token file writen by `vault login` always has the correct permissions and ownership. (#8867) 2020-04-27 19:55:13 -04:00
Calvin Leung Huang 660ce17dd0
seal/gcpkms: fix panic that occurs when the seal config map is nil (#8840) 2020-04-24 13:31:03 -07:00
Calvin Leung Huang 0509985408
http: ensure return after writing response by respondError (#8796) 2020-04-21 15:30:36 -07:00
Calvin Leung Huang 2659c34910
raft: check for nil on concrete type in SetupCluster (#8784)
* raft: check for nil on concrete type in SetupCluster

* raft: move check to its own func

* raft: func cleanup

* raft: disallow disable_clustering = true when raft storage is used

* docs: update disable_clustering to mention new behavior
2020-04-21 13:45:07 -07:00
Mike Jarmy 5ae2f103d8
Enable transit->shamir seal migration in Enterprise (#8737)
* Enable transit->shamir seal migration in Enterprise

* prove that we can stop the transit cluster after migration is complete
2020-04-14 10:36:46 -04:00
Vishal Nayak 9909dfb31f
Fix segfault when seal block is removed (#8517) 2020-04-13 22:54:21 -04:00
Becca Petrin 8e1ecefe59
minor docs updates (#8725) 2020-04-13 10:03:05 -07:00
Vishal Nayak b93bd39b97
Support unwrapping tokens that does not contain data (#8714)
* Support unwrapping tokens that does not contain data

* s/token/secret
2020-04-09 14:09:23 -04:00
Calvin Leung Huang e8cc90e8b6
raft: disallow ha_storage stanza when raft storage is used (#8707) 2020-04-08 13:07:42 -07:00
Calvin Leung Huang 712f15c0af
test: port test changes to oss (#8694) 2020-04-07 11:26:51 -07:00
Mike Jarmy ca711c2ac4
Expand seal migration tests (#8658)
* rename some seal migration tests

* add comments to seal migration tests

* add comments to seal migration tests

* rename sealhelper

* creat TestSealMigration_ShamirToTestSeal

* fix typo in test suite

* create TestSealMigration_TransitToTestSeal
2020-04-01 13:21:32 -04:00
Becca Petrin 3d3114b0c5
allow delete-version-after to be reset to 0 (#8635) 2020-03-30 16:19:44 -07:00
Vishal Nayak 5647f978f1
Raft cli docs (#8548)
* Raft cli docs

* s/raft/Raft

* s/raft/Raft

* Update website/pages/docs/commands/operator/raft.mdx

Co-Authored-By: Meggie <m.ladlow@gmail.com>

* Mention that shamir seal requires unseal keys

Co-authored-by: Meggie <m.ladlow@gmail.com>
2020-03-17 09:46:44 -04:00
Tommy Murphy a936a77f01
stackdriver: metric label extraction (#8073)
* stackdriver: use label extraction and add debug config

* go.mod: update go-metrics-stackdriver

* vendor go-metrics-stackdriver
2020-03-13 07:58:45 +01:00
Jeff Mitchell 0b09580c36 Remove misleading comment 2020-03-11 09:04:45 -04:00
Brian Kassouf fe90e0a9e0
Rename raft configuration command to list-peers and make output easier to read (#8484)
* Make the output of raft configuration easier to read

* Rename raft configuration sub command to list-peers

* Update command/operator_raft_listpeers.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-03-06 14:40:50 -08:00
Brian Kassouf 87d88a12c6
Fix migrating away from raft storage (#8485) 2020-03-06 09:35:17 -08:00
ncabatoff 50eb50e574
Use public seal helper to create a transit server to handle autounseal. (#8395) 2020-02-20 10:05:23 -05:00
Jeff Mitchell da5aef4878 Move kv-builder to internalshared 2020-02-19 11:13:13 -05:00
Jeff Mitchell 844b2c3a5d Bump API/SDK and adapt to move from SDK stuff 2020-02-15 14:58:05 -05:00
Gerardo Di Giacomo 8573eefe90
enabling TLS 1.3 support for TCP listeners (#8305)
* adding support for TLS 1.3 for TCP listeners

* removed test as CI uses go 1.12

* removed Cassandra support, added deprecation notice

* re-added TestTCPListener_tls13
2020-02-15 11:40:18 -08:00
Jason O'Donnell dd9f25a118
Add OpenLDAP Secret Plugin (#8360)
* Add openldap secret plugin

* go mod vendor

* Revert to go-ldap 3.1.3

* go mod vendor
2020-02-15 13:21:07 -05:00
Calvin Leung Huang 22ac66c922
command/generate-root: support returning json format on decode and generate otp (#7870) 2020-02-14 18:26:49 -08:00
Joe Dollard 8f74b4d2b6
provide vault server flag to exit on core shutdown (#7561)
* provide vault server flag to exit on core shutdown

* Update command/server.go

Co-Authored-By: Jeff Mitchell <jeffrey.mitchell@gmail.com>

Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-02-14 18:07:31 -08:00
Jeff Mitchell fc56999c5c
Switch output loggers in hclog instead of going through gatedlogger (#8228)
After flushing we no longer have to take a mutex hit for every log line.
2020-02-14 17:50:47 -08:00
Brian Kassouf 3bbd9dc34c
Port some changes back to OSS (#8359) 2020-02-14 16:39:13 -08:00
Clint 39f1d26902
Docs enforce autoauth token (#8270)
* rename UseAutoAuthForce to ForceAutoAuth, because I think it reads better

* Document 'ForceAuthAuthToken' option for Agent Cache

* Update website/pages/docs/agent/caching/index.mdx

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* Add additional tests around use_auto_auth=force and add documentation

* remove note, it's no longer correct

Co-authored-by: Jim Kalafut <jim@kalafut.net>
2020-02-14 15:48:12 -06:00
Clint d3cda0fe2c
Guard against using Raft as a seperate HA Storage (#8239)
* Guard against using Raft as a seperate HA Storage

* Document that Raft cannot be used as a seperate ha_storage backend at this time

* remove duplicate imports from updating with master
2020-02-14 14:25:53 -06:00
Vishal Nayak c87d34d1a4
Seal migration with Raft (#8103)
* Seal migration after unsealing

* Refactor migration fields migrationInformation in core

* Perform seal migration as part of postUnseal

* Remove the sleep logic

* Use proper seal in the unseal function

* Fix migration from Auto to Shamir

* Fix the recovery config missing issue

* Address the non-ha migration case

* Fix the multi cluster case

* Avoid re-running seal migration

* Run the post migration code in new leaders

* Fix the issue of wrong recovery being set

* Address review feedback

* Add more complete testing coverage for seal migrations.   (#8247)

* Add more complete testing coverage for seal migrations.  Also remove VAULT_ACC gate from some tests that just depend on docker, cleanup dangling recovery config in storage after migration, and fix a call in adjustCoreForSealMigration that seems broken.

* Fix the issue of wrong recovery key being set

* Adapt tests to work with multiple cores.

* Add missing line to disable raft join.

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* Fix all known issues

* Remove warning

* Review feedback.

* Revert my previous change that broke raft tests.  We'll need to come back and at least comment
this once we better understand why it's needed.

* Don't allow migration between same types for now

* Disable auto to auto tests for now since it uses migration between same types which is not allowed

* Update vault/core.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Add migration logs

* Address review comments

* Add the recovery config check back

* Skip a few steps if migration is already done

* Return from waitForLeadership if migration fails

Co-authored-by: ncabatoff <nick.cabatoff@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-02-13 16:27:31 -05:00
Becca Petrin 5a01a5af0f
Add Kubernetes service registration (#8249) 2020-02-13 09:56:29 -08:00
Jeff Malnick 27ad920399
Add redshift database plugin (#8299)
* feat: add redshift database plugin

* build: update vendored libraries

* docs: add reference doc for redshift variant of the database secrets engine

* feat: set middlewear type name for better metrics naming (#8346)

Co-authored-by: Becca Petrin <beccapetrin@gmail.com>
2020-02-13 09:42:30 -08:00
Jeff Mitchell 86327b8010 Bump api/sdk and fix imports 2020-02-13 10:41:16 -05:00
ncabatoff e63da2e183
Add support for telemetry option `metrics_prefix` (#8340)
When set, instead of starting with `vault_`, metrics will start with `${prefix}_`.
2020-02-13 09:00:32 -05:00
Daniel Spangenberg f7cfec47ea
Fix CLI namespace commands (#8315)
* Fix namespace commands help

* Fix useless prediction for namespace commands

* Add namespace prediction capability
2020-02-09 19:38:39 +01:00
Jim Kalafut 2ee7b76469
Bundle MongoDB Atlas (#8309) 2020-02-07 14:09:39 -08:00
Daniel Spangenberg f84288d74f
Support CLI autocompletion for nested mounts (#8303)
* Support CLI autocompletion for nested mounts

* Add test for nested autocomplete prediction
2020-02-07 15:58:30 +01:00
Becca Petrin 1459544630
update from github.com/hashicorp/gokrb5 to github.com/jcmturner/gokrb5/v8 (#8296) 2020-02-05 14:23:22 -08:00
Alex Antonov 3457d383ba
Added flag to disable X-Vault-Token header proxy if client passes the token (#8101)
* Added flag to disable X-Vault-Token header proxy if client passes the token

* Reveresed the flag value to better match the name intent

* Introduced UseAutoAuthTokenRaw for Cache to support triplicate value of true/false/force

Co-authored-by: Clint <catsby@users.noreply.github.com>
2020-01-30 09:08:42 -06:00
Calvin Leung Huang d8dfd81e47
test: fix TestAgent_Template_Basic (#8257)
* test: fix TestAgent_Template_Basic

* test: fix TestAgent_Template_ExitCounter
2020-01-29 09:31:29 -08:00
Michel Vocks 5ab64e11d7
Fix Vault Agent Template TLS config parameters (#8243) 2020-01-28 10:59:31 +01:00
Jim Kalafut f17fc4e5c1
Run goimports (#8251) 2020-01-27 21:11:00 -08:00
Becca Petrin 759f9b38f7
Observer pattern for service registration interface (#8123)
* use observer pattern for service discovery

* update perf standby method

* fix test

* revert usersTags to being called serviceTags

* use previous consul code

* vault isnt a performance standby before starting

* log err

* changes from feedback

* add Run method to interface

* changes from feedback

* fix core test

* update example
2020-01-24 09:42:03 -08:00
Jeff Mitchell 5dd56b76a6 Update sdk to use gatedwriter from it 2020-01-23 14:00:49 -05:00
Jeff Mitchell 7826f8a284
Update gated-writer logic a bit (#8227)
This is to smooth some other changes coming once
https://github.com/hashicorp/go-hclog/pull/56 lands
2020-01-23 13:57:18 -05:00
Vishal Nayak fb93b0bf9c Use Shamir as KeK when migrating from auto-seal to shamir (#8172)
* Use Shamir as KeK when migrating from auto-seal to shamir

* Use the correct number of shares/threshold for the migrated seal.

* Fix log message

* Add WaitForActiveNode to test

* Make test fail

* Minor updates

* Test with more shares and a threshold

* Add seal/unseal step to the test

* Update the logic that prepares seal migration (#8187)

* Update the logic that preps seal migration

* Add test and update recovery logic

Co-authored-by: ncabatoff <nick.cabatoff@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-01-21 09:24:33 -08:00
Jeff Mitchell 3956072c93 Update test var name and tidy 2020-01-16 20:18:59 -05:00
Jeff Mitchell 34333f5046
Pull wrapping creation to a var (#8137)
* Pull a func out to a var

* Funcouttovarextension (#8153)

* Update test

Co-authored-by: Lexman <Lexman42@users.noreply.github.com>
2020-01-16 13:10:30 -05:00
Vishal Nayak 8891f2ba88 Raft retry join (#7856)
* Raft retry join

* update

* Make retry join work with shamir seal

* Return upon context completion

* Update vault/raft.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Address some review comments

* send leader information slice as a parameter

* Make retry join work properly with Shamir case. This commit has a blocking issue

* Fix join goroutine exiting before the job is done

* Polishing changes

* Don't return after a successful join during unseal

* Added config parsing test

* Add test and fix bugs

* minor changes

* Address review comments

* Fix build error

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-01-13 17:02:16 -08:00
Jeff Mitchell a0694943cc
Migrate built in auto seal to go-kms-wrapping (#8118) 2020-01-10 20:39:52 -05:00
Becca Petrin 9f5b4caba0 fix AWS http import 2020-01-10 15:53:56 -08:00
Becca Petrin 6291c75175
Add support for AWS instance metadata v2 (#8062)
* add support for aws instance metadata v2

* use GetDynamicData instead
2020-01-10 09:31:16 -08:00
Becca Petrin c2894b8d05
Add Kerberos auth agent (#7999)
* add kerberos auth agent

* strip old comment

* changes from feedback

* strip appengine indirect dependency
2020-01-09 14:56:34 -08:00
Michel Vocks 246e3c7a2f
Fix error handling during client TLS config setup (#8025) 2019-12-18 11:22:15 +01:00
Becca Petrin a7383b6d86
Add Kerberos SPNEGO auth plugin (#7908) 2019-12-11 11:18:37 -08:00
Mike Jarmy e42bc0ffc0
Introduce optional service_registration stanza (#7887)
* move ServiceDiscovery into methods

* add ServiceDiscoveryFactory

* add serviceDiscovery field to vault.Core

* refactor ConsulServiceDiscovery into separate struct

* cleanup

* revert accidental change to go.mod

* cleanup

* get rid of un-needed struct tags in vault.CoreConfig

* add service_discovery parser

* add ServiceDiscovery to config

* cleanup

* cleanup

* add test for ConfigServiceDiscovery to Core

* unit testing for config service_discovery stanza

* cleanup

* get rid of un-needed redirect_addr stuff in service_discovery stanza

* improve test suite

* cleanup

* clean up test a bit

* create docs for service_discovery

* check if service_discovery is configured, but storage does not support HA

* tinker with test

* tinker with test

* tweak docs

* move ServiceDiscovery into its own package

* tweak a variable name

* fix comment

* rename service_discovery to service_registration

* tweak service_registration config

* Revert "tweak service_registration config"

This reverts commit 5509920a8ab4c5a216468f262fc07c98121dce35.

* simplify naming

* refactor into ./serviceregistration/consul
2019-12-06 09:46:39 -05:00
Michel Vocks beeec81df7
Add enable_hostname_label option to telementry stanza (#7902) 2019-12-02 09:26:46 +01:00
Andrea Scarpino d9ef12e5f0 Fix typo in the documentation (#7938) 2019-11-26 15:57:51 -06:00
Calvin Leung Huang b8daaa8ce0
agent/template: give more time for test to poll on directory before f… (#7872)
* agent/template: give more time for test to poll on directory before failing

* refactor timeout bit
2019-11-25 09:07:38 -08:00
Calvin Leung Huang 7f8b892534
command: fix namespace list help text (#7890) 2019-11-25 08:57:32 -08:00
Calvin Leung Huang ad6aaf9f8f
agent: add -exit-after-auth flag (#7920)
* agent: add -exit-after-auth flag

* use short timeout for tests to prevent long test runs on error

* revert sdk/go.mod
2019-11-21 14:46:15 -08:00
Clint 2f2e0e27dc
Agent Template: check rendering to match expectations (#7899)
* add regression test for #7883

* Add logic to count render events and match them to expected

* remove the WAIT label and make some changes to remove the break statements

* change the 'randomness' of the templateContents test helper method
2019-11-20 14:39:20 -06:00
Clint 847fcf8551 Update how Vault Agent configures Consul Templates internal logger (#7822)
* fix up logger in Vault Agent Template

* update deps
2019-11-12 11:29:29 -08:00
Michel Vocks 447bfa3956 Fix indefinitely hanging agent test (#7866) 2019-11-12 09:19:11 -08:00
Calvin Leung Huang c4d0391893 debug: fix test typo (#7862) 2019-11-11 16:30:49 -08:00
Calvin Leung Huang 609a854b74 debug: ignore invalid target flags (#7860) 2019-11-11 15:32:11 -08:00
Clint d0aa3ba053
Vault Agent Template follow-ups (#7739)
* Vault Agent Template: parse templates  (#7540)

* add template config parsing, but it's wrong b/c it's not using mapstructure

* parsing consul templates in agent config

* add additional test to configuration parsing, to cover basics

* another test fixture, rework simple test into table

* refactor into table test

* rename test

* remove flattenKeys and add other test fixture

* Update command/agent/config/config.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* return the decode error instead of swallowing it

* Update command/agent/config/config_test.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* go mod tidy

* change error checking style

* Add agent template doc

* TemplateServer: render secrets with Consul Template (#7621)

* add template config parsing, but it's wrong b/c it's not using mapstructure

* parsing consul templates in agent config

* add additional test to configuration parsing, to cover basics

* another test fixture, rework simple test into table

* refactor into table test

* rename test

* remove flattenKeys and add other test fixture

* add template package

* WIP: add runner

* fix panic, actually copy templates, etc

* rework how the config.Vault is created and enable reading from the environment

* this was supposed to be a part of the prior commit

* move/add methods to testhelpers for converting some values to pointers

* use new methods in testhelpers

* add an unblock channel to block agent until a template has been rendered

* add note

* unblock if there are no templates

* cleanups

* go mod tidy

* remove dead code

* simple test to starT

* add simple, empty templates test

* Update package doc, error logs, and add missing close() on channel

* update code comment to be clear what I'm referring to

* have template.NewServer return a (<- chan) type, even though it's a normal chan, as a better practice to enforce reading only

* Update command/agent.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* update with test

* Add README and doc.go to the command/agent directory (#7503)

* Add README and doc.go to the command/agent directory

* Add link to website

* address feedback for agent.go

* updated with feedback from Calvin

* Rework template.Server to export the unblock channel, and remove it from the NewServer function

* apply feedback from Nick

* fix/restructure rendering test

* Add pointerutil package for converting types to their pointers

* Remove pointer helper methods; use sdk/helper/pointerutil instead

* update newRunnerConfig to use pointerutil and empty strings

* only wait for unblock if template server is initialized

* update test structure

* some test cleanup

* follow up tests

* remove debugging, fix issue in replacing runner config

* need to handle first render/token

* Simplify the blocking logic to support exit after auth

* fix channel name

* expand TestAgent_Template to include multiple scenarios

* cleanup

* test cleanups after feedback
2019-11-11 17:27:23 -06:00
Calvin Leung Huang e5b8400dc0
server: automatically enable dev mode if dev-consul is provided (#7847) 2019-11-11 09:51:49 -08:00
Jeff Mitchell f2f984557e
Add ability to renew by accessor (#7817)
* Add renewing by accessor

* Add accessor renewing test and fix bug

* Update website docs

* Remove extra newline

* Add command-level test
2019-11-08 11:32:01 -05:00
Jim Kalafut 59e526614d
Run go fmt (#7823) 2019-11-07 08:54:34 -08:00
Calvin Leung Huang 294005e6e1 debug: check for nil secret on replication status response before accessing secret.Data (#7811) 2019-11-06 16:36:38 -05:00
Brian Shumate a55c0908ff Help text: update token create help about id value (#7792)
- Token id is no longer a 36 character UUID
2019-11-05 15:50:20 -08:00
ncabatoff d9205cd3c0
Use port 0 as the listener port so the OS assigns one not in use (#7763) 2019-10-30 10:36:23 -04:00
Dilan Bellinghoven 5f8528381c Add TLS server name to Vault stanza of Agent configuration (#7519) 2019-10-29 09:11:01 -04:00
Jeff Mitchell 5c3649defe Sync up Agent and API's renewers. (#7733)
* Sync up Agent and API's renewers.

This introduces a new type, LifetimeWatcher, which can handle both
renewable and non-renewable secrets, modeled after the version in Agent.
It allows the user to select behavior, with the new style being the
default when calling Start(), and old style if using the legacy Renew()
call.

No tests have been modified (except for reflect issues) and no other
code has been modified to make sure the changes are backwards
compatible.

Once this is accepted I'll pull the Agent version out.

* Move compat flags to NewRenewer

* Port agent to shared lifetime watcher lib
2019-10-28 17:28:59 -07:00
Calvin Leung Huang fcda73eac8
agent: fix data race on inmemSink's token (#7707)
* agent: fix data race on inmemSink's token

* use uber/atomic instead
2019-10-22 10:42:56 -07:00
Calvin Leung Huang 27acae7cea
cli: fix json output for namespace list command (#7705) 2019-10-21 13:38:37 -07:00
Clint 245935447b
Vault Agent Template (#7652)
* Vault Agent Template: parse templates  (#7540)

* add template config parsing, but it's wrong b/c it's not using mapstructure

* parsing consul templates in agent config

* add additional test to configuration parsing, to cover basics

* another test fixture, rework simple test into table

* refactor into table test

* rename test

* remove flattenKeys and add other test fixture

* Update command/agent/config/config.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* return the decode error instead of swallowing it

* Update command/agent/config/config_test.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* go mod tidy

* change error checking style

* Add agent template doc

* TemplateServer: render secrets with Consul Template (#7621)

* add template config parsing, but it's wrong b/c it's not using mapstructure

* parsing consul templates in agent config

* add additional test to configuration parsing, to cover basics

* another test fixture, rework simple test into table

* refactor into table test

* rename test

* remove flattenKeys and add other test fixture

* add template package

* WIP: add runner

* fix panic, actually copy templates, etc

* rework how the config.Vault is created and enable reading from the environment

* this was supposed to be a part of the prior commit

* move/add methods to testhelpers for converting some values to pointers

* use new methods in testhelpers

* add an unblock channel to block agent until a template has been rendered

* add note

* unblock if there are no templates

* cleanups

* go mod tidy

* remove dead code

* simple test to starT

* add simple, empty templates test

* Update package doc, error logs, and add missing close() on channel

* update code comment to be clear what I'm referring to

* have template.NewServer return a (<- chan) type, even though it's a normal chan, as a better practice to enforce reading only

* Update command/agent.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* update with test

* Add README and doc.go to the command/agent directory (#7503)

* Add README and doc.go to the command/agent directory

* Add link to website

* address feedback for agent.go

* updated with feedback from Calvin

* Rework template.Server to export the unblock channel, and remove it from the NewServer function

* apply feedback from Nick

* fix/restructure rendering test

* Add pointerutil package for converting types to their pointers

* Remove pointer helper methods; use sdk/helper/pointerutil instead

* update newRunnerConfig to use pointerutil and empty strings

* only wait for unblock if template server is initialized

* drain the token channel in this test

* conditionally send on channel
2019-10-18 16:21:46 -05:00
ncabatoff 1c98152fa0
Shamir seals now come in two varieties: legacy and new-style. (#7694)
Shamir seals now come in two varieties: legacy and new-style. Legacy
Shamir is automatically converted to new-style when a rekey operation
is performed. All new Vault initializations using Shamir are new-style.

New-style Shamir writes an encrypted master key to storage, just like
AutoUnseal. The stored master key is encrypted using the shared key that
is split via Shamir's algorithm. Thus when unsealing, we take the key
fragments given, combine them into a Key-Encryption-Key, and use that
to decrypt the master key on disk. Then the master key is used to read
the keyring that decrypts the barrier.
2019-10-18 14:46:00 -04:00
Lexman c86fe212c0
oss changes for entropy augmentation feature (#7670)
* oss changes for entropy augmentation feature

* fix oss command/server/config tests

* update go.sum

* fix logical_system and http/ tests

* adds vendored files

* removes unused variable
2019-10-17 10:33:00 -07:00
Calvin Leung Huang d2dbb8c963
Vault Debug (#7375)
* cli: initial work on debug; server-status target

* debug: add metrics capture target (#7376)

* check against DR secondary

* debug: add compression

* refactor check into preflight func

* debug: set short test time on tests, fix exit code bug

* debug: use temp dir for output on tests

* debug: use mholt/archiver for compression

* first pass on adding pprof

* use logger for output

* refactor polling target capture logic

* debug: poll and collect replication status

* debug: poll and collect host-info; rename output files and collection refactor

* fix comments

* add archive test; fix bugs found

* rename flag name to singular target

* add target output test; scaffold other tests cases

* debug/test: add pprof and index file tests

* debug/test: add min timing check tests

* debug: fix index gen race and collection goroutine race

* debug: extend archive tests, handle race between program exit and polling goroutines

* update docstring

* debug: correctly add to pollingWg

* debug: add config target support

* debug: don't wait on interrupt shutdown; add file exists unit tests

* move pprof bits into its goroutine

* debug: skip empty metrics and some pprof file creation if permission denied, add matching unit test

* address comments and feedback

* Vault debug using run.Group (#7658)

* debug: switch to use oklog/run.Group

* debug: use context to cancel requests and interrupt rungroups.

* debug: trigger the first interval properly

* debug: metrics collection should use metrics interval

* debug: add missing continue on metrics error

* debug: remove the use of buffered chan to trigger first interval

* debug: don't shadow BaseCommand's client, properly block on interval capture failures

* debug: actually use c.cachedClient everywhere

* go mod vendor

* debug: run all pprof in goroutines; bump pprof timings in tests to reduce flakiness

* debug: update help text
2019-10-15 15:39:19 -07:00
Vishal Nayak 0d077d7945
Recovery Mode (#7559)
* Initial work

* rework

* s/dr/recovery

* Add sys/raw support to recovery mode (#7577)

* Factor the raw paths out so they can be run with a SystemBackend.

# Conflicts:
#	vault/logical_system.go

* Add handleLogicalRecovery which is like handleLogical but is only
sufficient for use with the sys-raw endpoint in recovery mode.  No
authentication is done yet.

* Integrate with recovery-mode.  We now handle unauthenticated sys/raw
requests, albeit on path v1/raw instead v1/sys/raw.

* Use sys/raw instead raw during recovery.

* Don't bother persisting the recovery token.  Authenticate sys/raw
requests with it.

* RecoveryMode: Support generate-root for autounseals (#7591)

* Recovery: Abstract config creation and log settings

* Recovery mode integration test. (#7600)

* Recovery: Touch up (#7607)

* Recovery: Touch up

* revert the raw backend creation changes

* Added recovery operation token prefix

* Move RawBackend to its own file

* Update API path and hit it using CLI flag on generate-root

* Fix a panic triggered when handling a request that yields a nil response. (#7618)

* Improve integ test to actually make changes while in recovery mode and
verify they're still there after coming back in regular mode.

* Refuse to allow a second recovery token to be generated.

* Resize raft cluster to size 1 and start as leader (#7626)

* RecoveryMode: Setup raft cluster post unseal (#7635)

* Setup raft cluster post unseal in recovery mode

* Remove marking as unsealed as its not needed

* Address review comments

* Accept only one seal config in recovery mode as there is no scope for migration
2019-10-15 00:55:31 -04:00
Jim Kalafut abc0565ee1
Add region parameter to AWS agent auto auth (#7632) 2019-10-14 11:56:46 -07:00
Mike Jarmy 510d82551a
Vault Agent Cache Auto-Auth SSRF Protection (#7627)
* implement SSRF protection header

* add test for SSRF protection header

* cleanup

* refactor

* implement SSRF header on a per-listener basis

* cleanup

* cleanup

* creat unit test for agent SSRF

* improve unit test for agent SSRF

* add VaultRequest SSRF header to CLI

* fix unit test

* cleanup

* improve test suite

* simplify check for Vault-Request header

* add constant for Vault-Request header

* improve test suite

* change 'config' to 'agentConfig'

* Revert "change 'config' to 'agentConfig'"

This reverts commit 14ee72d21fff8027966ee3c89dd3ac41d849206f.

* do not remove header from request

* change header name to X-Vault-Request

* simplify http.Handler logic

* cleanup

* simplify http.Handler logic

* use stdlib errors package
2019-10-11 18:56:07 -04:00
Brian Kassouf 024c29c36a
OSS portions of raft non-voters (#7634)
* OSS portions of raft non-voters

* add file

* Update vault/raft.go

Co-Authored-By: Vishal Nayak <vishalnayak@users.noreply.github.com>
2019-10-11 11:56:59 -07:00
Jason O'Donnell e7974b5a14 bug: VAULT_CLUSTER_ADDR not used in raft (#7619)
* bug: VAULT_CLUSTER_ADDR not used in raft

* Make env take precedence
2019-10-10 11:42:59 -07:00
Calvin Leung Huang d8875b1991
sys/config: config state endpoint (#7424)
* sys/config: initial work on adding config state endpoint

* server/config: add tests, fix Sanitized method

* thread config through NewTestCluster's config to avoid panic on dev modes

* properly guard endpoint against request forwarding

* add http tests, guard against panics on nil RawConfig

* ensure non-nil rawConfig on NewTestCluster cores

* update non-forwarding logic

* fix imports; use no-forward handler

* add missing config test fixture; update gitignore

* return sanitized config as a map

* fix test, use deep.Equal to check for equality

* fix http test

* minor comment fix

* config: change Sanitized to return snake-cased keys, update tests

* core: hold rlock when reading config; add docstring

* update docstring
2019-10-08 10:57:15 -07:00
Anton Soroko 1ef95b240c Remove unused code in kv_get.go (#7583) 2019-10-07 10:18:04 -04:00
Michel Vocks a7a6dd55a5
Add config parameter to allow unauthenticated metrics access (#7550)
* Implement config parameter to allow unathenticated metricss access

* Add unit test for unauthenticated metrics access parameter

* go mod tidy
2019-10-04 09:29:51 +02:00
Jim Kalafut 9a05e95760
Log proxy settings from environment on startup (#7528) 2019-09-30 08:46:42 -07:00
Ivan Kurnosov 1ad67097cd Fixed github-prod path (#7516) 2019-09-26 08:46:41 -04:00
Jim Kalafut db0c672d23
Ignore any existing token during CLI login (#7508)
Fixes #6694
2019-09-25 10:59:42 -07:00
Dilan Bellinghoven 2625b66595 chore: Do not need logger for command/agent/config.LoadConfig (#7496) 2019-09-19 13:03:30 -07:00
Jim Kalafut 3621179560 Fix Agent handling of gzipped responses (#7470)
* Fix Agent handling of gzipped responses

Fixes #6606

* Only remove "gzip" member, if present

* Simplify to just removing Accept-Encoding altogether
2019-09-18 14:24:41 -07:00
Matthew Irish 6e4cc02f4d
expose 'storage_type' on the sys/seal-status endpoint (#7486)
* expose 'storage_type' on the sys/seal-status endpoint

* add comments

* Update vault/core.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>
2019-09-18 14:07:18 -05:00
Jeff Mitchell 09a9587523 Move renewer to internal package so tests don't barf 2019-09-09 16:09:53 -04:00
Jeff Mitchell a965378b17
Support batch tokens in renewer and caching proxy for transit auto-unseal (#7441)
Currently this uses a fork of the api.Renewer code, which we should consolidate in 1.3.
2019-09-09 12:58:28 -07:00