Commit graph

1428 commits

Author SHA1 Message Date
Hridoy Roy 1c427d3286
Port: add client ID to TWEs in activity log [vault-3136] (#12820)
* port for tracking twes as clients

* comment clean up

* changelog

* change changelog entry phrasing
2021-10-14 09:10:59 -07:00
Chris Capurso bbb4ab4a41
Add HTTP PATCH support to KV (#12687)
* handle HTTP PATCH requests as logical.PatchOperation

* update go.mod, go.sum

* a nil response for logical.PatchOperation should result in 404

* respond with 415 for incorrect MIME type in PATCH Content-Type header

* add abstraction to handle PatchOperation requests

* add ACLs for patch

* Adding JSON Merge support to the API client

* add HTTP PATCH tests to check high level response logic

* add permission-based 'kv patch' tests in prep to add HTTP PATCH

* adding more 'kv patch' CLI command tests

* fix TestHandler_Patch_NotFound

* Fix TestKvPatchCommand_StdinValue

* add audit log test for HTTP PATCH

* patch CLI changes

* add patch CLI tests

* change JSONMergePatch func to accept a ctx

* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag

* go fmt

* add a test to verify patching works by default with the root token

* add changelog entry

* get vault-plugin-secrets-kv@add-patch-support

* PR feedback

* reorder some imports; go fmt

* add doc comment for HandlePatchOperation

* add json-patch@v5.5.0 to go.mod

* remove unnecessary cancelFunc for WriteBytes

* remove default for -method

* use stable version of json-patch; go mod tidy

* more PR feedback

* temp go get vault-plugin-secrets-kv@master until official release

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
2021-10-13 15:24:31 -04:00
hghaf099 ad2ef412cc
Customizing HTTP headers in the config file (#12485)
* Customizing HTTP headers in the config file

* Add changelog, fix bad imports

* fixing some bugs

* fixing interaction of custom headers and /ui

* Defining a member in core to set custom response headers

* missing additional file

* Some refactoring

* Adding automated tests for the feature

* Changing some error messages based on some recommendations

* Incorporating custom response headers struct into the request context

* removing some unused references

* fixing a test

* changing some error messages, removing a default header value from /ui

* fixing a test

* wrapping ResponseWriter to set the custom headers

* adding a new test

* some cleanup

* removing some extra lines

* Addressing comments

* fixing some agent tests

* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before

* Removing default custom headers, and renaming some function varibles

* some refacotring

* Refactoring and addressing comments

* removing a function and fixing comments
2021-10-13 11:06:33 -04:00
contrun e292bc1b5d
remove trailing comma in json file (#12640)
Signed-off-by: YI <uuuuuu@protonmail.com>
2021-10-12 12:03:01 -04:00
Tom Proctor 0180ba2984
agent: tolerate partial restore failure from persistent cache (#12718)
* agent: tolerate partial restore failure from persistent cache

* Review comments: improved consistency, test robustness, comments, assertions
2021-10-08 11:30:04 +01:00
vinay-gopalan 458927c2ed
[VAULT-3157] Move mergeStates utils from Agent to api module (#12731)
* move merge and compare states to vault core

* move MergeState, CompareStates and ParseRequiredStates to api package

* fix merge state reference in API Proxy

* move mergeStates test to api package

* add changelog

* ghost commit to trigger CI

* rename CompareStates to CompareReplicationStates

* rename MergeStates and make compareStates and parseStates private methods

* improved error messaging in parseReplicationState

* export ParseReplicationState for enterprise files
2021-10-06 10:57:06 -07:00
Mayo 0bd0339c0b
cleanup unused code and fix t.Fatal usage in goroutine in testing (#11694) 2021-09-30 07:33:14 -04:00
hghaf099 8b29493e76
fix writing back the Filename into unusedKeys in ParseConfig (#12615) 2021-09-23 09:30:44 -04:00
Rachel Culpepper 9ff3fd39a2
fix build tag (#12588) 2021-09-20 17:10:54 -04:00
Pratyoy Mukhopadhyay 0819eac6a8
Update token renew docs (#12572)
* Update docs for token renew api and cli

* Clarify api docs for renew/renew-self

* Update wording around periodic tokens
2021-09-16 16:54:46 -07:00
Theron Voran ae0bda77b3
vault-agent: copy values retrieved from bolt (#12534)
Byte slices returned from Bolt are only valid during a transaction, so
this makes a copy.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2021-09-13 11:06:08 -07:00
Rachel Culpepper c0ada31a99
Vault-2840 Add no-op check for HTTP listener validity (#12421)
* fips tag and no-op code

* tls check

* change to error

* add tests

* add test file

* cleanup

* add changelog

* fix file difference

* remove changelog
2021-09-07 18:36:58 -04:00
Lars Lehtonen fe1cf762fd
command: fix dropped test error (#12474) 2021-09-01 16:05:14 -07:00
Chris Capurso 3f4a381f1b
Add kv custom key metadata (#12218)
* add custom-metdata flag to "kv metadata put" command

* add kv metadata put command test for custom-metadata flag

* add custom_metadata to kv-v2 api docs

* add custom_metadata to kv-v2 cli docs

* update go.mod

* Add custom metadata limits to docs

* add changelog entry

* update vault-plugin-secrets-kv to @master
2021-08-23 15:49:09 -04:00
hghaf099 f885d97774
VAULT-2285 adding capability to accept comma separated entries for au… (#12126)
* VAULT-2285 adding capability to accept comma separated entries for auth enable/tune

* Adding changelog

* Adding logic to detect invalid input parameter for auth enable config

* Updating tune.mdx

* Updating secret enable/tune for comma separated parameters

* Adding further parameter checks for auth/secret tests
Fixing changelog
using builtin type for a switch statement
Fixing a possible panic scenario

* Changing a function name, using deep.Equal instead of what reflect package provides

* Fixing auth/secret enable/tune mdx files

* One more mdx file fix

* Only when users provide a single comma separated string in a curl command, split the entries by commas

* Fixing API docs for auth/mount enable/tune for comma separated entries

* updating docs, removing an unnecessary switch case
2021-08-09 15:37:03 -04:00
Hridoy Roy a3fefdca35
oss part of license diagnose test fix (#12234)
* oss part of license diagnose test fix

* cl
2021-08-02 10:50:49 -07:00
Nick Cabatoff 6016e86115
Fix vault debug so that captured logs include newlines. (#12175) 2021-07-27 09:15:24 -04:00
Jeff Mitchell 33ff878946
Move awsutil over to the go-secure-stdlib version (#12128)
Unlike the other libraries that were migrated, there are no usages of
this lib in any of our plugins, and the only other known usage was in
go-kms-wrapping, which has been updated. Aliasing it like the other libs
would still keep the aws-sdk-go dep in the sdk module because of the
function signatures. So I've simply removed it entirely here.
2021-07-20 20:42:00 -04:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Scott Miller 555eb2ae0b
Add advice as a trace option to spot checks (#12105)
* Add advice as a trace option to spot checks

* typo

* Collect advice when forming the tree
2021-07-15 18:01:51 -05:00
Scott Miller 3f01d92d3b
Suppress logging during a diagnose run (#12101)
* Suppress logging during a diagnose run

* remove debugging
2021-07-15 16:54:59 -05:00
Scott Miller 1aa98e81f7
Make any section skippable via string contains skip checking. (#12055)
* Make any section skippable via string contains skip checking

* Fix unit test
2021-07-13 18:25:04 -05:00
Scott Miller 5ec576a810
Wire up remaining sections for config validation (#12048)
* wip

* Add validation to storage, serviceregistration

* Wire up remaining sections except Storage which is generic
2021-07-12 16:33:17 -05:00
Hridoy Roy 3c3555d6b4
skip seal check for manual sealing (#12046)
* skip seal check for manual sealing

* language
2021-07-12 13:22:56 -07:00
Hridoy Roy d3034f2851
Diagnose Language Pass (#11909)
* save

* save

* save

* first round of the diagnose language pass

* capitalization

* first round of feedback

* fix bug in advise

* a few more nouns to verbs
2021-07-11 15:44:19 -07:00
Pratyoy Mukhopadhyay 9b5e89bd34
[VAULT-2776] Add prefix_filter option to Vault (#12025)
* [VAULT-2776] Add prefix_filter support to vault

* [VAULT-2776] Add filter_default config, update docs

* [VAULT-2776] Add changelog file

* [VAULT-2776] Update telemetry tests and error handling

* [VAULT-2776] Add test fixtures, update test

* [VAULT-2776] Update gitignore hcl filter
2021-07-09 14:49:53 -05:00
Nick Cabatoff 518944c599
Make the list and kv list commands work with wrapping, e.g. for controlgroups (#12031) 2021-07-09 12:08:58 -04:00
hghaf099 fedc31e905
Diagnose warns if HTTPS is not used for ha-storage-tls-consul [VAULT-2773] (#11967)
* Diagnose warns if HTTPS is not used for ha-storage-tls-consul

* Skipping TLS verification if https is not used in ha storage tls consul

* Adding diagnose skip message for consul service registration
2021-07-07 14:35:25 -04:00
Nick Cabatoff a2dcb131ee
vault delete should allow the same output options as vault write,… (#11992)
* `vault delete` and `vault kv delete` should allow the same output options as `vault write`, as delete operations can similarly return data.  This is needed if you want to use control groups with deletion.
2021-07-06 10:36:07 -04:00
Hridoy Roy f3895f7cce
oss part of license diagnose (#11939) 2021-06-25 14:18:34 -07:00
hghaf099 6d7dbc85b5
diagnose: Add seal transit tls check (#11922)
* diagnose: Add seal transit tls check

* Fixing the path to the config file and the path to the cert files

* Addressing comment

* Addressing seal transit tls check comments
2021-06-24 18:30:42 -04:00
Jason O'Donnell b2b25be0ce
agent/template: add static_secret_render_interval configurable (#11934)
* agent/template: add default_lease_duration config

* go mod tidy

* Add changelog

* Fix panic

* Add documentation

* Change to static_secret_render_interval

* Update doc

* Update command/agent/template/template.go

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update changelog/11934.txt

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-06-24 15:40:31 -04:00
Hridoy Roy a8cc5e4557
TLS Verification Bugfixes (#11910)
* tls verification bugfix

* tls verification bugfix

* allow diagnose fail to report status when there are also warnings

* allow diagnose fail to report status when there are also warnings

* Update vault/diagnose/helpers_test.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* comments

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-06-24 10:43:49 -07:00
Hridoy Roy e2614979f7
Diagnose Storage Panic Bugfixes (#11923)
* partial

* fix raft panics and ensure checks are skipped if storage isnt initialized

* cleanup directories

* newline

* typo in nil check

* another nil check
2021-06-24 09:56:38 -07:00
hghaf099 4495b932d8
Fix diagnose panic when configuration file does not exist (#11932)
* Fix diagnose panic when configuration file does not exist

* Addressing comments

* Update command/operator_diagnose.go

Co-authored-by: Hridoy Roy <roy@hashicorp.com>

Co-authored-by: Hridoy Roy <roy@hashicorp.com>
2021-06-23 18:35:52 -04:00
Calvin Leung Huang c1a2a939f9
agent: restart template runner on retry for unlimited retries (#11775)
* agent: restart template runner on retry for unlimited retries

* template: log error message early

* template: delegate retries back to template if param is set to true

* agent: add and use the new template config stanza

* agent: fix panic, fix existing tests

* changelog: add changelog entry

* agent: add tests for exit_on_retry_failure

* agent: properly check on agent exit cases, add separate tests for missing key vs missing secrets

* agent: add note on difference between missing key vs missing secret

* docs: add docs for template_config

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

* docs: fix exit_on_retry_failure, fix Functionality section

* docs: update interaction title

* template: add internal note on behavior for persist case

* docs: update agent, template, and template-config docs

* docs: update agent docs on retry stanza

* Apply suggestions from code review

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Update changelog/11775.txt

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* agent/test: rename expectExit to expectExitFromError

* agent/test: add check on early exits on the happy path

* Update website/content/docs/agent/template-config.mdx

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-21 16:10:15 -07:00
hghaf099 38b753046b
Tls check client certs [VAULT-2117] (#11860)
* add expiration warning to certificate checking for diagnose

* Adding TLS Client CA checks to diagnose
Adding checks for tls_disable_client_certs and tls_require_and_verify_client_cert flags

* Check validity of TLSClientCAFile
Adding related tests

* Addressing comments

* Fixing some tls tests

* Addressing comments

Co-authored-by: HridoyRoy <roy@hashicorp.com>
2021-06-18 19:35:38 -04:00
Brian Kassouf 9dbdc4050d
Diagnose: Skip test-access-storage on raft storage (#11889) 2021-06-17 12:15:19 -07:00
Scott Miller ee0d6603f3
Wire configuration checks into diagnose and fix resulting bugs. (#11854)
* Actually call config.Validate in diagnose

* Wire configuration checks into diagnose and fix resulting bugs.

* go mod vendor

* Merge to vendorless version

* Remove sentinel section to allow diagnose_ok to pass

* Fix unit tests
2021-06-17 13:09:37 -05:00
Hridoy Roy e38f991054
Diagnose checks for raft quorum status and file backend permissions (#11771)
* raft file and quorum checks

* raft checks

* backup

* raft file checks test

* address comments and add more raft and file and process checks

* syntax issues

* modularize functions to compile differently on different os

* compile raft checks everywhere

* more build tag issues

* raft-diagnose

* correct file permission checks

* upgrade tests and add a getConfigOffline test that currently does not work

* comment

* update file checks method signature on windows

* Update physical/raft/raft_test.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* raft tests

* add todo comment for windows root ownership

* voter count message

* raft checks test fixes

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-17 10:04:21 -07:00
Hridoy Roy e58c39033b
Add expiration warning to certificate checking for diagnose [VAULT-1018] (#11850)
* add expiration warning to certificate checking for diagnose

* Update serviceregistration/consul/consul_service_registration.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* review comments

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-06-15 09:53:29 -07:00
Josh Black 3189526cc4
OSS parts of sighup license reload test (#11816) 2021-06-11 09:08:19 -07:00
Hridoy Roy 65e3489c45
Diagnose resource creation checks (#11627)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* merge main branch

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* backup while I fix something else

* fix storage latency test errors

* runtime checks

* diagnose with timeout on seal
2021-06-10 12:29:32 -07:00
Shanker Naik 35593978ed
fix typo in vault operator init cmd (#11792) 2021-06-10 10:39:53 -04:00
Nick Cabatoff bfae4e610b
Pull out license commands, and make the OSS changes needed for the license inspect PR in ent. (#11783) 2021-06-07 14:44:20 -04:00
Scott Miller 69d0242db9
Add Advice and Advise functions for adding an advice section to any span. (#11760)
* wip

* wip

* Finish implementing advice handling and word wrapping

* Properly word wrap messages and warnings

* Remove debugging

* Remove debugging

* Remove unnecessary test

* unit test bug

* go vendor
2021-06-07 11:29:36 -05:00
Josh Black c8cfcd9514
OSS parts of sighup license reload (#11767) 2021-06-04 10:24:35 -07:00
Scott Miller 5e55f1b3fe
Return different exit codes for different diagnose results (#11758)
This allows operators to run diagnose in scripts and detect the difference between success, warning, and failure.

Exit codes are now:
0: Success (no warnings)
1: Failure (some test failed)
2: Warning (some test warned)
3: User input failure such as a bad flag
4: Other error
2021-06-04 11:52:49 -05:00
Hridoy Roy 991f9eba56
remove operating system stanza from tests on account of disk usage being different across ci for different runs (#11764) 2021-06-03 14:11:31 -07:00
Scott Miller 9db384e0f7
Edge cases in the skipping gathering and output logic (#11752)
* Edge cases in the skipping gathering and output logic

* Fix unit test
2021-06-03 11:01:14 -05:00