Commit Graph

1742 Commits

Author SHA1 Message Date
Alexander Scheel fe289a8659
Shadow globally defined var to avoid race (#18856)
* Shadow globally defined var to avoid race

output is defined in format_test.go as a global variable, which the
login_test.go tests were unknowingly using. Since these tests execute in
parallel, under the correct circumstances, they'd race to write/read
from the same variable in separate tests.

Shadow to avoid the race.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove global output variable from tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-26 16:47:42 +00:00
Kit Haines 7ddac6e437
Vault 11795 vault cli verify s ign (#18437)
* The verify-sign command in it's cleanest existing form.

* Working state

* Updates to proper verification syntax

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>

* make fmt

* Git CI caught some stuff.

* Some tests

* PR-review updates.

* make fmt.

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
2023-01-26 10:21:13 -05:00
Peter Wilson 292207b7d1
Parallel migration (#18815) (#18817)
* Parallel migration (#18815)
* flagParallel sanity check
* Attempt to use ErrGroups
* Updated docs
* Allow 'start' and 'max-parallel' together
* parallel flag renamed to max-parallel
* tests for start + parallel
* Removed permit pool
* Updated docs to make it clearer that a high setting might not be honored based on storage backend setting
* System dependent max int size
* Default max-parallel 1 => 10
* Test folder/paths updated

Co-authored-by: Tomasz Pawelczak <10206601+gites@users.noreply.github.com>
Co-authored-by: Mike Palmiotto <mike.palmiotto@hashicorp.com>
2023-01-25 15:19:45 +00:00
Violet Hynes 6ec669bb07
VAULT-12564 Add new token_file auto-auth method (#18740)
* VAULT-12564 Work so far on token file auto-auth

* VAULT-12564 remove lifetime watcher struct modifications

* VAULT-12564 add other config items, and clean up

* VAULT-12564 clean-up and more tests

* VAULT-12564 clean-up

* VAULT-12564 lookup-self and some clean-up

* VAULT-12564 safer client usage

* VAULT-12564 some clean-up

* VAULT-12564 changelog

* VAULT-12564 some clean-ups

* VAULT-12564 batch token warning

* VAULT-12564 remove follow_symlink reference

* VAULT-12564 Remove redundant stat, change temp file creation

* VAULT-12564 Remove ability to delete token after auth
2023-01-24 16:09:32 -05:00
Scott Miller 25960fd034
Add the ability to unseal using recovery keys via an explicit seal option. (#18683)
* wip

* wip

* Got it 'working', but not happy about cleanliness yet

* Switch to a dedicated defaultSeal with recovery keys

This is simpler than trying to hijack SealAccess as before.  Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal.  Then everything proceeds
as if you had a shamir seal to begin with.

* Handle recovery rekeying

* changelog

* Revert go.mod redirect

* revert multi-blob info

* Dumb nil unmarshal target

* More comments

* Update vault/seal.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* pr feedback

* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split

* Better comment on recovery seal during adjustSealMigration

* Make it possible to migrate from an auto-seal in recovery mode to shamir

* Fix sealMigrated to account for a recovery seal

* comments

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Address PR feedback

* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate

* Don't shortcut the reast of seal migration

* get rid of redundant transit server cleanup

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-24 14:57:56 -06:00
Hamid Ghaf 65a41d4f08
named Login MFA methods (#18610)
* named MFA method configurations

* fix a test

* CL

* fix an issue with same config name different ID and add a test

* feedback

* feedback on test

* consistent use of passcode for all MFA methods (#18611)

* make use of passcode factor consistent for all MFA types

* improved type for MFA factors

* add method name to login CLI

* minor refactoring

* only accept MFA method name with its namespace path in the login request MFA header

* fix a bug

* fixing an ErrorOrNil return value

* more informative error message

* Apply suggestions from code review

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* feedback

* test refactor a bit

* adding godoc for a test

* feedback

* remove sanitize method name

* guard a possbile nil ref

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-23 15:51:22 -05:00
Alexander Scheel 9f03ea75a3
Remove t.Parallel() due to initialization race (#18751)
Using RunCommand(...) to set format to JSON for PKI HC tests results in
multiple initCommands(...) being called, overwriting the same global
variable. Nobody else calls the test suite in this way, so remove
t.Parallel() as the CLI isn't really meant to be called in parallel and
there might be other issues.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-18 10:40:35 -05:00
Anton Averchenkov a4973bc45a
Remove timeout logic from ReadRaw functions and add ReadRawWithContext (#18708)
Removing the timeout logic from raw-response functions and adding documentation comments. The following functions are affected:

- `ReadRaw`
- `ReadRawWithContext` (newly added)
- `ReadRawWithData`
- `ReadRawWithDataWithContext`

The previous logic of using `ctx, _ = c.c.withConfiguredTimeout(ctx)` could cause a potential [context leak](https://pkg.go.dev/context):

> Failing to call the CancelFunc leaks the child and its children until the parent is canceled or the timer fires. The go vet tool checks that CancelFuncs are used on all control-flow paths.

Cancelling the context would have caused more issues since the context would be cancelled before the request body is closed.

Resolves: #18658
2023-01-17 15:41:59 -05:00
Tom Proctor d5c35f39c3
Add experiment system + events experiment (#18682) 2023-01-16 16:07:18 +00:00
Violet Hynes e8aa9c6429
VAULT-12542 Add info encouraging users to upgrade if agent version is different to server (#18684)
* VAULT-12542 Add info encouraging users to upgrade if agent version is different to server

* VAULT-12542 Changelog

* VAULT-12542 Language update
2023-01-13 09:49:36 -05:00
Peter Wilson 5f5cad736a
VAULT-12264: Fix log rotation params which require an integer (#18666)
* integer values for some log flags
* Adjusted `log_flags` to expect `int` for max files and max bytes
* Updated `server` and `agent`
 Renamed updateConfig (and updateLogConfig)
* Added int log params to test
* Adjust config/params so we can identify when they're not present
* Removed pointer confusion
2023-01-11 20:04:57 +00:00
Ellie 6f7757e949
add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
Alexander Scheel a2c2f56923
Add pki health-check docs (#18517)
* Add documentation on vault pki health-check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refer users to online docs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-11 11:46:30 -05:00
Peter Wilson e4685c10ef
VAULT-9883: Agent Reloadable Config (#18638)
* Update command/agent.go
* Attempt to only reload log level and certs
* Mimicked 'server' test for cert reload in 'agent'

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Left out the `c.config` tweak that meant changes to lots of lines of code within the `Run` function of Agent command. :)
2023-01-10 17:45:34 +00:00
Mike Palmiotto 43a78c85f4
Mark deprecated builtins Removed (#18039)
* Remove logical database builtins

* Drop removed builtins from registry keys

* Update plugin prediction test

* Remove app-id builtin

* Add changelog
2023-01-09 09:16:35 -05:00
Violet Hynes 0b15ad18a2
VAULT-12095 Support multiple config files for Vault Agent (#18403)
* VAULT-12095 Code changes for multi-config

* VAULT-12095 typo

* VAULT-12095 make vault non-nil during update

* VAULT-12095 docs

* VAULT-12095 small refactor

* VAULT-12095 typos
2023-01-03 12:50:19 -05:00
Alexander Scheel efd2799390
Add PKI base command (#18512)
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-21 10:01:57 -05:00
divyaac f8ad8bc5a5
OSS PR for Config Changes PR (#18418)
* OSS PR for Config Changes PR

* Edited tests

* typo

* Added changelog

* Remove changelog
2022-12-15 12:19:19 -08:00
Mike Palmiotto cb3406b1eb
plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Mike Palmiotto 4b8747ab51
command/audit: Recommend multiple audit devices (#18348)
* command/audit: Add note about enabling multiple audit devices

* docs: Recommend multiple audit devices
2022-12-13 17:51:03 -05:00
Chris Capurso a545b1f22d
Add HCP connectivity info to `vault server` startup logs (#18315)
* add HCP org, proj, and resource ID to server startup logs

* add server config test for cloud stanza

* add changelog entry
2022-12-12 14:01:02 -05:00
Sanad Haj Yahya 3b2e74477e
Server: add and support unix listener (UDS) (#18227)
Co-authored-by: shaj13 <hajsanad@gamil.com>
2022-12-09 12:28:18 -08:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
Violet Hynes 398cf38e1e
VAULT-11510 Vault Agent can start listeners without caching (#18137)
* VAULT-11510 Vault Agent can start listeners without caching

* VAULT-11510 fix order of imports

* VAULT-11510 changelog

* VAULT-11510 typo and better switch

* VAULT-11510 update name

* VAULT-11510 New api_proxy stanza to configure API proxy

* VAULT-11510 First pass at API Proxy docs

* VAULT-11510 nav data

* VAULT-11510 typo

* VAULT-11510 docs update
2022-12-05 10:51:03 -05:00
Alexander Scheel 2398634862
Respond with data to all writes in PKI engine (#18222)
* Respond with data to all writes in PKI engine

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-12-05 10:40:39 -05:00
Ellie 695fe367c9
Log environment variable keys at startup (#18125)
* Log environment variable keys at startup

* run make fmt

* change name

* add changelog

* fix changelog nubmer

* fix title

* add test

* fix message

* Update changelog/18125.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* add trace test

* remove check for >= debug, trace

* Update changelog/18125.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2022-12-02 08:49:24 -06:00
Ellie 9dca708201
Fix vault cli namespace patch examples (#18143)
* fix vault cli namespace patch examples

* add changelog

* Update changelog/18143.txt

Co-authored-by: davidadeleon <56207066+davidadeleon@users.noreply.github.com>

Co-authored-by: davidadeleon <56207066+davidadeleon@users.noreply.github.com>
2022-12-01 14:42:40 -06:00
Luis (LT) Carbonell afdb571319
Skip Formatting For NIL Secret (#18163)
* Skip formatting for a nil secret data

* Same change for put

* Add changelog

* update changelog

* modify filtered output
2022-12-01 13:36:24 -06:00
Nick Cabatoff 12e1b609ac
Create global quotas of each type in every NewTestCluster. (#18038)
Create global quotas of each type in every NewTestCluster.  Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.

NewTestCluster also now starts the cluster, and the Start method becomes a no-op.  Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode.  This was needed because otherwise we can't safely make the quota api call.  We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start. 

The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint.  The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http.  The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
2022-11-29 14:38:33 -05:00
Peter Wilson 33e6a3a87c
VAULT-9900: Log rotation for 'agent' and 'server' commands (#18031)
* Work to unify log-file for agent/server and add rotation
* Updates to rotation code, tried to centralise the log config setup
* logging + tests
* Move LogFile to ShareConfig in test
* Docs
2022-11-29 14:07:04 +00:00
Violet Hynes 3d7f9a402f
VAULT-6368 Metrics-only listener for Agent (#18101)
* VAULT-6368 Metrics-only listener for Agent

* VAULT-6368 changelog

* VAULT-6368 Update config to use string instead of bool

* VAULT-6368 Fix leftover code

* VAULT-6368 Fix changelog

* VAULT-6368 fix typo

* VAULT-6368 recommended doc update

* VAULT-6368 use != over !(==)
2022-11-25 16:00:56 -05:00
Alexander Scheel 5ee7cc5e6d
Various health check improvements + tests (#18096)
* Rename common.go->healthcheck.go

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Push handling of no resources to the health checks

This allows us to better run on empty mounts.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Exit when no issuers are found

This makes health checks less useful.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add additional test criteria, refactor tests

This will allow us to setup more tests.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add more OK statuses when checks are good

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test cases for all bad results

The test for too-many-certs was elided for now due to being too hard to
setup in CI.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test for missing mount

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add expected failure test on empty mount

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test for only having an issuer in the mount

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* More consistently perform permission checks

Also return them to the caller when they're relevant.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test without token

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Run health check tests in parallel

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update command/healthcheck/healthcheck.go

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Update command/healthcheck/healthcheck.go

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2022-11-23 14:42:19 +00:00
Alexander Scheel e938f2080d
Add mount tuning health checks (#17902)
* Add shared helpers across health checks

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add audit_visibility health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add allow_if_modified_since health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add mount-related health checks to CLI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-22 15:44:34 +00:00
Hamid Ghaf 22f51dc6d6
improve kv CLI to remove data or custom metadata using kv patch (#18067)
* improve kv CLI to remove data or custom metadata using kv patch

* CL

* adding a comment
2022-11-21 17:11:36 -05:00
Alexander Scheel 1ba5e5fa28
Rename PKI health check helpers (#18017)
* Rename fetch helpers

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Soften language around managed key roots

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-18 13:42:48 -05:00
Alexander Scheel 8461f096e2
Add auto-tidy, last-tidy, and leaf cert health checks (#17901)
* Add enable_auto_tidy health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tidy_last_run health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add too_many_certs health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tidy, CRL, cert count checks to CLI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Cache stored leaf cert count

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Correctly parse last run

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-18 11:04:58 -05:00
Alexander Scheel 0217f7de65
Add role-based health checks to PKI Health Check (#17877)
* Add more PKI related common utilities

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add role_allows_localhost health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add role_allows_glob_wildcards health checks

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add role_no_store_false health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add new checks to the CLI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Switch to new guard style

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Provision role for test

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address review feedback

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix invalid version check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix message with auto-rebuild enabled

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-17 15:31:58 -05:00
Michael Anthony 0624d8f36e
[QT-309] Ensure environment variables are populated before proceeding (#17915)
* Ensure environment variables are populated before proceeding

* DRY up credNames var
2022-11-17 11:55:17 -07:00
akshya96 22029c696b
Vault 9800 Fix vault read handling for endpoints with no top-level data object (#17913)
* fix vault read

* changing json unmarshal

* adding changelog

* removing line

* removing format=raw

* add test

* fix tests

* add fix for resp.Errors

* adding comments
2022-11-17 10:51:37 -08:00
Alexander Scheel b6e27b7f8f
Fix semgrep false-positive due to co-dependent returns (#17984)
We previously tried to extract this log into a function (shouldExit),
but semgrep doesn't expand function invocations, leading us to be forced
to add another rule to the regex.

Instead, add the extraneous `err != nil` conditional into the if
statements, even though skip/err should always be true in these cases
and it should never be evaluated.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-17 09:26:13 -05:00
Nick Cabatoff 87f0939d11
I think there was a plan at some point to support a "vault server -diagnose" command line option, but that doesn't seem to have ever come to pass. Removing the dead code. (#17983) 2022-11-17 09:25:27 +00:00
Hamid Ghaf 07fe063a4b
fixing a typo in command login (#17996) 2022-11-16 19:21:01 -05:00
Alexander Scheel d1c078381e
Add hardware_backed_root, root_issued_leaves health checks (#17865)
* Add hardware_backed_root health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add root_issued_leaves health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add new health checks to CLI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add more helpers to common PKI health-check code

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Show config when listing, stable output order

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix %v->%w

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-16 15:24:54 -05:00
Alexander Scheel 5101e31857
PKI Health Check Command (#17750)
* Stub out initial health check command

This command will be used to generate health check results for the PKI
engine.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Start common health check implementation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add common health check utilities

These utilities will collect helpers not specific to PKI health checks,
such as formatting longer durations more legibly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add PKI health check common utils

Many health checks will need issuer and/or CRL information in order to
execute. We've centrally located these helpers to avoid particular
health checks from needing to reimplement them each time.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Adding ca_validity_period health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Begin using health-checks in PKI command

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow parsing raw requests afterwards

This shifts the last of the logic difference between Read(...) and
ReadRaw(...) to a new helper, allowing ReadRaw(...) requests to be
parsed into the same response structure afterwards as Read(...); this
allows API callers to fetch the raw secret and inspect the raw response
object in case something went wrong (error code &c) -- and when the
request succeeds, they can still get the api.Secret out.

This will be used with the PKI health check functionality, making both
LIST and READ operations use ReadRaw, and optionally parsing the secret
afterwards.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add crl_validity_period health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for PKI health check

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix bug in raw reading with contexts

When reading raw objects, don't manually call the context cancellation:
this causes timeouts and/or EOF errors when attempting to read or parse
the response body. See message in client.RawRequestWithContext(...) for
more information.

This was causing the test suite to randomly fail, due to the context
cancelling. The test suite's client usually had a default timeout,
whereas the CLI didn't, and thus didn't exhibit the same issue.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix typo in permissions message

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Move %v->%w for errs

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-16 09:27:56 -05:00
Jason O'Donnell de70878e16
core/server: Add environment variable to write stacktrace to file (#17929)
* Add env to write stacktrace to file

* changelog

* Use os.MkdirTemp

* Properly close file

* Adding path override

* Use temp file
2022-11-15 12:03:17 -05:00
Alexander Scheel c25b90831e
Move pki docker tests to pkiext (#17928)
* Export CreateBackendWithStorage for pkiext

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Move zlint_test.go to pkiext

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix mount all test to ignore pkiext

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-14 18:26:26 -05:00
Peter Wilson 0fad0c3864
VAULT-8732: Add `log-file` to Vault Agent (#17841)
* Started work on adding log-file support to Agent
* Allow log file to be picked up and appended
* Use NewLogFile everywhere
* Tried to pull out the config aggregation from Agent.Run

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-11-11 10:59:16 +00:00
Chris Capurso b31dc45ced
add noop ent supported storage check (#17883) 2022-11-10 16:46:59 -05:00
Violet Hynes 65e8eee0ba
VAULT-8703 Add warning for dangerous undocumented overrides, if used, in status response (#17855)
* VAULT-8703 Add warning for dangerous undocumented overrides, if used, in status response

* VAULT-8703 add changelog

* VAULT-8703 fix append
2022-11-09 11:04:36 -05:00
Peter Wilson 469c102fd3
Removed the legacy env var: LOGXI_FORMAT (#17822)
* Removed the legacy env var: LOGXI_FORMAT

* Added changelog

* Actually filled in the CL

* Added the name of legacy env var
2022-11-04 22:16:18 +00:00
Alexander Scheel 8e6e53cf63
Use hashicorp mirror for container pulls (#17778)
When running the test suite in CI (where requests are centralized from
relatively few IPs), we'd occasionally hit Dockerhub's rate limits.
Luckily Hashicorp runs a (limited) public mirror of the containers we
need, so we can switch to them here in the tests.

For consistency between developer and CI, we've opted to have the tests
always pull from the Hashicorp mirror, rather than updating the CI
runner to prefer the mirror.

We exclude nomad and influxdb as we don't presently mirror these repos.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-02 13:33:17 -04:00
Violet Hynes a11f62abf2
VAULT-8518 Increase HMAC limit to 4096, and limit approle names to the same limit (#17768)
* VAULT-8518 Increase HMAC limit to 4096, and limit approle names to the same limit

* VAULT-8518 Changelog

* VAULT-8518 Sprintf the byte limit
2022-11-02 10:42:09 -04:00
akshya96 2945924b2b
Vault 8305 Prevent Brute Forcing in Auth methods : Setting user lockout configuration (#17338)
* config file changes

* lockout config changes

* auth tune r/w and auth tune

* removing changes at enable

* removing q.Q

* go mod tidy

* removing comments

* changing struct name for config file

* fixing mount tune

* adding test file for user lockout

* fixing comments and add changelog

* addressing comments

* fixing mount table updates

* updating consts in auth_tune

* small fixes

* adding hcl parse test

* fixing config compare

* fixing github comments

* optimize userlockouts.go

* fixing test

* minor changes

* adding comments

* adding sort to flaky test

* fix flaky test
2022-11-01 11:02:07 -07:00
Chris Capurso f1f8bc1a0a
Fix kv -mount flag error when mount and secret path are the same (#17679)
* fix mount flag behavior for kv subcommands

* fix mount flag behavior for kv metadata subcommands

* add tests

* add changelog entry
2022-11-01 09:57:23 -04:00
Alexander Scheel 6d92ef4d9a
Fix raw format for other commands, add to docs! (#17730)
* Clarify when -format=raw fails

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Document Vault read's new -format=raw mode

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add raw format to usage, completion

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add missing support for raw format field printing

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Prohibit command execution with wrong formatter

This allows us to restrict the raw formatter to only commands that
understand it; otherwise, when running `vault write -format=raw`, we'd
actually hit the Vault server, but hide the output from the user. By
switching this to a flag-parse time check, we avoid running the rest of
the command if a bad formatter was specified.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-28 12:53:23 -04:00
Alexander Scheel df07170d26
Vault Raw Read Support (CLI & Client) (#14945)
* Expose raw request from client.Logical()

Not all Vault API endpoints return well-formatted JSON objects.
Sometimes, in the case of the PKI secrets engine, they're not even
printable (/pki/ca returns a binary (DER-encoded) certificate). While
this endpoint isn't authenticated, in general the API caller would
either need to use Client.RawRequestWithContext(...) directly (which
the docs advise against), or setup their own net/http client and
re-create much of Client and/or Client.Logical.

Instead, exposing the raw Request (via the new ReadRawWithData(...))
allows callers to directly consume these non-JSON endpoints like they
would nearly any other endpoint.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add raw formatter for direct []byte data

As mentioned in the previous commit, some API endpoints return non-JSON
data. We get as far as fetching this data (via client.Logical().Read),
but parsing it as an api.Secret fails (as in this case, it is non-JSON).
Given that we intend to update `vault read` to support such endpoints,
we'll need a "raw" formatter that accepts []byte-encoded data and simply
writes it to the UI.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support for reading raw API endpoints

Some endpoints, such as `pki/ca` and `pki/ca/pem` return non-JSON
objects. When calling `vault read` on these endpoints, an error
is returned because they cannot be parsed as api.Secret instances:

> Error reading pki/ca/pem: invalid character '-' in numeric literal

Indeed, we go to all the trouble of (successfully) fetching this value,
only to be unable to Unmarshal into a Secrets value. Instead, add
support for a new -format=raw option, allowing these endpoints to be
consumed by callers of `vault read` directly.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove panic

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-28 09:45:32 -04:00
Violet Hynes 6d9ea2862e
VAULT-8519 fix spurious "unknown or unsupported fields" warnings for JSON config (#17660)
* VAULT-8519 add tests for HCL unknown field bug

* VAULT-8519 upversion hcl

* VAULT-8519 include correct comitts in tag

* VAULT-8519 Add changelog
2022-10-27 10:28:03 -04:00
Mike Palmiotto cc96c6f470
Store login MFA secret with tokenhelper (#17040)
* Store login MFA secret with tokenhelper
* Clean up and refactor tokenhelper paths
* Refactor totp test code for re-use
* Add login MFA command tests
* Use longer sleep times and sha512 for totp test
* Add changelog
2022-10-26 17:02:26 -04:00
Alexander Scheel c81bec4d06
Clean up dev cert construction (#17657)
Vault's new TLS devvault mode has two nits with certificate
construction:

 1. The CA doesn't need to include any SANs, as these aren't checked.
    Technically this means the CA could be reused as a leaf certificate
    for the one specified IP SAN, which is less desirable.
 2. Add hostname to SANs in addition to CNs. This is a best practice, as
    (when the CN is a hostname), it is preferable to have everything in
    SANs as well.

Neither of these are major changes.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-26 15:29:37 -04:00
Alexander Scheel 1721cc9f75
Add PATCH support to Vault CLI (#17650)
* Add patch support to CLI

This is based off the existing write command, using the
JSONMergePatch(...) API client method rather than Write(...), allowing
us to update specific fields.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add documentation on PATCH support

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-26 14:30:40 -04:00
Peter Wilson 19d220cb82
Fixed typo in USAGE line (#17582) 2022-10-18 20:08:25 +01:00
Tom Proctor 455f1f6073
CLI: Fix secrets list -detailed headings (#17577) 2022-10-18 14:46:11 +01:00
akshya96 2d6c6c01c4
vault operator init -output-curl-string bug (#17514)
* fixing -output-curl-string bug

* add changelog

* fix for -output-policy

* adding comment for -output-policy
2022-10-13 15:15:01 -07:00
Tom Proctor df690ea414
Plugins: Add -version flag to 'vault plugin info' (#17454)
* Add -version flag to 'vault plugin info'
* Allow specifying a builtin tag when reading a single plugin from the catalog
2022-10-07 15:28:15 +01:00
Tom Proctor 2cf29f93c7
Plugins: Add version info to CLI and server log output (#17430) 2022-10-06 12:54:27 +01:00
Renato Costa ea1257be2e
fix incorrect use of loop variable (#16872)
This fixes a couple of references to loop variables in parallel tests
and deferred functions. When running a parallel test (calling
`t.Parallel()`) combined with the table-driven pattern, it's necessary
to copy the test case loop variable, otherwise only the last test case
is exercised. This is documented in the `testing` package:

https://pkg.go.dev/testing#hdr-Subtests_and_Sub_benchmarks

`defer` statements that invoke a closure should also not reference a
loop variable directly as the referenced value will change in each
iteration of the loop.

Issues were automatically found with the `loopvarcapture` linter.
2022-10-04 09:23:37 -04:00
Peter Wilson 42ba1384ff
Added flag and env var which will disable client redirection (#17352)
* Added flag and env var which will disable client redirection

* Added changelog

* Docs fix for unsaved file, and test single request made

* Updated test for case when redirect is enabled, updated docs based on suggestions
2022-09-30 09:29:37 +01:00
Mike Palmiotto 0078822d1d
core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool (#17319)
* core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool

* docs: Update VAULT_ALLOW_PENDING_REMOVAL_MOUNTS doc
2022-09-29 14:35:52 -04:00
Tom Proctor 12ca81bc9b
cli/api: Update plugin listing to always include version info in the response (#17347) 2022-09-29 18:22:33 +01:00
Alexander Scheel 3eaa4b0d75
Write explicit -help output to stdout (#17308)
* Write explicit -help output to stdout

Per the consensus of most programs, and mirroring the GNU Coding
Standards for CLI design, when users request -help explicitly via the
CLI, this should be written to stdout to allow paging of output. stderr
is fine when an invalid usage triggers the help text however.

In our case, mitchellh/cli helpfully adds a HelpWriter that we
previously set to stderr explicitly. This writer is only called to print
user-requested help text; it is not called on error cases (e.g., bad
usage triggering additional help text to the user).

Thus it should safely be settable to stdout, enabling pagers without
additional redirects.

Notably, we do have to set ErrorWriter as it defaults to initializing to
the value of HelpWriter, which we want to set to stdout now, which is
less useful.

See also: https://www.gnu.org/prep/standards/html_node/_002d_002dhelp.html
Resolves: #17004

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-09-26 12:15:48 -04:00
Chris Capurso 073018e372
fix namespace patch not found error message (#17242)
* fix namespace patch not found error message

* handle 404 in namespace patch cmd
2022-09-23 10:41:11 -04:00
Tom Proctor afccbcb15e
CLI: Add version info to auth/secrets list -detailed (#17293) 2022-09-23 10:40:42 +01:00
Tom Proctor 7d09d5a653
CLI: Tune plugin version for auth/secret mounts (#17277)
* Add -plugin-version flag to vault auth/secrets tune
* CLI tests for auth/secrets tune
* CLI test for plugin register
* Plugin catalog listing bug where plugins of different type with the same name could be double counted
* Use constant for -plugin-version flag name
2022-09-22 20:55:46 +01:00
Yoan Blanc fa8f7c793f
fix: PGP subkeys support (#16224)
* fix: PGP subkeys support

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

* fix: bump ProtonMail/go-crypto

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

* fix: bump ProtonMail/go-crypto

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2022-09-22 09:12:41 -04:00
Tom Proctor b0a580de47
CLI: Fix erroneous warning when reading from stdin (#17252) 2022-09-21 21:04:49 +01:00
Mike Palmiotto 9ced47be66
agent: Fix missing file suffix in config test (#17245) 2022-09-21 11:30:04 -04:00
Tom Proctor f5655ae857
Plugins: Consistently use plugin_version (#17171)
* Delete Sha field, rename RunningSha -> RunningSha256
* Rename version -> plugin_version
2022-09-20 12:35:50 +01:00
Austin Gebauer c1f51417b0
Adds ldap secrets to plugin registry and updates to v0.9.0 (#17152)
* Adds ldap secrets to plugin registry and updates to v0.9.0

* adds changelog

* fix test
2022-09-15 22:19:24 -07:00
Jason O'Donnell 87350f927f
agent/auto-auth: add exit_on_err configurable (#17091)
* agent/auto-auth: add exit_on_err configurable

* changelog

* Update backoff function to quit

* Clarify doc

* Fix test
2022-09-15 11:00:31 -07:00
georgethebeatle f9439a9c41
Make key completion work for both kv-v1 and kv-v2 (#16553)
Co-authored-by: Kieron Browne <kbrowne@vmware.com>
Co-authored-by: Georgi Sabev <georgethebeatle@gmail.com>
Co-authored-by: Danail Branekov <danailster@gmail.com>
2022-09-13 12:11:00 -04:00
Hamid Ghaf 77ec84cfb1
updating hcp link structs, and fix diagnose (#17097) 2022-09-12 11:10:01 -04:00
Max Coulombe 6b2f4e5354
+ added redis elasticache as a built-in plugin (#17075)
* added redis elasticache as a built-in plugin
2022-09-09 16:16:30 -04:00
Mike Palmiotto 9849af8663
Add deprecation status to plugin api and cli (#17077)
* api: Add deprecation status to plugin endpoints

* cli: Add -detailed flag to `plugin list`

* docs: Update plugin list/info docs
2022-09-09 16:03:07 -04:00
Milena Zlaticanin 0977bd1ddc
Import Redis OSS database plugin into Vault (#17070)
* Import Redis OSS database plugin into Vault

* update the total number of db plugins

* small nit for testing

* adding changelog
2022-09-09 13:42:25 -05:00
Josh Black d8e0a13aae
update gofumpt to 0.3.1 and reformat the repo (#17055)
* update gofumpt to 0.3.1 and reformat the repo

* output the version of the formatter we're using
2022-09-07 17:31:20 -07:00
Mike Palmiotto 991c574660
Mark database-specific secrets engines Pending Removal (#17038)
* plugins: Mark standalone database plugins Pending Removal
* Add changelog
2022-09-07 10:45:09 -04:00
Mike Palmiotto bf744e3bde
Handle deprecated builtins (#17005)
* core: Handle deprecated mounts on enable and unseal
* changelog: Deprecation Status handling
* core: Add Pending Removal override var
* core: Add some documentation for Pending Removal override
2022-09-06 15:49:35 -04:00
Hamid Ghaf a034ebfd27
HCP link integration (#16939)
* HCP link integration

* update configure-git.yml

* more OSS stuff

* removing internal repos

* adding a nil check

* removing config test to be included in ENT only

* updating hcp-sdk-go to v0.22.0

* remove Hostname and AuthURL link config params

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-09-06 14:11:04 -04:00
Violet Hynes 95b5449e8a
VAULT-6575 Vault agent respects retry config even with caching set (#16970)
* VAULT-6575 Vault agent respects retry config even with caching set

* VAULT-6575 Add changelog

* VAULT-6575 Change log levels
2022-09-06 10:35:54 -04:00
Mike Palmiotto 2c16be25e3
Add deprecation status to auth/secrets list (#16849)
* auth: Add Deprecation Status to auth list -detailed
* secrets: Add Deprecation Status to secrets list -detailed
* Add changelog entry for deprecation status list
2022-08-31 16:11:14 -04:00
Christopher Swenson 09ad6ab72c
Update mount table and CLI with plugin version for auth (#16856) 2022-08-31 19:23:05 +01:00
aphorise 35650c9b96
Add systemd notify support to Agent. Resolves: #7028 (#9802)
Agent systemd notify added similar to Consul. Resolves: #7028
2022-08-29 08:18:47 -04:00
davidadeleon 592949d7ba
command/debug: fix bug where monitor was not honoring configured duration (#16834)
* Use specific client with longer timeout for log debug target

* add changelog

* update changelog entry
2022-08-23 17:09:30 -04:00
Scott Miller 3bd38fd5dc
OSS portion of wrapper-v2 (#16811)
* OSS portion of wrapper-v2

* Prefetch barrier type to avoid encountering an error in the simple BarrierType() getter

* Rename the OveriddenType to WrapperType and use it for the barrier type prefetch

* Fix unit test
2022-08-23 15:37:16 -04:00
Max Coulombe 905ddff609
Env Flag Filtering (#16683)
* added filtering for global flags

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2022-08-17 21:29:16 -04:00
mickael-hc 3e4f7db237
dev docs: clarify internal and external token helpers (#13014)
* dev docs: clarify internal and external token helpers

* Add docs for DefaultTokenHelper
2022-08-16 16:48:24 -04:00
akshya96 ba15b1f822
Remove extra empty lines from vault.log - Debug command (#16714)
* debug empty lines in vault.log

* minor grammar fix

* add changelog

* add fmt changes
2022-08-15 14:16:57 -07:00
Chris Capurso a0c557f38a
VAULT-7256: Add custom_metadata to namespaces (#16640)
* add mapstructure tags to Namespace struct

* add custom metadata Parse helper

* add ns custom metadata and patch
2022-08-09 11:38:03 -04:00
Eng Zer Jun 61262ad98e
refactor: replace strings.Replace with strings.ReplaceAll (#15392)
strings.ReplaceAll(s, old, new) is a wrapper function for
strings.Replace(s, old, new, -1). But strings.ReplaceAll is more
readable and removes the hardcoded -1.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2022-08-03 15:22:48 -04:00
swayne275 4632a26a09
Use %q for quoted strings where appropriate (#15216)
* change '%s' to %q where single vs double quotes shouldn't matter

* replace double quotes with %q in logs and errors
2022-08-03 12:32:45 -06:00
Jason O'Donnell e3f942f51c
agent: add disable_keep_alives configurable (#16479)
agent: add disable_keep_alives config

Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
2022-07-28 12:59:49 -07:00
Chris Capurso d814ab3825
ignore leading slash in kv get command (#16443)
* ignore leading slash in kv get command

* add changelog entry
2022-07-28 14:11:58 -04:00
Max Coulombe aa3e1c8a2f
Added a small utility method to display warnings when parsing command arguments. (#16441)
* Added a small utility method to display warnings when parsing command arguments

Will print warning if flag is passed after arguments e.g.
vault <command> -a b -c
In this example -c will be interpreted as an argument which may be misleading
2022-07-27 14:00:03 -04:00
akshya96 6e5b4ec8c9
Validate parameters for vault operator init (#16379)
* adding code changes to check parameters for vault operator init

* fixing defaults in cli

* fixing comments

* remove unused consts

* adding validate parameters test

* add changelog

* adding changes to validation

* adding fmt changes

* fixing test

* add test for auto unseal
2022-07-25 12:45:04 -07:00
tdsacilowski 887e77c2ae
Agent JWT auto auth `remove_jwt_after_reading` config option (#11969)
Add a new config option for Vault Agent's JWT auto auth
`remove_jwt_after_reading`, which defaults to true. Can stop
Agent from attempting to delete the file, which is useful in k8s
where the service account JWT is mounted as a read-only file
and so any attempt to delete it generates spammy error logs.

When leaving the JWT file in place, the read period for new
tokens is 1 minute instead of 500ms to reflect the assumption
that there will always be a file there, so finding a file does not
provide any signal that it needs to be re-read. Kubernetes
has a minimum TTL of 10 minutes for tokens, so a period of
1 minute gives Agent plenty of time to detect new tokens,
without leaving it too unresponsive. We may want to add a
config option to override these default periods in the future.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2022-07-25 07:42:09 -06:00
Jason O'Donnell 140406143e
command/server: add dev-tls flag (#16421)
* command/server: add dev-tls flag

* Add website documentation

* changelog

* Lower file permissions

* Update cert gen per review

* Add dev-tls-cert-dir flag and cert clean up

* fmt

* Update cert generation per review

* Remove unused function

* Add better error messages

* Log errors in cleanup, fix directory not existing bug

* Remove hidden flag from -dev-tls-cert-dir

* Add usage

* Update 16421.txt

* Update variable names for files

* Remove directory on cleanup
2022-07-22 14:04:03 -04:00
Jason O'Donnell d25a3526af
command/audit: improve audit enable type missing error message (#16409)
* command/audit: improve audit enable type missing error message

* changelog
2022-07-21 16:43:50 -04:00
Mike Palmiotto f2705c1d66
Fix agent use_auto_auth_token force test (#16313)
Update the test to fix a copy-paste error.
2022-07-15 19:12:59 -04:00
Jason O'Donnell 110f754d97
agent/template: fix exec parsing error for templates (#16231)
* agent/template: fix exec parsing error for templates

* changelog
2022-07-06 21:21:35 +01:00
akshya96 c70a2cd198
Minor grammar correction in help for login command (#16211)
* Minor grammar correction in help for login command

* Fix login command help

Co-authored-by: Pero P <ppejovic@users.noreply.github.com>
2022-07-06 09:17:11 -07:00
Chris Capurso 9501d44ed5
Add endpoints to provide ability to modify logging verbosity (#16111)
* add func to set level for specific logger

* add endpoints to modify log level

* initialize base logger with IndependentLevels

* test to ensure other loggers remain unchanged

* add DELETE loggers endpoints to revert back to config

* add API docs page

* add changelog entry

* remove extraneous line

* add log level field to Core struct

* add godoc for getLogLevel

* add some loggers to c.allLoggers
2022-06-27 11:39:53 -04:00
Jason O'Donnell dd2ced661b
agent: add disable_idle_connections configurable (#15986)
* agent: add disable_keep_alives configurable

* Add empty test

* Add website doc

* Change to disable_idle_connections

* Update tests and doc

* Add note about env

* Changelog

* Change to slice

* Remove unused disable keep alive methods

* Add invalid value test
2022-06-16 18:06:22 -04:00
Josh Black d2ed39a04e
Correct drift between ENT and OSS (#15966) 2022-06-14 17:53:19 -07:00
Dave May 0f42131350
Fix debug bundle panic on Windows (#14399)
* Fix debug bundle panic on Windows

* Add changelog entry
2022-06-09 15:57:45 -07:00
akshya96 8f115a9904
Parse ha_storage in config (#15900)
* parsing values in config ha_storage

* adding changelog

* adding test to parse storage
2022-06-09 15:55:49 -07:00
Chris Capurso 75aa55eb57
add OSS noop check for valid ent storage (#15894) 2022-06-08 17:15:28 -04:00
Alexander Scheel 8d8a95cbf6
Add missing nil check to FIPS EA verification (#15883)
This was causing failures when running `vault server -dev`:

> panic: runtime error: invalid memory address or nil pointer dereference
> [signal SIGSEGV: segmentation violation code=0x2 addr=0x20 pc=0x105c41c1c]
>
> goroutine 1 [running]:
> github.com/hashicorp/vault/command.(*ServerCommand).parseConfig(0x140005a2180)
> 	.../vault/command/server.go:429 +0x5c

Interestingly, we do not have a test case for running the dev
sever.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-08 15:45:46 -04:00
Alexander Scheel dd6c339440
Add warning about EA in FIPS mode (#15858)
* Add warning about EA in FIPS mode

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-08 08:57:48 -04:00
davidadeleon 0026788d4b
api/monitor: Adding log format to monitor command and debug (#15536)
* Correct handling of "unspecified" log level

* Setting log-format default on monitor path

* Create changelog file

* Update website/content/api-docs/system/monitor.mdx

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-24 13:10:53 -04:00
Josh Black 416504d8c3
Add autopilot automated upgrades and redundancy zones (#15521) 2022-05-20 16:49:11 -04:00
VAL c789fe3c8d
Set special output flags whether single or double hyphen (#15528) 2022-05-19 17:56:30 -07:00
Christopher Swenson e6fb16be9c
Remove spurious fmt.Printf calls including one of a key (#15344)
And add a semgrep for fmt.Printf/Println.
2022-05-19 12:27:02 -07:00
Alexander Scheel 9a3f34a41e
Vault CLI: show detailed information with ListResponseWithInfo (#15417)
* CLI: Add ability to display ListResponseWithInfos

The Vault Server API includes a ListResponseWithInfo call, allowing LIST
responses to contain additional information about their keys. This is in
a key=value mapping format (both for each key, to get the additional
metadata, as well as within each metadata).

Expand the `vault list` CLI command with a `-detailed` flag (and env var
VAULT_DETAILED_LISTS) to print this additional metadata. This looks
roughly like the following:

    $ vault list -detailed pki/issuers
    Keys                                    issuer_name
    ----                                    -----------
    0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7    n/a
    35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0    n/a
    382fad1e-e99c-9c54-e147-bb1faa8033d3    n/a
    8bb4a793-2ad9-460c-9fa8-574c84a981f7    n/a
    8bd231d7-20e2-f21f-ae1a-7aa3319715e7    n/a
    9425d51f-cb81-426d-d6ad-5147d092094e    n/a
    ae679732-b497-ab0d-3220-806a2b9d81ed    n/a
    c5a44a1f-2ae4-2140-3acf-74b2609448cc    utf8
    d41d2419-efce-0e36-c96b-e91179a24dc1    something

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow detailed printing of LIST responses in JSON

When using the JSON formatter, only the absolute list of keys were
returned. Reuse the `-detailed` flag value for the `-format=json` list
response printer, allowing us to show the complete API response returned
by Vault.

This returns something like the following:

    {
      "request_id": "e9a25dcd-b67a-97d7-0f08-3670918ef3ff",
      "lease_id": "",
      "lease_duration": 0,
      "renewable": false,
      "data": {
        "key_info": {
          "0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7": {
            "issuer_name": ""
          },
          "35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0": {
            "issuer_name": ""
          },
          "382fad1e-e99c-9c54-e147-bb1faa8033d3": {
            "issuer_name": ""
          },
          "8bb4a793-2ad9-460c-9fa8-574c84a981f7": {
            "issuer_name": ""
          },
          "8bd231d7-20e2-f21f-ae1a-7aa3319715e7": {
            "issuer_name": ""
          },
          "9425d51f-cb81-426d-d6ad-5147d092094e": {
            "issuer_name": ""
          },
          "ae679732-b497-ab0d-3220-806a2b9d81ed": {
            "issuer_name": ""
          },
          "c5a44a1f-2ae4-2140-3acf-74b2609448cc": {
            "issuer_name": "utf8"
          },
          "d41d2419-efce-0e36-c96b-e91179a24dc1": {
            "issuer_name": "something"
          }
        },
        "keys": [
          "0cba84d7-bbbe-836a-4ff6-a11b31dc0fb7",
          "35dfb02d-0cdb-3d35-ee64-d0cd6568c6b0",
          "382fad1e-e99c-9c54-e147-bb1faa8033d3",
          "8bb4a793-2ad9-460c-9fa8-574c84a981f7",
          "8bd231d7-20e2-f21f-ae1a-7aa3319715e7",
          "9425d51f-cb81-426d-d6ad-5147d092094e",
          "ae679732-b497-ab0d-3220-806a2b9d81ed",
          "c5a44a1f-2ae4-2140-3acf-74b2609448cc",
          "d41d2419-efce-0e36-c96b-e91179a24dc1"
        ]
      },
      "warnings": null
    }

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use field on UI rather than secret.Data

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Only include headers from visitable key_infos

Certain API endpoints return data from non-visitable key_infos, by
virtue of using a hand-rolled response. Limit our headers to those
from visitable key_infos. This means we won't return entire columns with
n/a entries, if no key matches the key_info key that includes that
header.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use setupEnv sourced detailed info

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix changelog environment variable

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix broken tests using setupEnv

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-18 13:00:50 -04:00
Hridoy Roy 679ccc81a5
Query and Precompute Non-Contiguous Segments in the Activity Log (#15352)
* query and precompute non-contiguous segments in the activity log

* changelog

* newline formatting

* make fmt

* report listener and storage types as found keys

* report listener and storage types as found keys

* Update vault/activity_log_test.go

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

* review comments

* merge conflict

* merge conflict

* merge conflict

* fix unchecked merge conflict

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-17 12:17:32 -07:00
akshya96 4e9e9b7eda
Vault-6037 making filesystem permissions check opt-in (#15452)
* adding env var changes

* adding changelog

* adding strcov.ParseBool
2022-05-17 11:34:31 -07:00
Violet Hynes 2c6bcbdeb5
VAULT-5885: Fix erroneous success message in case of two-phase MFA, and provide MFA information in table format (#15428)
* VAULT-5885: Fix erroneous success message in case of two-phase MFA, and provide MFA information in table format

* VAULT-5885 Add changelog

* VAULT-5885 Update changelog as per PR comments

* VAULT-5885 Update changelog category to just 'auth'

* VAULT-5885 Hide useless token info in two-phase MFA case

* VAULT-5885 Update changelog to reflect token info now no longer present

* VAULT-5885 split up changelog into three blocks
2022-05-17 14:03:02 -04:00
Alexander Scheel 8750512f9f
Fix integer overflows with new parseutil (#15437)
* Use new parseutil helper: Safe variants

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update parseutil to v0.1.5

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix additional integer overflow in command/server

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-05-16 13:57:19 -04:00
Peter Wilson 2939159a4b
Added optional -log-level flag to 'operator migrate' command (#15405) 2022-05-12 15:56:25 -04:00
Christopher Swenson 0af0543bbe
VAULT-5935 agent: redact renew-self if using auto auth (#15380)
Vault agent redacts the token and accessor for `/auth/token/lookup-self` (and `lookup`)
if the token is the auto auth token to prevent it from leaking.

Similarly, we need to redact the token and accessor from `renew-self`
and `renew`, which also leak the token and accessor.

I tested this locally by starting up a Vault agent and querying the
agent endpoints, and ensuring that the accessor and token were set to
the empty string in the response.
2022-05-12 09:25:55 -07:00
Hridoy Roy 0c955eb743
report listener and storage types as found keys (#15383)
* report listener and storage types as found keys

* changelog
2022-05-12 09:04:56 -07:00
AnPucel ba88b1da16
[VAULT-14990] Support retrieving kv secret paths with trailing spaces (#15188) 2022-05-10 14:07:45 -07:00
davidadeleon 9e869c52fa
Add DR Metric scraping capability to debug command (#15316)
* Add server information as well as ability to collect metrics from DR secondary

* Update debug docs

Adding additional information around ability to gather metrics from DR secondary

* Fix broken link in updated doc

* Create 15316.txt

Create changelog entry

* Fix Formatting

* Update website/content/docs/commands/debug.mdx

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Update changelog/15316.txt

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>

* Trigger Build

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2022-05-06 16:04:08 -04:00
Christopher Swenson 0affe226ad
Update deps for consul-template 0.29.0 (#15293)
This requires bumping https://github.com/mitchellh/go-testing-interface.
For this new version, we have to create a wrapper to convert
the stdlib `testing.TB` interface to the
`mitchellh/go-testing-interface` `T` interface, since it uses
`Parallel()` now, which is not supported by `testing.TB`. This had to be
added to a new package, `benchhelpers`, to avoid a circular dependency
in `testhelpers`.

We also have to *unbump* https://github.com/armon/go-metrics since
updating it breaks our usage of
https://github.com/google/go-metrics-stackdriver

I verified that the new `pkiCert` template function works with agent
injection using annotations like:

```yaml
vault.hashicorp.com/agent-inject-secret-sample.crt: "pki/issue/example-dot-com"
vault.hashicorp.com/agent-inject-template-sample.crt: |
  {{ pkiCert "pki/issue/example-dot-com" "common_name=foo.example.com" "ttl=1h" }}
```
2022-05-05 10:30:40 -07:00
Chris Capurso 5f3d63a03a
clone vault agent client with headers (#15274) 2022-05-03 16:27:53 -04:00
Alexander Tischenko b89fc5af96
When running under systemd, send ready when server completed reloading config #7028 (#15041) 2022-05-03 08:34:11 -04:00
Jason O'Donnell e9535bda2f
agent/auto-auth: Add `min_backoff` to set first backoff value (#15204)
* Add initial_backoff to auto-auth method

* Disable retries in client

* Fix bug

* Thread initial backoff to CT

* Add comment

* Change to min_backoff

* changelog

* remove initial references, review

* fix test

* Thread max_backoff through

* Add doc note for max_backoff/templating
2022-04-29 12:31:32 -04:00
VAL 0ef529b710
Global flag that outputs minimum policy HCL required for an operation (#14899)
* WIP: output policy

* Outputs example policy HCL for given request

* Simplify conditional

* Add PATCH capability

* Use OpenAPI spec and regex patterns to determine if path is sudo

* Add test for isSudoPath

* Add changelog

* Fix broken CLI tests

* Add output-policy to client cloning code

* Smaller fixes from PR comments

* Clone client instead of saving and restoring custom values

* Fix test

* Address comments

* Don't unset output-policy flag on KV requests otherwise the preflight request will fail and not populate LastOutputPolicyError

* Print errors saved in buffer from preflight KV requests

* Unescape characters in request URL

* Rename methods and properties to improve readability

* Put KV-specificness at front of KV-specific error

* Simplify logic by doing more direct returns of strings and errors

* Use precompiled regexes and move OpenAPI call to tests

* Remove commented out code

* Remove legacy MFA paths

* Remove unnecessary use of client

* Move sudo paths map to plugin helper

* Remove unused error return

* Add explanatory comment

* Remove need to pass in address

* Make {name} regex less greedy

* Use method and path instead of info from retryablerequest

* Add test for IsSudoPaths, use more idiomatic naming

* Use precompiled regexes and move OpenAPI call to tests (#15170)

* Use precompiled regexes and move OpenAPI call to tests

* Remove commented out code

* Remove legacy MFA paths

* Remove unnecessary use of client

* Move sudo paths map to plugin helper

* Remove unused error return

* Add explanatory comment

* Remove need to pass in address

* Make {name} regex less greedy

* Use method and path instead of info from retryablerequest

* Add test for IsSudoPaths, use more idiomatic naming

* Make stderr writing more obvious, fix nil pointer deref
2022-04-27 16:35:18 -07:00
Tom Proctor b30e7d3545
Upgrade hashicorp/consul-template dependency (#15092)
* Includes sprig template functions
* Includes improvements to writeTo template function
* Add sprig functions test, improve failure message
2022-04-19 20:51:11 +01:00
Chris Capurso e69f89c279
Add build date (#14957)
* add BuildDate to version base

* populate BuildDate with ldflags

* include BuildDate in FullVersionNumber

* add BuildDate to seal-status and associated status cmd

* extend core/versions entries to include BuildDate

* include BuildDate in version-history API and CLI

* fix version history tests

* fix sys status tests

* fix TestStatusFormat

* remove extraneous LD_FLAGS from build.sh

* add BuildDate to build.bat

* fix TestSysUnseal_Reset

* attempt to add build-date to release builds

* add branch to github build workflow

* add get-build-date to build-* job needs

* fix release build command vars

* add missing quote in release build command

* Revert "add branch to github build workflow"

This reverts commit b835699ecb7c2c632757fa5fe64b3d5f60d2a886.

* add changelog entry
2022-04-19 14:28:08 -04:00
Vinny Mannello 6116903a37
[Vault-5248] MFA support for api login helpers (#14900)
* Add MFA support to login helpers
2022-04-15 11:13:15 -07:00
akshya96 87a2516e73
Fix nightly tests and debug (#14970)
* adding env var

* add fixes

* fixing debug

* removing umask from tests
2022-04-12 06:08:28 +05:30
akshya96 5e0ed27b87
adding env var (#14958) 2022-04-07 12:15:05 -07:00
Anton Averchenkov 23a3f950e4
Revert the WithContext changes to vault tests (#14947) 2022-04-07 15:12:58 -04:00
Christopher Swenson 0de8e8459e
Agent error log level is mismatched (#14424)
* [VAULT-1618] Agent error log level is mismatched

`logLevelToStringPtr` translates `go-hclog`'s `ERROR` to `"ERROR"` for
Consul Template's runner, but that expects `ERR` and is quite strict
about it.

This will address https://github.com/hashicorp/vault-k8s/issues/223
after it is set as the default image in `vault-k8s`.

I didn't find a simple way to test this other than starting up a full
server and agent and letting them run, which is unfortunately fairly
slow.

I confirmed that this addresses the original issue by modifying the helm
chart with the values in this commit and patching the log level to `err`.

* VAULT-1618 Add changelog/14424.txt

* VAULT-1618 Update changelog/14424.txt based on @kalafut suggestion

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>

* VAULT-1618 Move cancel and server stop into defer in tests

* VAULT-1618 Triggering CircleCI tests

* VAULT-1618 Replace ioutil with os functions for agent template tests

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2022-04-07 11:03:38 -07:00
VAL 2113ae1021
Mount flag syntax to mitigate confusion from KV-v2 path discrepancies (#14807)
* Add explanation to help text and flag usage text

* KV get with new mount flag

* Clearer naming

* KV Put, Patch, Metadata Get + corresponding tests

* KV Delete, Destroy, Rollback, Undelete, MetadataDelete, MetadataPatch, MetadataPut

* Update KV-v2 docs to use mount flag syntax

* Add changelog

* Run make fmt

* Clarify deprecation message in help string

* Address style comments
2022-04-06 13:58:06 -07:00
Anton Averchenkov c74feaa6ac
Use WriteWithContext in auth helpers (#14775) 2022-04-06 11:20:34 -04:00
akshya96 796003ddda
Vault 3992 ToB Config and Plugins Permissions (#14817)
* updating changes from ent PR

* adding changelog

* fixing err

* fixing semgrep error
2022-04-04 09:45:41 -07:00
akshya96 9f03f86077
Vault 3999 Change permissions for directory/archive created by debug command (#14846)
* adding debug changes from ent

* adding changelog
2022-04-04 09:44:03 -07:00
hghaf099 9ae2a85700
Fixing excessive unix file permissions (#14791)
* Fixing excessive unix file permissions

* CL

* reduce the permission from 750 to 700
2022-04-01 12:57:38 -04:00