Commit graph

2538 commits

Author SHA1 Message Date
John-Michael Faircloth 7e170e7d87
AutoMTLS for secrets/auth plugins (#15671)
* use automtls for v5 secrets/auth plugins

* add automtls env guard

* start backend without metadata mode

* use PluginClientConfig for backend's NewPluginClient param

refactor

* - fix pluginutil test
- do not expect plugin to be unloaded in UT
- fix pluginutil tests --need new env var
- use require in UT
- fix lazy load test

* add changelog

* prioritize automtls; improve comments

* user multierror; refactor pluginSet for v4 unit test

* add test cases for v4 and v5 plugin versions

* remove unnecessary call to AutoMTLSSupported

* update comment on pluginSets

* use runconfig directly in sdk newpluginclient

* use automtls without metadatamode for v5 backend plugin registration

* use multierror for plugin runconfig calls

* remove some unnecessary code
2022-07-18 16:25:18 -05:00
Mike Palmiotto 439e35f50f
Vault 6773/raft rejoin nonvoter (#16324)
* raft: Ensure init before setting suffrage

As reported in https://hashicorp.atlassian.net/browse/VAULT-6773:

	The /sys/storage/raft/join endpoint is intended to be unauthenticated. We rely
	on the seal to manage trust.

	It’s possible to use multiple join requests to switch nodes from voter to
	non-voter. The screenshot shows a 3 node cluster where vault_2 is the leader,
	and vault_3 and vault_4 are followers with non-voters set to false.  sent two
	requests to the raft join endpoint to have vault_3 and vault_4 join the cluster
	with non_voters:true.

This commit fixes the issue by delaying the call to SetDesiredSuffrage until after
the initialization check, preventing unauthenticated mangling of voter status.

Tested locally using
https://github.com/hashicorp/vault-tools/blob/main/users/ncabatoff/cluster/raft.sh
and the reproducer outlined in VAULT-6773.

* raft: Return join err on failure

This is necessary to correctly distinguish errors returned from the Join
workflow. Previously, errors were being masked as timeouts.

* raft: Default autopilot parameters in teststorage

Change some defaults so we don't have to pass in parameters or set them
in the originating tests. These storage types are only used in two
places:

1) Raft HA testing
2) Seal migration testing

Both consumers have been tested and pass with this change.

* changelog: Unauthn voter status change bugfix
2022-07-18 14:37:12 -04:00
Hridoy Roy 573e01af1d
Throw SSCT Error In Some Cases (#16270)
* throw the ssct error if it signifies that the server needs to take an action

* use errors.Is instead of checking string comparison
2022-07-11 11:12:02 -07:00
Chris Capurso 068a413311
remove GetCoreConfigInternal from logger API tests (#16263) 2022-07-08 19:23:18 +02:00
Violet Hynes 0c80ee5cf5
VAULT-6614 Enable role based quotas for lease-count quotas (OSS) (#16157)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* VAULT-6614 start of changes for roles in LCQs

* Expiration changes for leases

* Add role information to RequestAuth

* VAULT-6614 Test updates

* VAULT-6614 Add expiration test with roles

* VAULT-6614 fix comment

* VAULT-6614 Protobuf on OSS

* VAULT-6614 Add rlock to determine role code

* VAULT-6614 Try lock instead of rlock

* VAULT-6614 back to rlock while I think about this more

* VAULT-6614 Additional safety for nil dereference

* VAULT-6614 Use %q over %s

* VAULT-6614 Add overloading to plugin backends

* VAULT-6614 RLocks instead

* VAULT-6614 Fix return for backend factory
2022-07-05 13:02:00 -04:00
AnPucel 3215cdbd32
Dynamic parameter for mountpaths in OpenApi Spec generation(#15835)
"generic_mount_paths" query parameter for OpenApiSpec generation
2022-06-30 07:43:04 -07:00
Hridoy Roy 2f14d60a4b
Port: Use Stored Hll to Compute New Clients For Current Month (#16184)
* port hll storage changes

* changelog
2022-06-29 10:51:23 -07:00
Hamid Ghaf fa754c7fa5
Replicate member_entity_ids and policies in identity/group across nodes identically (#16088)
* Replicate values of group member_entity_ids and policies across nodes identically

* Adding CL

* fixing tests
2022-06-28 19:54:24 -04:00
Hridoy Roy 895e422c4c
move unused GetCoreConfigInternal to common file to prevent ent panics (#16165) 2022-06-27 14:41:56 -07:00
Hridoy Roy b3959534c2
activity log refactoring port (#16162)
* activity log refactoring port

* changelog
2022-06-27 13:33:45 -07:00
akshya96 42b13448f9
ActivityLog Implement HyperLogLog Store Functionality During Precomputation (#16146)
* adding hll for each month

* add changelog

* removing influxdb

* removing influxdb

* removing influxdb

* changing switch to if-else for semgrep
2022-06-27 09:38:32 -07:00
Chris Capurso 9501d44ed5
Add endpoints to provide ability to modify logging verbosity (#16111)
* add func to set level for specific logger

* add endpoints to modify log level

* initialize base logger with IndependentLevels

* test to ensure other loggers remain unchanged

* add DELETE loggers endpoints to revert back to config

* add API docs page

* add changelog entry

* remove extraneous line

* add log level field to Core struct

* add godoc for getLogLevel

* add some loggers to c.allLoggers
2022-06-27 11:39:53 -04:00
Violet Hynes 5d21fa1e8f
VAULT-6613 Missed a part of OSS for rate limit role quotas (#16132)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* Handle body error

* VAULT-6613 Return early if error with json parsing

* VAULT-6613 add to teardown function
2022-06-24 09:45:53 -04:00
Violet Hynes d57fea2cd1
VAULT-6613 Add role support for rate limit quotas (OSS Changes) (#16115)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* Handle body error

* VAULT-6613 Return early if error with json parsing
2022-06-24 08:58:02 -04:00
Josh Black 2ee2b6ed7c
Return a 403 for a bad SSCT instead of 500 (#16112) 2022-06-23 13:01:20 -07:00
Jason O'Donnell f957573108
Fix bug where id not existing in multiplexing map causes panic (#16094)
* multiplexing: guard against connection panic

* changelog

* Update vault/plugin_catalog.go

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2022-06-22 14:29:25 -04:00
Violet Hynes 56ed9ca8ae
Start of implementation of the plumbing for role resolution logic on auth mounts (#16049)
* VAULT-6612 Initial scaffolding for role determination

* VAULT-6612 Simplify code

* Fix fmt error that somehow happened

* VAULT-6612 Refactor resolve role response
2022-06-21 09:31:36 -04:00
Nick Cabatoff 27ea9a0e41
Set NumCores=1 since we only return a single client anyway. (#16039) 2022-06-17 11:28:25 -04:00
Violet Hynes 96371ed0a5
Small refactor of mount entry manipulation (#16024) 2022-06-17 08:52:43 -04:00
Violet Hynes abed5cf6e7
(OSS) Path Suffix Support for Rate Limit Quotas (#15989)
* Support for rate limit path suffix quotas

* Support for rate limit path suffix quotas

* Precedence test for support for rate limit path suffix quotas

* Update clone method

* Fix mount determination

* Add changelog

* use constant for mounts

* Fix read endpoint, and remount/disable mount

* update godocs for queryquota
2022-06-16 13:23:02 -04:00
akshya96 7e313e29fd
Activity Log Filtering Limit Parameter (#16000)
* adding changes from ent branch

* adding fmt changes

* adding changelog
2022-06-15 15:41:31 -07:00
swayne275 54262d2f4e
clarify lazy revoke prefix if not sync (#15967)
* clarify lazy revoke prefix if not sync

* comment improvement
2022-06-15 12:03:56 -06:00
Josh Black d2ed39a04e
Correct drift between ENT and OSS (#15966) 2022-06-14 17:53:19 -07:00
Chris Capurso 94c5936e27
return bad request instead of server error for identity group cycle detection (#15912)
* return bad request for identity group cycle detection

* add changelog entry

* use change release note instead of improvement

* fix err reference

* fix TestIdentityStore_GroupHierarchyCases
2022-06-10 10:15:31 -04:00
Hridoy Roy 934989809b
Limit SSCT WAL Check on Perf Standbys to Raft Backends Only (#15879)
* ensure that ssct wal check only occurs for non-raft storage on perf standbys

* changelog
2022-06-08 13:58:22 -07:00
VAL 98ed9c7ab9
Add PutMetadata and PatchMetadata KV helpers (#15755)
* Add PutMetadata and PatchMetadata KV helpers

* Refactor tests, stop needlessly converting zero values to nil

* Merge fix for delete-version-after zero value bug

* Cast nils to zero value on Put

* Use 2 different structs for put/patch metadata input, clarity fixes

* Use local constants and simplify stringification
2022-06-08 09:29:22 -07:00
Alexander Scheel b3ad79fb70
Fix listing_visibility value documentation (#15833)
* Match listing_visibility in system/auth with system/mounts

See also: #15209

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix path-help for listing_visibility

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-06 16:44:46 -04:00
Josh Black 9c48c62d6e
Use the incoming request version to populate follower state (#15806) 2022-06-06 08:44:24 -07:00
Violet Hynes d62b140b7c
VAULT-6371 Fix issue with lease quotas on read requests that generate leases (#15735)
* VAULT-6371 Fix issue with lease quotas on non-auth mounts

* VAULT-6371 Add changelog

* VAULT-6371 Amend changelog given new understanding
2022-06-03 15:45:21 -04:00
VAL 3cfafe619b
KV helpers for DeleteMetadata, Undelete, Destroy, and Rollback (#15637)
* KV helpers for DeleteMetadata, Undelete, Destroy, and Rollback

* Allow rollback when no secret data on latest version, and update error messages
2022-06-03 10:42:43 -07:00
Hridoy Roy e64d7df041
refactor some code in modifyResponseMonths and ensure that the last mo… (#15767)
* refactr some code in modifyResponseMonths and ensure that the last month comparison with end is comparing end of month with end of month

* calibrate end of month apropriately and fix parens issue for lastmonth
2022-06-03 10:34:54 -07:00
Hridoy Roy 671aaf1fe0
iterate through all available logs for precomputation and query gets (#15768) 2022-06-03 09:53:53 -07:00
Hridoy Roy a5f70d7fe0
fix off by one error in activity log nil padding for month data (#15731) 2022-06-01 11:09:06 -07:00
VAL ed7c1d4800
Add Patch KV helper (#15587)
* Add Read methods for KVClient

* KV write helper

* Add changelog

* Add Delete method

* Use extractVersionMetadata inside extractDataAndVersionMetadata

* Return nil, nil for v1 writes

* Add test for extracting version metadata

* Split kv client into v1 and v2-specific clients

* Add ability to set options on Put

* Add test for KV helpers

* Add custom metadata to top level and allow for getting versions as sorted slice

* Update tests

* Separate KV v1 and v2 into different files

* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity

* Move structs and godoc comments to more appropriate files

* Add more tests for extract methods

* Rework custom metadata helper to be more consistent with other helpers

* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method

* Add Patch KV helper

* Add godoc comment and use WithOption ourselves in other KVOption functions

* Clean up options-handling and resp parsing logic; add more tests

* Add constants and more patch tests
2022-06-01 07:50:56 -07:00
Hamid Ghaf bf087f9d0d
prevent deleting MFA method through an invalid path (#15482)
* prevent deleting MFA method through an invalid path

* Adding CL
2022-05-31 14:22:04 -04:00
Nick Cabatoff 69c5e8c946
Avoid deadlocking on stateLock in emitMetrics (#15693)
When stopCh is closed we should stop trying to get the lock.
2022-05-31 12:15:39 -04:00
Violet Hynes 4aac96238c
VAULT-6131 OpenAPI schema now includes /auth/token endpoints when explicit permission has been granted (#15552)
* VAULT-6131 OpenAPI schema now includes /auth/token endpoints when explicit permission has been granted

* VAULT-6131 add changelog

* VAULT-6131 Update changelog and fix related bug
2022-05-31 11:25:27 -04:00
Nick Cabatoff ea099fdffd
Like #15682 but this time fix the correct test, i.e. TestRateLimitQuota_Allow (#15684) 2022-05-30 15:08:01 -04:00
Nick Cabatoff 8c2b69f961
Spawn fewer goroutines to make TestRateLimitQuota_Allow less vulnerable to scheduler. (#15682) 2022-05-30 13:33:41 -04:00
Chris Capurso cdb73ab265
use provided namespace for wrapping lookup cubbyhole request (#15583)
* use provided namespace for wrapping lookup cubbyhole request

* add changelog entry
2022-05-26 15:17:29 -04:00
Peter Wilson b7fc4645f3
Only add distinct policies to identity group (#15638)
* Only add distinct policies to identity group
2022-05-26 13:52:19 +01:00
John-Michael Faircloth fc04699f57
Fix plugin reload mounts (#15579)
* fix plugin reload mounts

* do not require sys/ prefix

* update plugin reload docs with examples

* fix unit test credential read path

* update docs to reflect correct cli usage

* allow sys/auth/foo or auth/foo

* append trailing slash if it doesn't exist in request

* add changelog

* use correct changelog number
2022-05-25 13:37:42 -05:00
VAL 64448b62a4
KV helper methods for api package (#15305)
* Add Read methods for KVClient

* KV write helper

* Add changelog

* Add Delete method

* Use extractVersionMetadata inside extractDataAndVersionMetadata

* Return nil, nil for v1 writes

* Add test for extracting version metadata

* Split kv client into v1 and v2-specific clients

* Add ability to set options on Put

* Add test for KV helpers

* Add custom metadata to top level and allow for getting versions as sorted slice

* Update tests

* Separate KV v1 and v2 into different files

* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity

* Move structs and godoc comments to more appropriate files

* Add more tests for extract methods

* Rework custom metadata helper to be more consistent with other helpers

* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method

* Return early for readability and make test value name less confusing
2022-05-25 11:17:13 -07:00
Brian Kassouf df8ae055be
Add an API for exporting activity log data (#15586)
* Add an API for exporting activity log data

* Add changelog entry

* Switch to error logs
2022-05-24 17:00:46 -07:00
davidadeleon 0026788d4b
api/monitor: Adding log format to monitor command and debug (#15536)
* Correct handling of "unspecified" log level

* Setting log-format default on monitor path

* Create changelog file

* Update website/content/api-docs/system/monitor.mdx

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-24 13:10:53 -04:00
Josh Black ebbb828b80
Update autopilot update interval (#15558) 2022-05-20 15:56:24 -07:00
Josh Black 416504d8c3
Add autopilot automated upgrades and redundancy zones (#15521) 2022-05-20 16:49:11 -04:00
Violet Hynes 6d4497bcbf
VAULT-4306 Ensure /raft/bootstrap/challenge call ignores erroneous namespaces set (#15519)
* VAULT-4306 Ensure /raft/bootstrap/challenge call ignores erroneous namespaces set

* VAULT-4306 Add changelog

* VAULT-4306 Update changelog/15519.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-05-19 16:27:51 -04:00
Nick Cabatoff bc9f69af2e
Forward autopilot state reqs, avoid self-dialing (#15493)
Make sure that autopilot is disabled when we step down from active node state.  Forward autopilot state requests to the active node.  Avoid self-dialing due to stale advertisement.
2022-05-18 14:50:18 -04:00
Pratyoy Mukhopadhyay 62c09bc2be
oss changes (#15487)
* oss changes

* add changelog
2022-05-18 09:16:13 -07:00