Commit graph

2647 commits

Author SHA1 Message Date
Nick Cabatoff 488858e919
Clone created entities that were inserted into memdb... (#16487)
* Clone created entities that were inserted into memdb to prevent possibility of data race.
2022-07-28 09:43:24 -04:00
Nick Cabatoff b9181077fd
Fix a panic at cleanup time in an expiration restore lease benchmark. (#16485) 2022-07-28 05:54:03 -07:00
Hridoy Roy 3429d7dfc5
Current Month Activity Estimate, Probabilistic Tests, And Bugfixes (#16447)
* bugfixes and probabilistic hll tests

* changelog

* changelog fix

* remove activity log test and keep in ent

* update cl
2022-07-26 13:00:27 -07:00
Anton Averchenkov 166c618589
Fix linter issues in policy.go & acl.go (#16366) 2022-07-22 14:13:14 -04:00
Violet Hynes 8163271ee2
VAULT-7046 Allow trailing globbing at the end of a path suffix quota (#16386)
* VAULT-7046 OSS changes for trailing glob quotas

* VAULT-7046 allow glob of 'a*' to match 'a'

* VAULT-7046 Add changelog

* VAULT-7046 fix minor typo
2022-07-21 15:31:23 -04:00
Pratyoy Mukhopadhyay 77ca499c6e
oss changes (#16407) 2022-07-21 10:53:42 -07:00
Austin Gebauer 7df39640e0
Update gopsutil to v3 to fix MacOS deprecation warnings (#16321)
* Update gopsutil to v3

* Adds v2 field names in host-info response to allow eventual deprecation in favor of v3 field names

* Map v3 to v2 field names to keep host-info api compat

* copy gopsutil license into source
2022-07-20 16:37:10 -07:00
Brian Kassouf d6bb62a0ab
Increase the allowed concurrent gRPC streams (#16327)
* Increase the allowed concurrent gRPC streams

* Add a env override for the max streams setting

* Add changelog

* go fmt

* fix builds on 32bit systems
2022-07-20 15:26:52 -04:00
John-Michael Faircloth a5349bd1ef
Revert "AutoMTLS for secrets/auth plugins (#15671)" (#16377)
This reverts commit 39bcd5c71529f5f4eb61aae68b17d06d192ea55f.
2022-07-20 10:36:23 -05:00
John-Michael Faircloth 7e170e7d87
AutoMTLS for secrets/auth plugins (#15671)
* use automtls for v5 secrets/auth plugins

* add automtls env guard

* start backend without metadata mode

* use PluginClientConfig for backend's NewPluginClient param

refactor

* - fix pluginutil test
- do not expect plugin to be unloaded in UT
- fix pluginutil tests --need new env var
- use require in UT
- fix lazy load test

* add changelog

* prioritize automtls; improve comments

* user multierror; refactor pluginSet for v4 unit test

* add test cases for v4 and v5 plugin versions

* remove unnecessary call to AutoMTLSSupported

* update comment on pluginSets

* use runconfig directly in sdk newpluginclient

* use automtls without metadatamode for v5 backend plugin registration

* use multierror for plugin runconfig calls

* remove some unnecessary code
2022-07-18 16:25:18 -05:00
Mike Palmiotto 439e35f50f
Vault 6773/raft rejoin nonvoter (#16324)
* raft: Ensure init before setting suffrage

As reported in https://hashicorp.atlassian.net/browse/VAULT-6773:

	The /sys/storage/raft/join endpoint is intended to be unauthenticated. We rely
	on the seal to manage trust.

	It’s possible to use multiple join requests to switch nodes from voter to
	non-voter. The screenshot shows a 3 node cluster where vault_2 is the leader,
	and vault_3 and vault_4 are followers with non-voters set to false.  sent two
	requests to the raft join endpoint to have vault_3 and vault_4 join the cluster
	with non_voters:true.

This commit fixes the issue by delaying the call to SetDesiredSuffrage until after
the initialization check, preventing unauthenticated mangling of voter status.

Tested locally using
https://github.com/hashicorp/vault-tools/blob/main/users/ncabatoff/cluster/raft.sh
and the reproducer outlined in VAULT-6773.

* raft: Return join err on failure

This is necessary to correctly distinguish errors returned from the Join
workflow. Previously, errors were being masked as timeouts.

* raft: Default autopilot parameters in teststorage

Change some defaults so we don't have to pass in parameters or set them
in the originating tests. These storage types are only used in two
places:

1) Raft HA testing
2) Seal migration testing

Both consumers have been tested and pass with this change.

* changelog: Unauthn voter status change bugfix
2022-07-18 14:37:12 -04:00
Hridoy Roy 573e01af1d
Throw SSCT Error In Some Cases (#16270)
* throw the ssct error if it signifies that the server needs to take an action

* use errors.Is instead of checking string comparison
2022-07-11 11:12:02 -07:00
Chris Capurso 068a413311
remove GetCoreConfigInternal from logger API tests (#16263) 2022-07-08 19:23:18 +02:00
Violet Hynes 0c80ee5cf5
VAULT-6614 Enable role based quotas for lease-count quotas (OSS) (#16157)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* VAULT-6614 start of changes for roles in LCQs

* Expiration changes for leases

* Add role information to RequestAuth

* VAULT-6614 Test updates

* VAULT-6614 Add expiration test with roles

* VAULT-6614 fix comment

* VAULT-6614 Protobuf on OSS

* VAULT-6614 Add rlock to determine role code

* VAULT-6614 Try lock instead of rlock

* VAULT-6614 back to rlock while I think about this more

* VAULT-6614 Additional safety for nil dereference

* VAULT-6614 Use %q over %s

* VAULT-6614 Add overloading to plugin backends

* VAULT-6614 RLocks instead

* VAULT-6614 Fix return for backend factory
2022-07-05 13:02:00 -04:00
AnPucel 3215cdbd32
Dynamic parameter for mountpaths in OpenApi Spec generation(#15835)
"generic_mount_paths" query parameter for OpenApiSpec generation
2022-06-30 07:43:04 -07:00
Hridoy Roy 2f14d60a4b
Port: Use Stored Hll to Compute New Clients For Current Month (#16184)
* port hll storage changes

* changelog
2022-06-29 10:51:23 -07:00
Hamid Ghaf fa754c7fa5
Replicate member_entity_ids and policies in identity/group across nodes identically (#16088)
* Replicate values of group member_entity_ids and policies across nodes identically

* Adding CL

* fixing tests
2022-06-28 19:54:24 -04:00
Hridoy Roy 895e422c4c
move unused GetCoreConfigInternal to common file to prevent ent panics (#16165) 2022-06-27 14:41:56 -07:00
Hridoy Roy b3959534c2
activity log refactoring port (#16162)
* activity log refactoring port

* changelog
2022-06-27 13:33:45 -07:00
akshya96 42b13448f9
ActivityLog Implement HyperLogLog Store Functionality During Precomputation (#16146)
* adding hll for each month

* add changelog

* removing influxdb

* removing influxdb

* removing influxdb

* changing switch to if-else for semgrep
2022-06-27 09:38:32 -07:00
Chris Capurso 9501d44ed5
Add endpoints to provide ability to modify logging verbosity (#16111)
* add func to set level for specific logger

* add endpoints to modify log level

* initialize base logger with IndependentLevels

* test to ensure other loggers remain unchanged

* add DELETE loggers endpoints to revert back to config

* add API docs page

* add changelog entry

* remove extraneous line

* add log level field to Core struct

* add godoc for getLogLevel

* add some loggers to c.allLoggers
2022-06-27 11:39:53 -04:00
Violet Hynes 5d21fa1e8f
VAULT-6613 Missed a part of OSS for rate limit role quotas (#16132)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* Handle body error

* VAULT-6613 Return early if error with json parsing

* VAULT-6613 add to teardown function
2022-06-24 09:45:53 -04:00
Violet Hynes d57fea2cd1
VAULT-6613 Add role support for rate limit quotas (OSS Changes) (#16115)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* Handle body error

* VAULT-6613 Return early if error with json parsing
2022-06-24 08:58:02 -04:00
Josh Black 2ee2b6ed7c
Return a 403 for a bad SSCT instead of 500 (#16112) 2022-06-23 13:01:20 -07:00
Jason O'Donnell f957573108
Fix bug where id not existing in multiplexing map causes panic (#16094)
* multiplexing: guard against connection panic

* changelog

* Update vault/plugin_catalog.go

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2022-06-22 14:29:25 -04:00
Violet Hynes 56ed9ca8ae
Start of implementation of the plumbing for role resolution logic on auth mounts (#16049)
* VAULT-6612 Initial scaffolding for role determination

* VAULT-6612 Simplify code

* Fix fmt error that somehow happened

* VAULT-6612 Refactor resolve role response
2022-06-21 09:31:36 -04:00
Nick Cabatoff 27ea9a0e41
Set NumCores=1 since we only return a single client anyway. (#16039) 2022-06-17 11:28:25 -04:00
Violet Hynes 96371ed0a5
Small refactor of mount entry manipulation (#16024) 2022-06-17 08:52:43 -04:00
Violet Hynes abed5cf6e7
(OSS) Path Suffix Support for Rate Limit Quotas (#15989)
* Support for rate limit path suffix quotas

* Support for rate limit path suffix quotas

* Precedence test for support for rate limit path suffix quotas

* Update clone method

* Fix mount determination

* Add changelog

* use constant for mounts

* Fix read endpoint, and remount/disable mount

* update godocs for queryquota
2022-06-16 13:23:02 -04:00
akshya96 7e313e29fd
Activity Log Filtering Limit Parameter (#16000)
* adding changes from ent branch

* adding fmt changes

* adding changelog
2022-06-15 15:41:31 -07:00
swayne275 54262d2f4e
clarify lazy revoke prefix if not sync (#15967)
* clarify lazy revoke prefix if not sync

* comment improvement
2022-06-15 12:03:56 -06:00
Josh Black d2ed39a04e
Correct drift between ENT and OSS (#15966) 2022-06-14 17:53:19 -07:00
Chris Capurso 94c5936e27
return bad request instead of server error for identity group cycle detection (#15912)
* return bad request for identity group cycle detection

* add changelog entry

* use change release note instead of improvement

* fix err reference

* fix TestIdentityStore_GroupHierarchyCases
2022-06-10 10:15:31 -04:00
Hridoy Roy 934989809b
Limit SSCT WAL Check on Perf Standbys to Raft Backends Only (#15879)
* ensure that ssct wal check only occurs for non-raft storage on perf standbys

* changelog
2022-06-08 13:58:22 -07:00
VAL 98ed9c7ab9
Add PutMetadata and PatchMetadata KV helpers (#15755)
* Add PutMetadata and PatchMetadata KV helpers

* Refactor tests, stop needlessly converting zero values to nil

* Merge fix for delete-version-after zero value bug

* Cast nils to zero value on Put

* Use 2 different structs for put/patch metadata input, clarity fixes

* Use local constants and simplify stringification
2022-06-08 09:29:22 -07:00
Alexander Scheel b3ad79fb70
Fix listing_visibility value documentation (#15833)
* Match listing_visibility in system/auth with system/mounts

See also: #15209

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix path-help for listing_visibility

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-06-06 16:44:46 -04:00
Josh Black 9c48c62d6e
Use the incoming request version to populate follower state (#15806) 2022-06-06 08:44:24 -07:00
Violet Hynes d62b140b7c
VAULT-6371 Fix issue with lease quotas on read requests that generate leases (#15735)
* VAULT-6371 Fix issue with lease quotas on non-auth mounts

* VAULT-6371 Add changelog

* VAULT-6371 Amend changelog given new understanding
2022-06-03 15:45:21 -04:00
VAL 3cfafe619b
KV helpers for DeleteMetadata, Undelete, Destroy, and Rollback (#15637)
* KV helpers for DeleteMetadata, Undelete, Destroy, and Rollback

* Allow rollback when no secret data on latest version, and update error messages
2022-06-03 10:42:43 -07:00
Hridoy Roy e64d7df041
refactor some code in modifyResponseMonths and ensure that the last mo… (#15767)
* refactr some code in modifyResponseMonths and ensure that the last month comparison with end is comparing end of month with end of month

* calibrate end of month apropriately and fix parens issue for lastmonth
2022-06-03 10:34:54 -07:00
Hridoy Roy 671aaf1fe0
iterate through all available logs for precomputation and query gets (#15768) 2022-06-03 09:53:53 -07:00
Hridoy Roy a5f70d7fe0
fix off by one error in activity log nil padding for month data (#15731) 2022-06-01 11:09:06 -07:00
VAL ed7c1d4800
Add Patch KV helper (#15587)
* Add Read methods for KVClient

* KV write helper

* Add changelog

* Add Delete method

* Use extractVersionMetadata inside extractDataAndVersionMetadata

* Return nil, nil for v1 writes

* Add test for extracting version metadata

* Split kv client into v1 and v2-specific clients

* Add ability to set options on Put

* Add test for KV helpers

* Add custom metadata to top level and allow for getting versions as sorted slice

* Update tests

* Separate KV v1 and v2 into different files

* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity

* Move structs and godoc comments to more appropriate files

* Add more tests for extract methods

* Rework custom metadata helper to be more consistent with other helpers

* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method

* Add Patch KV helper

* Add godoc comment and use WithOption ourselves in other KVOption functions

* Clean up options-handling and resp parsing logic; add more tests

* Add constants and more patch tests
2022-06-01 07:50:56 -07:00
Hamid Ghaf bf087f9d0d
prevent deleting MFA method through an invalid path (#15482)
* prevent deleting MFA method through an invalid path

* Adding CL
2022-05-31 14:22:04 -04:00
Nick Cabatoff 69c5e8c946
Avoid deadlocking on stateLock in emitMetrics (#15693)
When stopCh is closed we should stop trying to get the lock.
2022-05-31 12:15:39 -04:00
Violet Hynes 4aac96238c
VAULT-6131 OpenAPI schema now includes /auth/token endpoints when explicit permission has been granted (#15552)
* VAULT-6131 OpenAPI schema now includes /auth/token endpoints when explicit permission has been granted

* VAULT-6131 add changelog

* VAULT-6131 Update changelog and fix related bug
2022-05-31 11:25:27 -04:00
Nick Cabatoff ea099fdffd
Like #15682 but this time fix the correct test, i.e. TestRateLimitQuota_Allow (#15684) 2022-05-30 15:08:01 -04:00
Nick Cabatoff 8c2b69f961
Spawn fewer goroutines to make TestRateLimitQuota_Allow less vulnerable to scheduler. (#15682) 2022-05-30 13:33:41 -04:00
Chris Capurso cdb73ab265
use provided namespace for wrapping lookup cubbyhole request (#15583)
* use provided namespace for wrapping lookup cubbyhole request

* add changelog entry
2022-05-26 15:17:29 -04:00
Peter Wilson b7fc4645f3
Only add distinct policies to identity group (#15638)
* Only add distinct policies to identity group
2022-05-26 13:52:19 +01:00
John-Michael Faircloth fc04699f57
Fix plugin reload mounts (#15579)
* fix plugin reload mounts

* do not require sys/ prefix

* update plugin reload docs with examples

* fix unit test credential read path

* update docs to reflect correct cli usage

* allow sys/auth/foo or auth/foo

* append trailing slash if it doesn't exist in request

* add changelog

* use correct changelog number
2022-05-25 13:37:42 -05:00
VAL 64448b62a4
KV helper methods for api package (#15305)
* Add Read methods for KVClient

* KV write helper

* Add changelog

* Add Delete method

* Use extractVersionMetadata inside extractDataAndVersionMetadata

* Return nil, nil for v1 writes

* Add test for extracting version metadata

* Split kv client into v1 and v2-specific clients

* Add ability to set options on Put

* Add test for KV helpers

* Add custom metadata to top level and allow for getting versions as sorted slice

* Update tests

* Separate KV v1 and v2 into different files

* Add test for GetVersionsAsList, rename Metadata key to VersionMetadata for clarity

* Move structs and godoc comments to more appropriate files

* Add more tests for extract methods

* Rework custom metadata helper to be more consistent with other helpers

* Remove KVSecret from custom metadata test now that we don't append to it as part of helper method

* Return early for readability and make test value name less confusing
2022-05-25 11:17:13 -07:00
Brian Kassouf df8ae055be
Add an API for exporting activity log data (#15586)
* Add an API for exporting activity log data

* Add changelog entry

* Switch to error logs
2022-05-24 17:00:46 -07:00
davidadeleon 0026788d4b
api/monitor: Adding log format to monitor command and debug (#15536)
* Correct handling of "unspecified" log level

* Setting log-format default on monitor path

* Create changelog file

* Update website/content/api-docs/system/monitor.mdx

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-24 13:10:53 -04:00
Josh Black ebbb828b80
Update autopilot update interval (#15558) 2022-05-20 15:56:24 -07:00
Josh Black 416504d8c3
Add autopilot automated upgrades and redundancy zones (#15521) 2022-05-20 16:49:11 -04:00
Violet Hynes 6d4497bcbf
VAULT-4306 Ensure /raft/bootstrap/challenge call ignores erroneous namespaces set (#15519)
* VAULT-4306 Ensure /raft/bootstrap/challenge call ignores erroneous namespaces set

* VAULT-4306 Add changelog

* VAULT-4306 Update changelog/15519.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-05-19 16:27:51 -04:00
Nick Cabatoff bc9f69af2e
Forward autopilot state reqs, avoid self-dialing (#15493)
Make sure that autopilot is disabled when we step down from active node state.  Forward autopilot state requests to the active node.  Avoid self-dialing due to stale advertisement.
2022-05-18 14:50:18 -04:00
Pratyoy Mukhopadhyay 62c09bc2be
oss changes (#15487)
* oss changes

* add changelog
2022-05-18 09:16:13 -07:00
Hamid Ghaf 66c6de50a7
Username format login mfa (#15363)
* change username_template to username_format for login MFA

* fixing a test

* Update website/content/docs/auth/login-mfa/faq.mdx

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2022-05-17 16:31:50 -04:00
Hamid Ghaf 77be41c83d
possibly forward cached MFA auth response to leader (#15469)
* possibly forward cached MFA auth response to leader

* adding CL
2022-05-17 16:30:36 -04:00
Hridoy Roy 679ccc81a5
Query and Precompute Non-Contiguous Segments in the Activity Log (#15352)
* query and precompute non-contiguous segments in the activity log

* changelog

* newline formatting

* make fmt

* report listener and storage types as found keys

* report listener and storage types as found keys

* Update vault/activity_log_test.go

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

* review comments

* merge conflict

* merge conflict

* merge conflict

* fix unchecked merge conflict

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-17 12:17:32 -07:00
Chris Hoffman 24e8b73c73
Updating Okta MFA to use official SDK (#15355)
* updating MFA to use official Okta SDK

* add changelog

* Update vault/login_mfa.go

Co-authored-by: swayne275 <swayne@hashicorp.com>

* cleanup query param building

* skip if not user factor

* updating struct tags to be more explicit

* fixing incorrect merge

* worrying that URL construction may change in the future, reimplementing GetFactorTransactionStatus

* adding some safety around url building

Co-authored-by: swayne275 <swayne@hashicorp.com>
2022-05-17 15:14:26 -04:00
Hamid Ghaf 364f8789cd
Globally scoped MFA method Get/List endpoints (#15248)
* Globally scoped MFA method Get/List endpoints

* Adding CL

* minor changes

* removing unwanted information from an error msg
2022-05-17 14:54:16 -04:00
akshya96 4e9e9b7eda
Vault-6037 making filesystem permissions check opt-in (#15452)
* adding env var changes

* adding changelog

* adding strcov.ParseBool
2022-05-17 11:34:31 -07:00
Brian Kassouf b5472aadf3
Add list of granting policies audit logs (#15457)
* Add list of granting policies audit logs

* Add changelog
2022-05-16 16:23:08 -07:00
Chris Hoffman b14dc0d95d Remove duplicate policies when creating/updating identity groups (#15055)
* Remove duplicate policies for identity groups

* adding changelog

* test cleanup
2022-05-16 17:20:48 -04:00
Hridoy Roy 90538739bd
append nil months to query get to cover all requested months (OSS) (#15420)
* fill out nil response months in activity log query handle response based on requested month data

* changelog

* reverse month ordering for nil end months

* typo caught in ent test
2022-05-16 13:01:28 -07:00
Chris Hoffman 63b31a13f9
Disabling client side rate limiting in Okta login MFA client (#15369)
* disabling client side rate limiting for MFA client

* add changelog
2022-05-12 15:55:33 -04:00
John-Michael Faircloth 45efa37c4a
unit test: fix oidc periodicfunc flaky test (#15320)
* unit test: fix oidc periodicfunc flaky test

* update cycle 1 for two test cases
2022-05-09 13:43:23 -05:00
Hamid Ghaf 2ee602cfdd
removing prem/pro references as part of removing some build targets (#15278) 2022-05-06 09:09:42 -04:00
Hamid Ghaf 0fc7a363cc
loading MFA configs upont restart (#15261)
* loading MFA configs upont restart

* Adding CL

* feedback

* Update vault/core.go

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-05 18:53:57 -04:00
Christopher Swenson 0affe226ad
Update deps for consul-template 0.29.0 (#15293)
This requires bumping https://github.com/mitchellh/go-testing-interface.
For this new version, we have to create a wrapper to convert
the stdlib `testing.TB` interface to the
`mitchellh/go-testing-interface` `T` interface, since it uses
`Parallel()` now, which is not supported by `testing.TB`. This had to be
added to a new package, `benchhelpers`, to avoid a circular dependency
in `testhelpers`.

We also have to *unbump* https://github.com/armon/go-metrics since
updating it breaks our usage of
https://github.com/google/go-metrics-stackdriver

I verified that the new `pkiCert` template function works with agent
injection using annotations like:

```yaml
vault.hashicorp.com/agent-inject-secret-sample.crt: "pki/issue/example-dot-com"
vault.hashicorp.com/agent-inject-template-sample.crt: |
  {{ pkiCert "pki/issue/example-dot-com" "common_name=foo.example.com" "ttl=1h" }}
```
2022-05-05 10:30:40 -07:00
Vinayak 988468a67b
Skip metric increment during existence check (#12763)
* Skip metric increment during existence check

Signed-off-by: Vinayak Kadam <kadamvinayak03@gmail.com>

* Adding changelog.txt

Signed-off-by: Vinayak Kadam <kadamvinayak03@gmail.com>

* Updated changelog text

Signed-off-by: Vinayak Kadam <kadamvinayak03@gmail.com>
2022-05-05 10:22:19 -07:00
Hridoy Roy 9a5b46436e
change ordering of activity log month data to sort by ascending order… (#15259)
* change ordering of activity log month data to sort by ascending order of timestamp

* changelog

* changelog
2022-05-03 13:39:29 -07:00
Scott Miller bef350c916
Allow callers to choose the entropy source for the random endpoints. (#15213)
* Allow callers to choose the entropy source for the random endpoints

* Put source in the URL for sys as well

* changelog

* docs

* Fix unit tests, and add coverage

* refactor to use a single common implementation

* Update documentation

* one more tweak

* more cleanup

* Readd lost test expected code

* fmt
2022-05-02 14:42:07 -05:00
Steven Clark 6d9888e09b
Allow looking up mount entries by their backend UUIDs (#15217) 2022-04-29 16:15:29 -04:00
VAL a06c8a139f
Add enterprise sudo paths to api.SudoPaths map (#15219)
* Add enterprise sudo paths to api.SudoPaths map

* add comment to denote ent-only sudo paths

* go fmt

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-04-29 10:09:25 -04:00
Nick Cabatoff c5928c1d15
Raft: use a larger initial heartbeat/election timeout (#15042) 2022-04-29 08:32:16 -04:00
VAL 0ef529b710
Global flag that outputs minimum policy HCL required for an operation (#14899)
* WIP: output policy

* Outputs example policy HCL for given request

* Simplify conditional

* Add PATCH capability

* Use OpenAPI spec and regex patterns to determine if path is sudo

* Add test for isSudoPath

* Add changelog

* Fix broken CLI tests

* Add output-policy to client cloning code

* Smaller fixes from PR comments

* Clone client instead of saving and restoring custom values

* Fix test

* Address comments

* Don't unset output-policy flag on KV requests otherwise the preflight request will fail and not populate LastOutputPolicyError

* Print errors saved in buffer from preflight KV requests

* Unescape characters in request URL

* Rename methods and properties to improve readability

* Put KV-specificness at front of KV-specific error

* Simplify logic by doing more direct returns of strings and errors

* Use precompiled regexes and move OpenAPI call to tests

* Remove commented out code

* Remove legacy MFA paths

* Remove unnecessary use of client

* Move sudo paths map to plugin helper

* Remove unused error return

* Add explanatory comment

* Remove need to pass in address

* Make {name} regex less greedy

* Use method and path instead of info from retryablerequest

* Add test for IsSudoPaths, use more idiomatic naming

* Use precompiled regexes and move OpenAPI call to tests (#15170)

* Use precompiled regexes and move OpenAPI call to tests

* Remove commented out code

* Remove legacy MFA paths

* Remove unnecessary use of client

* Move sudo paths map to plugin helper

* Remove unused error return

* Add explanatory comment

* Remove need to pass in address

* Make {name} regex less greedy

* Use method and path instead of info from retryablerequest

* Add test for IsSudoPaths, use more idiomatic naming

* Make stderr writing more obvious, fix nil pointer deref
2022-04-27 16:35:18 -07:00
Rémi Lapeyre 089b6ea970
Remove dead code in setupCredentials() (#15194)
This should have been removed as part of f09e39ea42 but somehow got
forgotten.
2022-04-27 10:47:04 -04:00
Josh Black a4593e8913
When tainting a route during setup, pre-calculate the namespace specific path (#15067) 2022-04-26 09:13:45 -07:00
Chris Capurso cc531c793d
fix raft tls key rotation panic when rotation time in past (#15156)
* fix raft tls key rotation panic when rotation time in past

* add changelog entry

* push out next raft TLS rotation time in case close to elapsing

* consolidate tls key rotation duration calculation

* reduce raft getNextRotationTime padding to 10 seconds

* move tls rotation ticker reset to where its duration is calculated
2022-04-25 21:48:34 -04:00
Nick Cabatoff 7e64e105a0
Clone identity objects to prevent races. (#15123) 2022-04-22 13:04:34 -04:00
Chris Capurso e69f89c279
Add build date (#14957)
* add BuildDate to version base

* populate BuildDate with ldflags

* include BuildDate in FullVersionNumber

* add BuildDate to seal-status and associated status cmd

* extend core/versions entries to include BuildDate

* include BuildDate in version-history API and CLI

* fix version history tests

* fix sys status tests

* fix TestStatusFormat

* remove extraneous LD_FLAGS from build.sh

* add BuildDate to build.bat

* fix TestSysUnseal_Reset

* attempt to add build-date to release builds

* add branch to github build workflow

* add get-build-date to build-* job needs

* fix release build command vars

* add missing quote in release build command

* Revert "add branch to github build workflow"

This reverts commit b835699ecb7c2c632757fa5fe64b3d5f60d2a886.

* add changelog entry
2022-04-19 14:28:08 -04:00
Vinny Mannello 6116903a37
[Vault-5248] MFA support for api login helpers (#14900)
* Add MFA support to login helpers
2022-04-15 11:13:15 -07:00
John-Michael Faircloth b9acac050a
fix regression on Core.pluginDirectory when backend is builtin (#15027)
* fix dev-plugin-dir when backend is builtin

* use builtinRegistry.Contains

* revert aa76337

* use correct plugin type for logical backend after revert

* fix factory func default setting after revert

* add ut coverage for builtin plugin with plugin directory set

* add coverage for secrets plugin type

* use totp in tests to avoid test import cycle in ssh package

* use nomad in tests to avoid test import cycle

* remove secrets mount tests due to unavoidable test import cycle
2022-04-14 15:54:23 -05:00
Hamid Ghaf a1d73ddfec
VAULT-5422: Add rate limit for TOTP passcode attempts (#14864)
* VAULT-5422: Add rate limit for TOTP passcode attempts

* fixing the docs

* CL

* feedback

* Additional info in doc

* rate limit is done per entity per methodID

* refactoring a test

* rate limit OSS work for policy MFA

* adding max_validation_attempts to TOTP config

* feedback

* checking for non-nil reference
2022-04-14 13:48:24 -04:00
Hamid Ghaf a12271af46
forwarding requests subjected to Login MFA to the active node (#15009)
* forwarding requests subjected to Login MFA to the active node

* CL, and making fmt happy
2022-04-13 10:11:53 -04:00
Anton Averchenkov 23a3f950e4
Revert the WithContext changes to vault tests (#14947) 2022-04-07 15:12:58 -04:00
Austin Gebauer f9ce8d7615
Adds Vault version prerelease and metadata to logical.PluginEnvironment (#14851) 2022-04-04 22:31:01 -07:00
akshya96 796003ddda
Vault 3992 ToB Config and Plugins Permissions (#14817)
* updating changes from ent PR

* adding changelog

* fixing err

* fixing semgrep error
2022-04-04 09:45:41 -07:00
hghaf099 aafb5d6427
VAULT-4240 time.After() in a select statement can lead to memory leak (#14814)
* VAULT-4240 time.After() in a select statement can lead to memory leak

* CL
2022-04-01 10:17:11 -04:00
Steven Clark 77feaad6fe
Address incorrect table metric value for local mounts (#14755)
* Address incorrect table metric value for local mounts

 - Reported within issue #14750 as a panic, it was identified that
   we were using the wrong value for local mounts within the table metrics.

* Add changelog
2022-03-30 13:06:49 -04:00
Chris Capurso 2354435262
treat logical.ErrRelativePath as 400 instead of 500 (#14328)
* treat logical.ErrRelativePath as 400 instead of 500

* add changelog entry

* return UserError for logical.ErrRelativePath
2022-03-30 09:08:02 -04:00
VAL f9372145dd
Warn on upper case in policy name (#14670)
* Warn on upper case in policy name

* Rename name variable to be less confusing

* Use more general solution for other string issues

* Clarify changelog

* Remove unnecessary check

* Don't throw CLI warning until after past errors

* Add before and after names with quotes to show spacing changes
2022-03-24 13:29:11 -07:00
Anton Averchenkov 1222375d1a
Add context-aware functions to vault/api (#14388) 2022-03-23 17:47:43 -04:00
Nick Cabatoff 34af6bab1e
Add a check for missing entity during local alias invalidation. (#14622) 2022-03-21 15:09:31 -04:00
John-Michael Faircloth 3c2659b09d
oidc provider: add test case for clients sharing keys (#14555) 2022-03-18 16:21:21 -05:00
Hridoy Roy 7e0abe3c7e
Add Semgrep Rules to OSS (#14513)
* add semgrep yml

* add semgrep ci job

* remove replication semgrep rule in oss

* fix makefile

* add semgrep to ci

* upwind triple if in ui.go semgrep refactoring
2022-03-18 11:14:03 -07:00
Josh Black 92b97b19ed
sys/mfa/validate should forward (#14566) 2022-03-18 10:14:17 -07:00
Jason O'Donnell 219df7087c
identity/token: fix duplicate keys in well-known (#14543)
* identity/token: fix duplicate kids in well-known

* Remove unused check

* changelog

* use map-based approach to dedup key IDs

* improve changelog description

* move jwks closer to usage; specify capacity

Co-authored-by: Austin Gebauer <agebauer@hashicorp.com>
2022-03-16 18:48:10 -07:00
Lars Lehtonen 50a9dedbcb
vault/external_tests/raft: fix dropped test error (#14519) 2022-03-16 09:32:57 -06:00
Nick Cabatoff 9e18350cf4
Handle the empty mount accessor case. (#14507) 2022-03-16 09:28:05 -04:00
Hridoy Roy 1558387af4
port of semgrep fixes oss (#14488) 2022-03-15 13:17:55 -07:00
Pratyoy Mukhopadhyay d222981cec
Fixes from mount move testing (#14492)
* Add validation, fix docs

* add changelog

* fmt fix

* Update vault/logical_system.go

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>

* Update vault/logical_system.go

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>

* Update vault/logical_system_test.go

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>

* Update vault/logical_system_test.go

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
2022-03-15 11:11:23 -07:00
hghaf099 361646ab26
add MFA validation support to vault login command (#14425)
* add MFA validation support to vault login command

* correctly report new totp code availability period
2022-03-14 15:54:41 -04:00
Vinny Mannello 2290ca5e83
[VAULT-5003] Use net/http client in Sys().RaftSnapshotRestore (#14269)
Use net/http client when body could be too big for retryablehttp client
2022-03-14 10:13:33 -07:00
Anton Averchenkov c425078008
Change OpenAPI code generator to extract request objects (#14217) 2022-03-11 19:00:26 -05:00
Austin Gebauer d016b67915
identity/oidc: prevent key rotation on performance secondary clusters (#14426) 2022-03-09 15:41:02 -08:00
hghaf099 b358bd6ffa
remove mount accessor from MFA config (#14406)
* remove mount accessor from MFA config

* Update login_mfa_duo_test.go

* DUO test with entity templating

* using identitytpl.PopulateString to perform templating

* minor refactoring

* fixing fmt failures in CI

* change username format to username template

* fixing username_template example
2022-03-09 09:14:30 -08:00
Lars Lehtonen b9a6ec67c9
vault: fix dropped test errors (#14402) 2022-03-08 12:32:27 -07:00
John-Michael Faircloth baafd9ff38
plugin multiplexing: add catalog test coverage (#14398)
* plugin client and plugin catalog tests

* add v5 plugin cases and more checks

* improve err msg

* refactor tests; fix test err msg
2022-03-08 10:33:24 -06:00
Josh Black 1f8d282c2b
fix version check (#14395) 2022-03-07 15:42:06 -08:00
Josh Black 5c43bf4864
Only create new batch tokens if we're on at least 1.10.0 (#14370) 2022-03-04 14:16:51 -08:00
Hridoy Roy d8155aa7c4
SSCT Optimizations (OSS) (#14323)
* update ci.hcl to remove 1.6.x and add in 1.10.x

* SSCT OSS PR review comments and optimizations

* check errors in populate token entry calls
2022-03-01 12:24:45 -08:00
Scott Miller ba49365ecb
Fix autoseal health check race by passing metrics sink in CoreConfig (#14196) 2022-03-01 09:00:39 -05:00
Pratyoy Mukhopadhyay 69c22b8078
Fix raft paralle retry bug (#14303) 2022-02-28 10:38:34 -08:00
Josh Black c9d5734924
Add login enforcement listing (#14284) 2022-02-25 13:48:28 -08:00
Steven Clark 6f8e3c618b
Address slow CI causing failures in TestRateLimitQuota_Allow_WithBlock (#14042)
* Address slow CI causing failures in TestRateLimitQuota_Allow_WithBlock

 - An attempt to fix CI runs that are extremely slow and the for loop
   runs across two BlockIntervals within the rate limit window of operation.
 - Increasing BlockInterval was looked at but the normal test times would
   be increased due to us also validating that we are releasing clients post
   BlockInterval.

* Address TestRateLimitQuota_Allow_WithBlock slowness issue (take 2)

 - Increase the overall Interval value within the blocking test so that we
   should always be able to request at least 17 requests within the interval
   value.
 - Tested by changing the time.Sleep within the for loop to 20 from 2
   and could see that within the response, coming back from the rlq.allow,
   that we were no longer being rate limited by going over the 1 second
   interval value per host.

* Limit the number of active go routines in TestRateLimitQuota_Allow_WithBlock
2022-02-25 16:05:44 -05:00
hghaf099 671cdbcadb
interactive CLI for mfa login (#14131)
* Login MFA

* ENT OSS segragation (#14088)

* Delete method id if not used in an MFA enforcement config (#14063)

* Delete an MFA methodID only if it is not used by an MFA enforcement config

* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path

* adding use_passcode field to DUO config (#14059)

* add changelog

* preventing replay attack on MFA passcodes (#14056)

* preventing replay attack on MFA passcodes

* using %w instead of %s for error

* Improve CLI command for login mfa (#14106)

CLI prints a warning message indicating the login request needs to get validated

* adding the validity period of a passcode to error messages (#14115)

* interactive CLI for mfa login

* minor fixes

* bail if no input was inserted

* change label name

* interactive CLI when single methodID is returned from login request

* minor fix

* adding changelog

* addressing feedback

* a user with a terminal should be able to choose between interactive and non-interactive.  A user without a terminal should not be able to use the interactive mode.

Co-authored-by: Josh Black <raskchanky@gmail.com>
2022-02-24 15:16:15 -05:00
Hridoy Roy 3438f5dbae
Fix ent diff check 2022/02/23 (#14237)
* achieve parity with ent in core.go

* add VAULT_DISABLE_LOCAL_AUTH_MOUNT_ENTITIES

* parity in build.yml with ent but without adding the +ent

* pass base version to ldflags

Co-authored-by: Kyle Penfound <kpenfound11@gmail.com>
2022-02-24 11:57:40 -08:00
Nick Cabatoff 0421b6895f
Remove use of compressionutil from activitylog segments. We can leave it in for the precomputed queries since they're JSON, not protobuf. (#14239) 2022-02-23 17:33:24 -05:00
Vishal Nayak 35d3d4226d
Minor fixes in response (#14230) 2022-02-23 13:17:53 -05:00
Nick Cabatoff 2551a3e8ce
Ensure that fewer goroutines survive after a test completes (#14197)
* Various changes to try to ensure that fewer goroutines survive after a test completes:
* add Core.ShutdownWait that doesn't return until shutdown is done
* create the usedCodes cache on seal and nil it out on pre-seal so that the finalizer kills the janitor goroutine
* stop seal health checks on seal rather than wait for them to discover the active context is done
* make sure all lease-loading goroutines are done before returning from restore
* make uniquePoliciesGc discover closed quitCh immediately instead of only when the ticker fires
* make sure all loading goroutines are done before returning from loadEntities, loadCachedEntitiesOfLocalAliases
2022-02-23 10:33:52 -05:00
mickael-hc 6b753fb2f5
Replace dgrijava/jwt-go with golang-jwt/jwt (#14167) 2022-02-22 15:12:15 -05:00
Austin Gebauer 462a924722
identity/oidc: Adds default provider, key, and allow_all assignment (#14119) 2022-02-22 08:33:19 -08:00
Victor Rodriguez f7f6f937e4
Invalidate the ManagedKeyRegistry cache when Vault config is updated. (#14179)
Add the hook needed to implement this functionality in vault-enterprise.
2022-02-21 09:55:44 -05:00
Pratyoy Mukhopadhyay c8cca2cab5
add missed test (#14168) 2022-02-18 14:01:43 -08:00
Vishal Nayak e29cc5989f
s/path/mount_path (#14164) 2022-02-18 13:44:43 -05:00
Chris Capurso 9e2f6f3b4f
wrap kv reqs with kvRequestWithRetry in external tests (#14163) 2022-02-18 13:24:46 -05:00
Vishal Nayak 6898e038ec
OSS Port: Activity log breakdowns (#14160)
* Activity log breakdowns

* s/path/namespace_path

* Remove ent specific tests
2022-02-18 13:01:28 -05:00
Chris Capurso c7ba8b1f25
Add HTTP-level tests for KV subkeys endpoint (#14079)
* go get plugin-secrets-kv@v0.11.0; go mod tidy

* add HTTP-level tests for KV subkeys endpoint

* check status in TestKV_Subkeys_CurrentVersion

* some test cleanup
2022-02-18 11:54:36 -05:00
Pratyoy Mukhopadhyay 475b55b460
Vault 4632 auth remount oss (#14141)
* Update plugin-portal.mdx (#13229)

Add a Vault plugin to allow authentication via SSH certificates and public keys

* oss changes

Co-authored-by: Wim <wim@42.be>
2022-02-18 08:04:21 -08:00
swayne275 e66e6b920c
validate cipher length before decrypting (#14098)
* validate cipher length before decrypting

* also protect decrypt from short cipher
2022-02-18 07:37:22 -07:00
swayne275 24d512f0d1
Cleanup bool checks (#14102)
* clean up == true cases

* cleanup == false where it didn't seem to hurt readability
2022-02-18 07:35:53 -07:00
Calvin Leung Huang 2a937fe717
plugin/catalog: support plugin registration when type is explicitly provided (#14142)
* plugin/catalog: support plugin registration whe type is explicitly provided

* don't use database type on plugin backend test; mock doesn't satisfy the DB interface

* check multiplexing support from plugin directly on newPluginClient

* do not return mutiplexed bool on catalog helper funcs
2022-02-17 18:40:33 -08:00
Josh Black e83471d7de
Login MFA (#14025)
* Login MFA

* ENT OSS segragation (#14088)

* Delete method id if not used in an MFA enforcement config (#14063)

* Delete an MFA methodID only if it is not used by an MFA enforcement config

* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path

* adding use_passcode field to DUO config (#14059)

* add changelog

* preventing replay attack on MFA passcodes (#14056)

* preventing replay attack on MFA passcodes

* using %w instead of %s for error

* Improve CLI command for login mfa (#14106)

CLI prints a warning message indicating the login request needs to get validated

* adding the validity period of a passcode to error messages (#14115)

* PR feedback

* duo to handle preventing passcode reuse

Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
2022-02-17 13:08:51 -08:00
John-Michael Faircloth c6e64f51b6
feature: multiplexing: handle error plugin shutdown (#14105)
* feat: DB plugin multiplexing (#13734)

* WIP: start from main and get a plugin runner from core

* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode

* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient

* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient

* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists

* add ID to dbplugin.DatabasePluginClient struct

* only create one plugin process per plugin type

* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server

* add v6 multiplexing  protocol version

* WIP: backwards compat for db plugins

* Ensure locking on plugin catalog access

- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin

* use the New constructor for db plugins

* grpc server: use write lock for Close and rlock for CRUD

* cleanup MultiplexedClients on Close

* remove TODO

* fix multiplexing regression with grpc server connection

* cleanup grpc server instances on close

* embed ClientProtocol in Multiplexer interface

* use PluginClientConfig arg to make NewPluginClient plugin type agnostic

* create a new plugin process for non-muxed plugins

* feat: plugin multiplexing: handle plugin client cleanup (#13896)

* use closure for plugin client cleanup

* log and return errors; add comments

* move rpcClient wrapping to core for ID injection

* refactor core plugin client and sdk

* remove unused ID method

* refactor and only wrap clientConn on multiplexed plugins

* rename structs and do not export types

* Slight refactor of system view interface

* Revert "Slight refactor of system view interface"

This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.

* Revert "Revert "Slight refactor of system view interface""

This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.

* only provide pluginRunner arg to the internal newPluginClient method

* embed ClientProtocol in pluginClient and name logger

* Add back MLock support

* remove enableMlock arg from setupPluginCatalog

* rename plugin util interface to PluginClient

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>

* feature: multiplexing: fix unit tests (#14007)

* fix grpc_server tests and add coverage

* update run_config tests

* add happy path test case for grpc_server ID from context

* update test helpers

* feat: multiplexing: handle v5 plugin compiled with new sdk

* add mux supported flag and increase test coverage

* set multiplexingSupport field in plugin server

* remove multiplexingSupport field in sdk

* revert postgres to non-multiplexed

* add comments on grpc server fields

* use pointer receiver on grpc server methods

* add changelog

* use pointer for grpcserver instance

* Use a gRPC server to determine if a plugin should be multiplexed

* Apply suggestions from code review

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* add lock to removePluginClient

* add multiplexingSupport field to externalPlugin struct

* do not send nil to grpc MultiplexingSupport

* check err before logging

* handle locking scenario for cleanupFunc

* allow ServeConfigMultiplex to dispense v5 plugin

* reposition structs, add err check and comments

* add comment on locking for cleanupExternalPlugin

* feature: multiplexing: handle error plugin shutdown

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2022-02-17 14:32:31 -06:00
Jordan Reimer b936db8332
Revert "MFA (#14049)" (#14135)
This reverts commit 5f17953b5980e6438215d5cb62c8575d16c63193.
2022-02-17 13:17:59 -07:00
Hridoy Roy ebf8e5308a
SSCT Tokens Feature [OSS] (#14109)
* port SSCT OSS

* port header hmac key to ent and generate token proto without make command

* remove extra nil check in request handling

* add changelog

* add comment to router.go

* change test var to use length constants

* remove local index is 0 check and extra defer which can be removed after use of ExternalID
2022-02-17 11:43:07 -08:00
Jordan Reimer 36ccfaa3aa
MFA (#14049)
* adds development workflow to mirage config

* adds mirage handler and factory for mfa workflow

* adds mfa handling to auth service and cluster adapter

* moves auth success logic from form to controller

* adds mfa form component

* shows delayed auth message for all methods

* adds new code delay to mfa form

* adds error views

* fixes merge conflict

* adds integration tests for mfa-form component

* fixes auth tests

* updates mfa response handling to align with backend

* updates mfa-form to handle multiple methods and constraints

* adds noDefault arg to Select component

* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios

* adds tests

* flaky test fix attempt

* reverts test fix attempt

* adds changelog entry

* updates comments for todo items

* removes faker from mfa mirage factory and handler

* adds number to word helper

* fixes tests

* Revert "Merge branch 'main' into ui/mfa"

This reverts commit 8ee6a6aaa1b6c9ec16b985c10d91c3806819ec40, reversing
changes made to 2428dd6cca07bb41cda3f453619646ca3a88bfd0.

* format-ttl helper fix from main
2022-02-17 09:10:56 -07:00
John-Michael Faircloth 1cf74e1179
feature: multiplexing support for database plugins (#14033)
* feat: DB plugin multiplexing (#13734)

* WIP: start from main and get a plugin runner from core

* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode

* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient

* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient

* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists

* add ID to dbplugin.DatabasePluginClient struct

* only create one plugin process per plugin type

* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server

* add v6 multiplexing  protocol version

* WIP: backwards compat for db plugins

* Ensure locking on plugin catalog access

- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin

* use the New constructor for db plugins

* grpc server: use write lock for Close and rlock for CRUD

* cleanup MultiplexedClients on Close

* remove TODO

* fix multiplexing regression with grpc server connection

* cleanup grpc server instances on close

* embed ClientProtocol in Multiplexer interface

* use PluginClientConfig arg to make NewPluginClient plugin type agnostic

* create a new plugin process for non-muxed plugins

* feat: plugin multiplexing: handle plugin client cleanup (#13896)

* use closure for plugin client cleanup

* log and return errors; add comments

* move rpcClient wrapping to core for ID injection

* refactor core plugin client and sdk

* remove unused ID method

* refactor and only wrap clientConn on multiplexed plugins

* rename structs and do not export types

* Slight refactor of system view interface

* Revert "Slight refactor of system view interface"

This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.

* Revert "Revert "Slight refactor of system view interface""

This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.

* only provide pluginRunner arg to the internal newPluginClient method

* embed ClientProtocol in pluginClient and name logger

* Add back MLock support

* remove enableMlock arg from setupPluginCatalog

* rename plugin util interface to PluginClient

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>

* feature: multiplexing: fix unit tests (#14007)

* fix grpc_server tests and add coverage

* update run_config tests

* add happy path test case for grpc_server ID from context

* update test helpers

* feat: multiplexing: handle v5 plugin compiled with new sdk

* add mux supported flag and increase test coverage

* set multiplexingSupport field in plugin server

* remove multiplexingSupport field in sdk

* revert postgres to non-multiplexed

* add comments on grpc server fields

* use pointer receiver on grpc server methods

* add changelog

* use pointer for grpcserver instance

* Use a gRPC server to determine if a plugin should be multiplexed

* Apply suggestions from code review

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* add lock to removePluginClient

* add multiplexingSupport field to externalPlugin struct

* do not send nil to grpc MultiplexingSupport

* check err before logging

* handle locking scenario for cleanupFunc

* allow ServeConfigMultiplex to dispense v5 plugin

* reposition structs, add err check and comments

* add comment on locking for cleanupExternalPlugin

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2022-02-17 08:50:33 -06:00
Pratyoy Mukhopadhyay 0ceccaa51d
oss changes for cross ns remount (#14104) 2022-02-16 11:21:42 -08:00
Austin Gebauer 34d295e28f
identity/oidc: Adds proof key for code exchange (PKCE) support (#13917) 2022-02-15 12:02:22 -08:00
VAL 772cfcab91
Typo and clarity fixes (#14081) 2022-02-15 10:43:49 -08:00
Chris Capurso dd0a2c15a3
fix fmt (#14062) 2022-02-14 18:06:02 -05:00
Chris Capurso f9e9b4d327
Add sys/version-history endpoint and associated command (#13766)
* store version history as utc; add self-heal logic

* add sys/version-history endpoint

* change version history from GET to LIST, require auth

* add "vault version-history" CLI command

* add vault-version CLI error message for version string parsing

* adding version-history API and CLI docs

* add changelog entry

* some version-history command fixes

* remove extraneous cmd args

* fix version-history command help text

* specify in docs that endpoint was added in 1.10.0

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* enforce UTC within storeVersionTimestamp directly

* fix improper use of %w in logger.Warn

* remove extra err check and erroneous return from loadVersionTimestamps

* add >= 1.10.0 warning to version-history cmd

* move sys/version-history tests

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-02-14 15:26:57 -05:00
Austin Gebauer 88a8103ad6
identity/oidc: use inherited group membership for client assignments (#14013) 2022-02-11 11:40:44 -08:00
Lars Lehtonen e4c1877552
vault/external_tests/approle: fix dropped test error (#13947) 2022-02-10 19:41:45 -05:00