Commit Graph

2410 Commits

Author SHA1 Message Date
Josh Black e83471d7de
Login MFA (#14025)
* Login MFA

* ENT OSS segragation (#14088)

* Delete method id if not used in an MFA enforcement config (#14063)

* Delete an MFA methodID only if it is not used by an MFA enforcement config

* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path

* adding use_passcode field to DUO config (#14059)

* add changelog

* preventing replay attack on MFA passcodes (#14056)

* preventing replay attack on MFA passcodes

* using %w instead of %s for error

* Improve CLI command for login mfa (#14106)

CLI prints a warning message indicating the login request needs to get validated

* adding the validity period of a passcode to error messages (#14115)

* PR feedback

* duo to handle preventing passcode reuse

Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
2022-02-17 13:08:51 -08:00
John-Michael Faircloth c6e64f51b6
feature: multiplexing: handle error plugin shutdown (#14105)
* feat: DB plugin multiplexing (#13734)

* WIP: start from main and get a plugin runner from core

* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode

* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient

* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient

* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists

* add ID to dbplugin.DatabasePluginClient struct

* only create one plugin process per plugin type

* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server

* add v6 multiplexing  protocol version

* WIP: backwards compat for db plugins

* Ensure locking on plugin catalog access

- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin

* use the New constructor for db plugins

* grpc server: use write lock for Close and rlock for CRUD

* cleanup MultiplexedClients on Close

* remove TODO

* fix multiplexing regression with grpc server connection

* cleanup grpc server instances on close

* embed ClientProtocol in Multiplexer interface

* use PluginClientConfig arg to make NewPluginClient plugin type agnostic

* create a new plugin process for non-muxed plugins

* feat: plugin multiplexing: handle plugin client cleanup (#13896)

* use closure for plugin client cleanup

* log and return errors; add comments

* move rpcClient wrapping to core for ID injection

* refactor core plugin client and sdk

* remove unused ID method

* refactor and only wrap clientConn on multiplexed plugins

* rename structs and do not export types

* Slight refactor of system view interface

* Revert "Slight refactor of system view interface"

This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.

* Revert "Revert "Slight refactor of system view interface""

This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.

* only provide pluginRunner arg to the internal newPluginClient method

* embed ClientProtocol in pluginClient and name logger

* Add back MLock support

* remove enableMlock arg from setupPluginCatalog

* rename plugin util interface to PluginClient

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>

* feature: multiplexing: fix unit tests (#14007)

* fix grpc_server tests and add coverage

* update run_config tests

* add happy path test case for grpc_server ID from context

* update test helpers

* feat: multiplexing: handle v5 plugin compiled with new sdk

* add mux supported flag and increase test coverage

* set multiplexingSupport field in plugin server

* remove multiplexingSupport field in sdk

* revert postgres to non-multiplexed

* add comments on grpc server fields

* use pointer receiver on grpc server methods

* add changelog

* use pointer for grpcserver instance

* Use a gRPC server to determine if a plugin should be multiplexed

* Apply suggestions from code review

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* add lock to removePluginClient

* add multiplexingSupport field to externalPlugin struct

* do not send nil to grpc MultiplexingSupport

* check err before logging

* handle locking scenario for cleanupFunc

* allow ServeConfigMultiplex to dispense v5 plugin

* reposition structs, add err check and comments

* add comment on locking for cleanupExternalPlugin

* feature: multiplexing: handle error plugin shutdown

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2022-02-17 14:32:31 -06:00
Jordan Reimer b936db8332
Revert "MFA (#14049)" (#14135)
This reverts commit 5f17953b5980e6438215d5cb62c8575d16c63193.
2022-02-17 13:17:59 -07:00
Hridoy Roy ebf8e5308a
SSCT Tokens Feature [OSS] (#14109)
* port SSCT OSS

* port header hmac key to ent and generate token proto without make command

* remove extra nil check in request handling

* add changelog

* add comment to router.go

* change test var to use length constants

* remove local index is 0 check and extra defer which can be removed after use of ExternalID
2022-02-17 11:43:07 -08:00
Jordan Reimer 36ccfaa3aa
MFA (#14049)
* adds development workflow to mirage config

* adds mirage handler and factory for mfa workflow

* adds mfa handling to auth service and cluster adapter

* moves auth success logic from form to controller

* adds mfa form component

* shows delayed auth message for all methods

* adds new code delay to mfa form

* adds error views

* fixes merge conflict

* adds integration tests for mfa-form component

* fixes auth tests

* updates mfa response handling to align with backend

* updates mfa-form to handle multiple methods and constraints

* adds noDefault arg to Select component

* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios

* adds tests

* flaky test fix attempt

* reverts test fix attempt

* adds changelog entry

* updates comments for todo items

* removes faker from mfa mirage factory and handler

* adds number to word helper

* fixes tests

* Revert "Merge branch 'main' into ui/mfa"

This reverts commit 8ee6a6aaa1b6c9ec16b985c10d91c3806819ec40, reversing
changes made to 2428dd6cca07bb41cda3f453619646ca3a88bfd0.

* format-ttl helper fix from main
2022-02-17 09:10:56 -07:00
John-Michael Faircloth 1cf74e1179
feature: multiplexing support for database plugins (#14033)
* feat: DB plugin multiplexing (#13734)

* WIP: start from main and get a plugin runner from core

* move MultiplexedClient map to plugin catalog
- call sys.NewPluginClient from PluginFactory
- updates to getPluginClient
- thread through isMetadataMode

* use go-plugin ClientProtocol interface
- call sys.NewPluginClient from dbplugin.NewPluginClient

* move PluginSets to dbplugin package
- export dbplugin HandshakeConfig
- small refactor of PluginCatalog.getPluginClient

* add removeMultiplexedClient; clean up on Close()
- call client.Kill from plugin catalog
- set rpcClient when muxed client exists

* add ID to dbplugin.DatabasePluginClient struct

* only create one plugin process per plugin type

* update NewPluginClient to return connection ID to sdk
- wrap grpc.ClientConn so we can inject the ID into context
- get ID from context on grpc server

* add v6 multiplexing  protocol version

* WIP: backwards compat for db plugins

* Ensure locking on plugin catalog access

- Create public GetPluginClient method for plugin catalog
- rename postgres db plugin

* use the New constructor for db plugins

* grpc server: use write lock for Close and rlock for CRUD

* cleanup MultiplexedClients on Close

* remove TODO

* fix multiplexing regression with grpc server connection

* cleanup grpc server instances on close

* embed ClientProtocol in Multiplexer interface

* use PluginClientConfig arg to make NewPluginClient plugin type agnostic

* create a new plugin process for non-muxed plugins

* feat: plugin multiplexing: handle plugin client cleanup (#13896)

* use closure for plugin client cleanup

* log and return errors; add comments

* move rpcClient wrapping to core for ID injection

* refactor core plugin client and sdk

* remove unused ID method

* refactor and only wrap clientConn on multiplexed plugins

* rename structs and do not export types

* Slight refactor of system view interface

* Revert "Slight refactor of system view interface"

This reverts commit 73d420e5cd2f0415e000c5a9284ea72a58016dd6.

* Revert "Revert "Slight refactor of system view interface""

This reverts commit f75527008a1db06d04a23e04c3059674be8adb5f.

* only provide pluginRunner arg to the internal newPluginClient method

* embed ClientProtocol in pluginClient and name logger

* Add back MLock support

* remove enableMlock arg from setupPluginCatalog

* rename plugin util interface to PluginClient

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>

* feature: multiplexing: fix unit tests (#14007)

* fix grpc_server tests and add coverage

* update run_config tests

* add happy path test case for grpc_server ID from context

* update test helpers

* feat: multiplexing: handle v5 plugin compiled with new sdk

* add mux supported flag and increase test coverage

* set multiplexingSupport field in plugin server

* remove multiplexingSupport field in sdk

* revert postgres to non-multiplexed

* add comments on grpc server fields

* use pointer receiver on grpc server methods

* add changelog

* use pointer for grpcserver instance

* Use a gRPC server to determine if a plugin should be multiplexed

* Apply suggestions from code review

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* add lock to removePluginClient

* add multiplexingSupport field to externalPlugin struct

* do not send nil to grpc MultiplexingSupport

* check err before logging

* handle locking scenario for cleanupFunc

* allow ServeConfigMultiplex to dispense v5 plugin

* reposition structs, add err check and comments

* add comment on locking for cleanupExternalPlugin

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2022-02-17 08:50:33 -06:00
Pratyoy Mukhopadhyay 0ceccaa51d
oss changes for cross ns remount (#14104) 2022-02-16 11:21:42 -08:00
Austin Gebauer 34d295e28f
identity/oidc: Adds proof key for code exchange (PKCE) support (#13917) 2022-02-15 12:02:22 -08:00
VAL 772cfcab91
Typo and clarity fixes (#14081) 2022-02-15 10:43:49 -08:00
Chris Capurso dd0a2c15a3
fix fmt (#14062) 2022-02-14 18:06:02 -05:00
Chris Capurso f9e9b4d327
Add sys/version-history endpoint and associated command (#13766)
* store version history as utc; add self-heal logic

* add sys/version-history endpoint

* change version history from GET to LIST, require auth

* add "vault version-history" CLI command

* add vault-version CLI error message for version string parsing

* adding version-history API and CLI docs

* add changelog entry

* some version-history command fixes

* remove extraneous cmd args

* fix version-history command help text

* specify in docs that endpoint was added in 1.10.0

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* enforce UTC within storeVersionTimestamp directly

* fix improper use of %w in logger.Warn

* remove extra err check and erroneous return from loadVersionTimestamps

* add >= 1.10.0 warning to version-history cmd

* move sys/version-history tests

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-02-14 15:26:57 -05:00
Austin Gebauer 88a8103ad6
identity/oidc: use inherited group membership for client assignments (#14013) 2022-02-11 11:40:44 -08:00
Lars Lehtonen e4c1877552
vault/external_tests/approle: fix dropped test error (#13947) 2022-02-10 19:41:45 -05:00
Austin Gebauer 4c12c2bb42
identity/oidc: adds tests for validation of loopback IP redirect URIs (#13939)
* identity/oidc: adds tests for validation of loopback IP redirect URIs

* Update vault/identity_store_oidc_provider_test.go

Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>

Co-authored-by: John-Michael Faircloth <fairclothjm@users.noreply.github.com>
2022-02-07 12:50:36 -08:00
Joe 8d169d48d3
identity/oidc: loopback redirect dynamic port (#13871)
* Add check for OIDC provider to permit a non-exact redirect URI from OIDC client if it is the IPv4 or IPv6 loopback address.

* Update changelog/13871.txt

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

* Update redirectURI check to match that for the OIDC auth method.

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
2022-02-07 10:34:33 -08:00
Victor Rodriguez 0bb112520e
Seal wrap storage entries of the ManagedKeyRegistry. (#13905)
Seal wrap storage entries of the ManagedKeyRegistry.

Specify the managed key registry's path in SealWrapStorage in the SystemBackend.

Enable SealWrap in the system mount entry.
2022-02-04 16:06:32 -05:00
Alexander Scheel 705439885d
Remove deprecated call to BuildNameToCertificate (#13811)
This function call was previously used to generate mappings from
potential subjects (or SANs) to certificates within the TLS client
object. However, newer Go versions have deprecated this method, instead
building the mapping automatically based on present certificates at
request time. Because the corresponding client configuration field is
not used in Vault (NameToCertificate), it is safe to remove this call
and leave it nil.

See also: 67d894ee65
See also: https://pkg.go.dev/crypto/tls#Config.BuildNameToCertificate

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-01-27 14:56:21 -05:00
Josh Black d249fad2df
reformat using 'make fmt' (#13794) 2022-01-27 10:06:34 -08:00
Rémi Lapeyre 978311fee2
Add read support to sys/auth/:path (#12793)
* Add read support to sys/auth/:path

Closes https://github.com/hashicorp/vault/issues/7411

* Add changelog entry
2022-01-25 11:56:40 -08:00
Rémi Lapeyre d6a4a3b53c
Add LIST support to sys/policies/password (#12787)
* Add read support to sys/policies/password

Closes https://github.com/hashicorp/vault/issues/12562

* Add changelog

* Empty commit to trigger CI

* Add optional /

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>

* Use a ListOperation

Co-authored-by: Calvin Leung Huang <1883212+calvn@users.noreply.github.com>
2022-01-24 13:42:14 -08:00
John-Michael Faircloth 4f40189d1a
oidc: check for nil signing key on rotation (#13716)
* check for nil signing key on rotation

* add changelog

* Update nil signing key handling

- bypass setting ExpireAt if signing key is nil in rotate
- return err if singing key is nil in signPayload

* add comment; update error msg on signPayload; refactor UT
2022-01-24 12:05:49 -06:00
Nick Cabatoff 279e0d4332
Add the duration and start time to logged completed requests. (#13682) 2022-01-20 08:55:30 -05:00
Nick Cabatoff 4230749d9e
Revert #13679 since TestCluster_ListenForRequest now fails intermittently in alarming ways. (#13714) 2022-01-20 08:34:40 -05:00
Sung Hon Wu 194c9e32d3
Enhance sys/raw to read and write values that cannot be encoded in json (#13537) 2022-01-20 07:52:53 -05:00
Scott Miller f7e3ab72a8
OSS side of the managed key interfaces (#13699)
* OSS side of the managed key interfaces

* Not needed in OSS
2022-01-19 11:15:33 -06:00
Nick Cabatoff 21be98ee7a
Support go-sockaddr templates in top-level cluster_addr config (#13678)
In doing some testing I found that the listener clusteraddr isn't really used, or at least isn't as important as the top-level clusteraddr setting.  As such, go-sockaddr templating needs to be implemented for the top-level `cluster_addr` setting or it's unusable for HA.

Also fix a nil pointer panic I discovered at the same time.
2022-01-19 10:56:04 -05:00
Nick Cabatoff d96298461f
Don't say we've shut down cluster listener before having done so (#13679) 2022-01-19 10:51:40 -05:00
Rémi Lapeyre 0d6c2acbd9
Make auth/token/revoke-accessor idempotent (#13661)
The auth/token/revoke will not error out if the token does not exists, it
always tries to revoke the token and return success to the client whether
or not the token exists. This makes the behavior of
auth/token/revoke-accessor coherent with this and remove the need to
check whether the token still exists.
2022-01-18 06:56:38 -05:00
Nick Cabatoff 400996ef0d
Parallel retry join (#13606) 2022-01-17 10:33:03 -05:00
akshya96 319a76d8d1
Vault-3991 Code Scanning Alerts Changes (#13667)
* code scanning alerts changes

* adding changelog
2022-01-14 15:35:27 -08:00
Chris Capurso d52d69e4bb
Add HTTP PATCH support for KV key metadata (#13215)
* go get vault-plugin-secrets-kv@vault-4290-patch-metadata

* add kv metadata patch command

* add changelog entry

* success tests for kv metadata patch flags

* add more kv metadata patch flags tests

* add kv metadata patch cas warning test

* add kv-v2 key metadata patch API docs

* add kv metadata patch to docs

* prevent unintentional field overwriting in kv metadata put cmd

* like create/update ops, prevent patch to paths ending in /

* fix kv metadata patch cmd in docs

* fix flag defaults for kv metadata put

* go get vault-plugin-secrets-kv@vault-4290-patch-metadata

* fix TestKvMetadataPatchCommand_Flags test

* doc fixes

* go get vault-plugin-secrets-kv@master; go mod tidy
2022-01-12 12:05:27 -05:00
Chris Capurso c925324057
add retry logic to TestKV_Patch_RootToken (#13586) 2022-01-07 14:46:26 -05:00
Scott Miller 89f617a97c
Convert to Go 1.17 go:build directive (#13579) 2022-01-05 12:02:03 -06:00
Scott Miller 4a4c9932b1
Add the most basic interfaces to OSS (#13561) 2022-01-04 14:07:18 -06:00
Steven Clark b9e5aeb459
Attempt to address a data race issue within identity store - take 2 (#13476)
* Attempt to address a data race issue within identity store
* Testcase TestIdentityStore_LocalAliasInvalidations identified a data race issue.
* This reverts the previous attempt to address the issue from #13093
2021-12-22 09:51:13 -05:00
Anthony (Ryo) Wright e0ac921b8f
Fixed null token panic from 'v1/auth/token/' endpoints and returned p… (#13233)
* Fixed null token panic from 'v1/auth/token/' endpoints and returned proper error response

* added changelog entry for PR #13233

* changed error message from 'bad token' to 'null token'

* rebased off of main

* Revert "changed error message from 'bad token' to 'null token'"

This reverts commit 381ed9b32c5ddd5e47adb1643ef7e46fb768bc76.

* changed 'bad token' error message to 'invalid token' after revert

* remove unnecessary vault-data folder
2021-12-21 09:46:56 -08:00
Scott Miller 82f333002a
One small diff between ENT and OSS managed key config (#13482)
* Changes to bring ENT in line with OSS for managed key changes

* Revert this
2021-12-20 15:20:18 -06:00
Victor Rodriguez 1136381637
Add no-op method setupManagedKeyRegistry(). (#13433) 2021-12-14 16:00:17 -05:00
Steven Clark 0bb0996072
Fix logging statement using formatting args (#13407) 2021-12-13 15:28:35 -05:00
Ben Ash 6ec3367648
Support clearing an identity alias' custom_metadata (#13395)
* Support clearing an identity alias' custom_metadata

Previously, an update to an entity alias supported updating the
custom_metadata as long as the update was not empty, which makes it
impossible to clear the metadata values completely.

Fixes:
- empty custom_metadata parameters are honoured on entity alias update
- update related tests
- drop dependency on mapstructure
- reformat with gofumpt
2021-12-10 18:07:47 -05:00
Nick Cabatoff fed74de3fe
Remove another use gopsutil/host. (#13390) 2021-12-10 09:59:52 -05:00
Nick Cabatoff f729dfc18f
Use os.Hostname instead of a dependency that doesn't work on OpenBSD. (#13389) 2021-12-10 08:57:15 -05:00
hghaf099 65845c7531
VAULT-1564 report in-flight requests (#13024)
* VAULT-1564 report in-flight requests

* adding a changelog

* Changing some variable names and fixing comments

* minor style change

* adding unauthenticated support for in-flight-req

* adding documentation for the listener.profiling stanza

* adding an atomic counter for the inflight requests
addressing comments

* addressing comments

* logging completed requests

* fixing a test

* providing log_requests_info as a config option to determine at which level requests should be logged

* removing a member and a method from the StatusHeaderResponseWriter struct

* adding api docks

* revert changes in NewHTTPResponseWriter

* Fix logging invalid log_requests_info value

* Addressing comments

* Fixing a test

* use an tomic value for logRequestsInfo, and moving the CreateClientID function to Core

* fixing go.sum

* minor refactoring

* protecting InFlightRequests from data race

* another try on fixing a data race

* another try to fix a data race

* addressing comments

* fixing couple of tests

* changing log_requests_info to log_requests_level

* minor style change

* fixing a test

* removing the lock in InFlightRequests

* use single-argument form for interface assertion

* adding doc for the new configuration paramter

* adding the new doc to the nav data file

* minor fix
2021-12-08 17:34:42 -05:00
Matt Schultz 85f5cfc356
Adds support for SHA-3 to transit (#13367)
* Adding support for SHA3 in the transit backend.

* Adds SHA-3 tests for transit sign/verify path. Adds SHA-3 tests for logical system tools path hash functionality. Updates documentation to include SHA-3 algorithms in system tools path hashing.

* Adds changelog entry.

Co-authored-by: robison jacka <robison@packetized.io>
2021-12-08 12:29:33 -06:00
Jim Kalafut 22c4ae5933
Rename master key to root key (#13324)
* See what it looks like to replace "master key" with "root key".  There are two places that would require more challenging code changes: the storage path `core/master`, and its contents (the JSON-serialized EncodedKeyringtructure.)

* Restore accidentally deleted line

* Add changelog

* Update root->recovery

* Fix test

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-12-06 17:12:20 -08:00
Anthony (Ryo) Wright bcd29f2b68
Fix null token type bug (#13236)
* Fixed null token panic from 'v1/auth/token/' endpoints and returned proper error response

* Fixed panic resulting from null token_type in /auth/token/roles/{role_name} to returne proper error response

* added changelog entry for PR #13236

* edit changelog entry for PR #13236
2021-12-06 09:38:53 -08:00
Eugene R c2a92cd351
Fix possible nil pointer dereference (#13318) 2021-12-02 08:23:41 -05:00
Pavlos Tzianos 0abc8f43fa
Add helper for encoding/decoding root tokens and OTP generation in SDK module (#10504) (#10505) 2021-12-01 08:05:49 -05:00
Nick Cabatoff a47a2c9fc4
Add "operator members" command to list nodes in the cluster. (#13292) 2021-11-30 14:49:58 -05:00
Hridoy Roy 1ff2e8d9d0
some small version test changes (#13310) 2021-11-30 09:52:33 -08:00