Commit Graph

567 Commits

Author SHA1 Message Date
John-Michael Faircloth b6c05fae33
feature: secrets/auth plugin multiplexing (#14946)
* enable registering backend muxed plugins in plugin catalog

* set the sysview on the pluginconfig to allow enabling secrets/auth plugins

* store backend instances in map

* store single implementations in the instances map

cleanup instance map and ensure we don't deadlock

* fix system backend unit tests

move GetMultiplexIDFromContext to pluginutil package

fix pluginutil test

fix dbplugin ut

* return error(s) if we can't get the plugin client

update comments

* refactor/move GetMultiplexIDFromContext test

* add changelog

* remove unnecessary field on pluginClient

* add unit tests to PluginCatalog for secrets/auth plugins

* fix comment

* return pluginClient from TestRunTestPlugin

* add multiplexed backend test

* honor metadatamode value in newbackend pluginconfig

* check that connection exists on cleanup

* add automtls to secrets/auth plugins

* don't remove apiclientmeta parsing

* use formatting directive for fmt.Errorf

* fix ut: remove tls provider func

* remove tlsproviderfunc from backend plugin tests

* use env var to prevent test plugin from running as a unit test

* WIP: remove lazy loading

* move non lazy loaded backend to new package

* use version wrapper for backend plugin factory

* remove backendVersionWrapper type

* implement getBackendPluginType for plugin catalog

* handle backend plugin v4 registration

* add plugin automtls env guard

* modify plugin factory to determine the backend to use

* remove old pluginsets from v5 and log pid in plugin catalog

* add reload mechanism via context

* readd v3 and v4 to pluginset

* call cleanup from reload if non-muxed

* move v5 backend code to new package

* use context reload for for ErrPluginShutdown case

* add wrapper on v5 backend

* fix run config UTs

* fix unit tests

- use v4/v5 mapping for plugin versions
- fix test build err
- add reload method on fakePluginClient
- add multiplexed cases for integration tests

* remove comment and update AutoMTLS field in test

* remove comment

* remove errwrap and unused context

* only support metadatamode false for v5 backend plugins

* update plugin catalog errors

* use const for env variables

* rename locks and remove unused

* remove unneeded nil check

* improvements based on staticcheck recommendations

* use const for single implementation string

* use const for context key

* use info default log level

* move pid to pluginClient struct

* remove v3 and v4 from multiplexed plugin set

* return from reload when non-multiplexed

* update automtls env string

* combine getBackend and getBrokeredClient

* update comments for plugin reload, Backend return val and log

* revert Backend return type

* allow non-muxed plugins to serve v5

* move v5 code to existing sdk plugin package

* do next export sdk fields now that we have removed extra plugin pkg

* set TLSProvider in ServeMultiplex for backwards compat

* use bool to flag multiplexing support on grpc backend server

* revert userpass main.go

* refactor plugin sdk

- update comments
- make use of multiplexing boolean and single implementation ID const

* update comment and use multierr

* attempt v4 if dispense fails on getPluginTypeForUnknown

* update comments on sdk plugin backend
2022-08-29 21:42:26 -05:00
Scott Miller 3bd38fd5dc
OSS portion of wrapper-v2 (#16811)
* OSS portion of wrapper-v2

* Prefetch barrier type to avoid encountering an error in the simple BarrierType() getter

* Rename the OveriddenType to WrapperType and use it for the barrier type prefetch

* Fix unit test
2022-08-23 15:37:16 -04:00
Violet Hynes 0b3d7fdf10
VAULT-7698 Fix ignored parameter warnings for endpoint arbitrary data options (#16794)
* VAULT-7698 Fix warnings for endpoint arbitrary data options

* VAULT-7698 Add changelog
2022-08-23 08:51:23 -04:00
Steven Clark e024324c34
Add an OCSP responder to Vault's PKI plugin (#16723)
* Refactor existing CRL function to storage getRevocationConfig

* Introduce ocsp_disable config option in config/crl

* Introduce OCSPSigning usage flag on issuer

* Add ocsp-request passthrough within lower layers of Vault

* Add OCSP responder to Vault PKI

* Add API documentation for OCSP

* Add cl

* Revert PKI storage migration modifications for OCSP

* Smaller PR feedback items

 - pki.mdx doc update
 - parens around logical.go comment to indicate DER encoded request is
   related to OCSP and not the snapshots
 - Use AllIssuers instead of writing them all out
 - Drop zero initialization of crl config's Disable flag if not present
 - Upgrade issuer on the fly instead of an initial migration

* Additional clean up backing out the writeRevocationConfig refactoring

* Remove Dirty issuer flag and update comment about not writing upgrade to
storage

* Address PR feedback and return Unknown response when mismatching issuer

* make fmt

* PR Feedback.

* More PR feedback

 - Leverage ocsp response constant
 - Remove duplicate errors regarding unknown issuers
2022-08-22 14:06:15 -04:00
swayne275 4632a26a09
Use %q for quoted strings where appropriate (#15216)
* change '%s' to %q where single vs double quotes shouldn't matter

* replace double quotes with %q in logs and errors
2022-08-03 12:32:45 -06:00
Steven Clark fc4c8d8c67
Address test flakiness in TestLogical_AuditPort (#16546)
- Based on group test fixing session from July 29, 2022
 - Leverage the RetryUntil to catch and re-attempt a kv store creation
   if the test receives an error about upgrading the KV store
 - Update the expected audit log entries accordingly along with the
   captured failures if any
 - Fix up a copy/paste error within the test error message if the
   remote_address field is not of the expected type.
2022-08-03 10:14:17 -04:00
Austin Gebauer b3f138679c
identity/oidc: allow filtering the list providers response by an allowed_client_id (#16181)
* identity/oidc: allow filtering the list providers response by an allowed_client_id

* adds changelog

* adds api documentation

* use identity store view in list provider test
2022-07-28 09:47:53 -07:00
akshya96 6e5b4ec8c9
Validate parameters for vault operator init (#16379)
* adding code changes to check parameters for vault operator init

* fixing defaults in cli

* fixing comments

* remove unused consts

* adding validate parameters test

* add changelog

* adding changes to validation

* adding fmt changes

* fixing test

* add test for auto unseal
2022-07-25 12:45:04 -07:00
Violet Hynes 0c80ee5cf5
VAULT-6614 Enable role based quotas for lease-count quotas (OSS) (#16157)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* VAULT-6614 start of changes for roles in LCQs

* Expiration changes for leases

* Add role information to RequestAuth

* VAULT-6614 Test updates

* VAULT-6614 Add expiration test with roles

* VAULT-6614 fix comment

* VAULT-6614 Protobuf on OSS

* VAULT-6614 Add rlock to determine role code

* VAULT-6614 Try lock instead of rlock

* VAULT-6614 back to rlock while I think about this more

* VAULT-6614 Additional safety for nil dereference

* VAULT-6614 Use %q over %s

* VAULT-6614 Add overloading to plugin backends

* VAULT-6614 RLocks instead

* VAULT-6614 Fix return for backend factory
2022-07-05 13:02:00 -04:00
Violet Hynes d57fea2cd1
VAULT-6613 Add role support for rate limit quotas (OSS Changes) (#16115)
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core

* Fix body handling

* Role resolution for rate limit quotas

* VAULT-6613 update precedence test

* Add changelog

* Handle body error

* VAULT-6613 Return early if error with json parsing
2022-06-24 08:58:02 -04:00
Brian Kassouf df8ae055be
Add an API for exporting activity log data (#15586)
* Add an API for exporting activity log data

* Add changelog entry

* Switch to error logs
2022-05-24 17:00:46 -07:00
davidadeleon 0026788d4b
api/monitor: Adding log format to monitor command and debug (#15536)
* Correct handling of "unspecified" log level

* Setting log-format default on monitor path

* Create changelog file

* Update website/content/api-docs/system/monitor.mdx

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>

Co-authored-by: Chris Capurso <1036769+ccapurso@users.noreply.github.com>
2022-05-24 13:10:53 -04:00
akshya96 4e9e9b7eda
Vault-6037 making filesystem permissions check opt-in (#15452)
* adding env var changes

* adding changelog

* adding strcov.ParseBool
2022-05-17 11:34:31 -07:00
Christopher Swenson 0affe226ad
Update deps for consul-template 0.29.0 (#15293)
This requires bumping https://github.com/mitchellh/go-testing-interface.
For this new version, we have to create a wrapper to convert
the stdlib `testing.TB` interface to the
`mitchellh/go-testing-interface` `T` interface, since it uses
`Parallel()` now, which is not supported by `testing.TB`. This had to be
added to a new package, `benchhelpers`, to avoid a circular dependency
in `testhelpers`.

We also have to *unbump* https://github.com/armon/go-metrics since
updating it breaks our usage of
https://github.com/google/go-metrics-stackdriver

I verified that the new `pkiCert` template function works with agent
injection using annotations like:

```yaml
vault.hashicorp.com/agent-inject-secret-sample.crt: "pki/issue/example-dot-com"
vault.hashicorp.com/agent-inject-template-sample.crt: |
  {{ pkiCert "pki/issue/example-dot-com" "common_name=foo.example.com" "ttl=1h" }}
```
2022-05-05 10:30:40 -07:00
Chris Capurso e69f89c279
Add build date (#14957)
* add BuildDate to version base

* populate BuildDate with ldflags

* include BuildDate in FullVersionNumber

* add BuildDate to seal-status and associated status cmd

* extend core/versions entries to include BuildDate

* include BuildDate in version-history API and CLI

* fix version history tests

* fix sys status tests

* fix TestStatusFormat

* remove extraneous LD_FLAGS from build.sh

* add BuildDate to build.bat

* fix TestSysUnseal_Reset

* attempt to add build-date to release builds

* add branch to github build workflow

* add get-build-date to build-* job needs

* fix release build command vars

* add missing quote in release build command

* Revert "add branch to github build workflow"

This reverts commit b835699ecb7c2c632757fa5fe64b3d5f60d2a886.

* add changelog entry
2022-04-19 14:28:08 -04:00
Vishal Nayak ad3bf3173c
Warnings indicating ignored and replaced parameters (#14962)
* Warnings indicating ignored and replaced parameters

* Avoid additional var creation

* Add warnings only if the response is non-nil

* Return the response even when error is non-nil

* Fix tests

* Rearrange comments

* Print warning in the log

* Fix another test

* Add CL
2022-04-11 09:57:12 -04:00
Anton Averchenkov 23a3f950e4
Revert the WithContext changes to vault tests (#14947) 2022-04-07 15:12:58 -04:00
akshya96 796003ddda
Vault 3992 ToB Config and Plugins Permissions (#14817)
* updating changes from ent PR

* adding changelog

* fixing err

* fixing semgrep error
2022-04-04 09:45:41 -07:00
Chris Capurso 2354435262
treat logical.ErrRelativePath as 400 instead of 500 (#14328)
* treat logical.ErrRelativePath as 400 instead of 500

* add changelog entry

* return UserError for logical.ErrRelativePath
2022-03-30 09:08:02 -04:00
akshya96 e59aee02bd
Vault-4010 Unauthenticated panic when processing "help" requests (#14704)
* fix help panic

* add changelog entry
2022-03-24 12:19:14 -07:00
Anton Averchenkov 1222375d1a
Add context-aware functions to vault/api (#14388) 2022-03-23 17:47:43 -04:00
Chris Capurso cbc3115d9b
Make web_ui complient with Go's native embedding (#14246)
If directory is embeded using Go's embed package it should be present in
git repository to avoid failures/erros messages in Go commands.

Co-authored-by: Henri Koski <henri.t.koski@gmail.com>
2022-02-24 09:12:36 -05:00
Pratyoy Mukhopadhyay 475b55b460
Vault 4632 auth remount oss (#14141)
* Update plugin-portal.mdx (#13229)

Add a Vault plugin to allow authentication via SSH certificates and public keys

* oss changes

Co-authored-by: Wim <wim@42.be>
2022-02-18 08:04:21 -08:00
swayne275 24d512f0d1
Cleanup bool checks (#14102)
* clean up == true cases

* cleanup == false where it didn't seem to hurt readability
2022-02-18 07:35:53 -07:00
Josh Black e83471d7de
Login MFA (#14025)
* Login MFA

* ENT OSS segragation (#14088)

* Delete method id if not used in an MFA enforcement config (#14063)

* Delete an MFA methodID only if it is not used by an MFA enforcement config

* Fixing a bug: mfa/validate is an unauthenticated path, and goes through the handleLoginRequest path

* adding use_passcode field to DUO config (#14059)

* add changelog

* preventing replay attack on MFA passcodes (#14056)

* preventing replay attack on MFA passcodes

* using %w instead of %s for error

* Improve CLI command for login mfa (#14106)

CLI prints a warning message indicating the login request needs to get validated

* adding the validity period of a passcode to error messages (#14115)

* PR feedback

* duo to handle preventing passcode reuse

Co-authored-by: hghaf099 <83242695+hghaf099@users.noreply.github.com>
Co-authored-by: hamid ghaf <hamid@hashicorp.com>
2022-02-17 13:08:51 -08:00
Jordan Reimer b936db8332
Revert "MFA (#14049)" (#14135)
This reverts commit 5f17953b5980e6438215d5cb62c8575d16c63193.
2022-02-17 13:17:59 -07:00
Hridoy Roy ebf8e5308a
SSCT Tokens Feature [OSS] (#14109)
* port SSCT OSS

* port header hmac key to ent and generate token proto without make command

* remove extra nil check in request handling

* add changelog

* add comment to router.go

* change test var to use length constants

* remove local index is 0 check and extra defer which can be removed after use of ExternalID
2022-02-17 11:43:07 -08:00
Jordan Reimer 36ccfaa3aa
MFA (#14049)
* adds development workflow to mirage config

* adds mirage handler and factory for mfa workflow

* adds mfa handling to auth service and cluster adapter

* moves auth success logic from form to controller

* adds mfa form component

* shows delayed auth message for all methods

* adds new code delay to mfa form

* adds error views

* fixes merge conflict

* adds integration tests for mfa-form component

* fixes auth tests

* updates mfa response handling to align with backend

* updates mfa-form to handle multiple methods and constraints

* adds noDefault arg to Select component

* updates mirage mfa handler to align with backend and adds generator for various mfa scenarios

* adds tests

* flaky test fix attempt

* reverts test fix attempt

* adds changelog entry

* updates comments for todo items

* removes faker from mfa mirage factory and handler

* adds number to word helper

* fixes tests

* Revert "Merge branch 'main' into ui/mfa"

This reverts commit 8ee6a6aaa1b6c9ec16b985c10d91c3806819ec40, reversing
changes made to 2428dd6cca07bb41cda3f453619646ca3a88bfd0.

* format-ttl helper fix from main
2022-02-17 09:10:56 -07:00
Pratyoy Mukhopadhyay 0ceccaa51d
oss changes for cross ns remount (#14104) 2022-02-16 11:21:42 -08:00
Victor Rodriguez 0bb112520e
Seal wrap storage entries of the ManagedKeyRegistry. (#13905)
Seal wrap storage entries of the ManagedKeyRegistry.

Specify the managed key registry's path in SealWrapStorage in the SystemBackend.

Enable SealWrap in the system mount entry.
2022-02-04 16:06:32 -05:00
Josh Black d249fad2df
reformat using 'make fmt' (#13794) 2022-01-27 10:06:34 -08:00
Rémi Lapeyre cf8b5642f2
Add remote_port in the audit logs when it is available (#12790)
* Add remote_port in the audit logs when it is available

The `request.remote_port` field is now present in the audit log when it
is available:

```
{
  "time": "2021-10-10T13:53:51.760039Z",
  "type": "response",
  "auth": {
    "client_token": "hmac-sha256:1304aab0ac65747684e1b58248cc16715fa8f558f8d27e90fcbcb213220c0edf",
    "accessor": "hmac-sha256:f8cf0601dadd19aac84f205ded44c62898e3746a42108a51105a92ccc39baa43",
    "display_name": "root",
    "policies": [
      "root"
    ],
    "token_policies": [
      "root"
    ],
    "token_type": "service",
    "token_issue_time": "2021-10-10T15:53:44+02:00"
  },
  "request": {
    "id": "829c04a1-0352-2d9d-9bc9-00b928d33df5",
    "operation": "update",
    "mount_type": "system",
    "client_token": "hmac-sha256:1304aab0ac65747684e1b58248cc16715fa8f558f8d27e90fcbcb213220c0edf",
    "client_token_accessor": "hmac-sha256:f8cf0601dadd19aac84f205ded44c62898e3746a42108a51105a92ccc39baa43",
    "namespace": {
      "id": "root"
    },
    "path": "sys/audit/file",
    "data": {
      "description": "hmac-sha256:321a1d105f8c6fd62be4f34c4da4f0e6d1cdee9eb2ff4af0b59e1410950fe86b",
      "local": false,
      "options": {
        "file_path": "hmac-sha256:2421b5bf8dab1f9775b2e6e66e58d7bca99ab729f3f311782fda50717eee55b3"
      },
      "type": "hmac-sha256:30dff9607b4087e3ae6808b4a3aa395b1fc064e467748c55c25ddf0e9b150fcc"
    },
    "remote_address": "127.0.0.1",
    "remote_port": 54798
  },
  "response": {
    "mount_type": "system"
  }
}
```

Closes https://github.com/hashicorp/vault/issues/7716

* Add changelog entry

* Empty commit to trigger CI

* Add test and explicit error handling

* Change temporary file pattern in test
2022-01-26 15:47:15 -08:00
Rémi Lapeyre 961ff4a363
Return num_uses during authentication (#12791)
* Return num_uses during authentication

https://github.com/hashicorp/vault/issues/10664

* Add changelog entry
2022-01-25 18:59:53 -08:00
Chris Capurso c925324057
add retry logic to TestKV_Patch_RootToken (#13586) 2022-01-07 14:46:26 -05:00
Scott Miller 89f617a97c
Convert to Go 1.17 go:build directive (#13579) 2022-01-05 12:02:03 -06:00
hghaf099 65845c7531
VAULT-1564 report in-flight requests (#13024)
* VAULT-1564 report in-flight requests

* adding a changelog

* Changing some variable names and fixing comments

* minor style change

* adding unauthenticated support for in-flight-req

* adding documentation for the listener.profiling stanza

* adding an atomic counter for the inflight requests
addressing comments

* addressing comments

* logging completed requests

* fixing a test

* providing log_requests_info as a config option to determine at which level requests should be logged

* removing a member and a method from the StatusHeaderResponseWriter struct

* adding api docks

* revert changes in NewHTTPResponseWriter

* Fix logging invalid log_requests_info value

* Addressing comments

* Fixing a test

* use an tomic value for logRequestsInfo, and moving the CreateClientID function to Core

* fixing go.sum

* minor refactoring

* protecting InFlightRequests from data race

* another try on fixing a data race

* another try to fix a data race

* addressing comments

* fixing couple of tests

* changing log_requests_info to log_requests_level

* minor style change

* fixing a test

* removing the lock in InFlightRequests

* use single-argument form for interface assertion

* adding doc for the new configuration paramter

* adding the new doc to the nav data file

* minor fix
2021-12-08 17:34:42 -05:00
Pavlos Tzianos 0abc8f43fa
Add helper for encoding/decoding root tokens and OTP generation in SDK module (#10504) (#10505) 2021-12-01 08:05:49 -05:00
Nick Cabatoff a47a2c9fc4
Add "operator members" command to list nodes in the cluster. (#13292) 2021-11-30 14:49:58 -05:00
Lars Lehtonen 8b6477a0f0
http: fix dropped test error (#13247) 2021-11-24 14:08:22 -08:00
hghaf099 9640d35136
Unify HTTPResponseWriter and StatusHeaderResponseWriter (#13200)
* Unify NewHTTPResponseWriter ant NewStatusHeaderResponseWriter to fix ResponseWriter issues

* adding changelog

* removing unnecessary function from the WrappingResponseWriter interface

* changing logical requests responseWriter type

* reverting change to HTTPResponseWriter
2021-11-23 14:30:25 -05:00
Chris Capurso 15b06780ab
remove nil response to 404 translation for PatchOperation (#13167)
* remove nil response to 404 translation for PatchOperation

* go get vault-plugin-secrets-kv@master
2021-11-23 13:57:22 -05:00
akshya96 f77223bfe5
Authenticate to "login" endpoint for non-existent mount path bug (#13162)
* changing response from missing client token to permission denied

* removing todo comment

* fix tests

* adding changelog

* fixing changelog
2021-11-22 17:06:59 -08:00
divyapola5 5236fe93aa
Add a new parameter "allowed_managed_keys" to mount config (#13202)
* Add a new parameter "allowed_managed_keys" to mount config

* Adjust formatting in mount.go

* Add changelog entry
2021-11-21 19:08:38 -06:00
Nick Cabatoff ff74f49047
Move to go 1.17 (#12868)
Also ensure that the go 1.17 breaking changes to net.ParseCIDR don't make us choke on stored CIDRs that were acceptable to older Go versions.
2021-10-21 09:32:03 -04:00
Chris Capurso 6e95c59762
change return in handler test to explicit nil (#12884) 2021-10-20 13:41:51 -04:00
Chris Capurso eb6df00992
add retry logic when kv is upgrading in handler test (#12864)
* add retry logic when kv is upgrading in handler test

* make retry func for kv cli test more generic

* use ticker for kv retry logic in tests
2021-10-20 08:44:56 -04:00
Austin Gebauer a302293c9a
Modify error response format for resources protected by OIDC access tokens (#12840) 2021-10-15 11:22:44 -07:00
Austin Gebauer 0551f91068
Adds OIDC Token and UserInfo endpoints (#12711) 2021-10-13 18:59:36 -07:00
Chris Capurso bbb4ab4a41
Add HTTP PATCH support to KV (#12687)
* handle HTTP PATCH requests as logical.PatchOperation

* update go.mod, go.sum

* a nil response for logical.PatchOperation should result in 404

* respond with 415 for incorrect MIME type in PATCH Content-Type header

* add abstraction to handle PatchOperation requests

* add ACLs for patch

* Adding JSON Merge support to the API client

* add HTTP PATCH tests to check high level response logic

* add permission-based 'kv patch' tests in prep to add HTTP PATCH

* adding more 'kv patch' CLI command tests

* fix TestHandler_Patch_NotFound

* Fix TestKvPatchCommand_StdinValue

* add audit log test for HTTP PATCH

* patch CLI changes

* add patch CLI tests

* change JSONMergePatch func to accept a ctx

* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag

* go fmt

* add a test to verify patching works by default with the root token

* add changelog entry

* get vault-plugin-secrets-kv@add-patch-support

* PR feedback

* reorder some imports; go fmt

* add doc comment for HandlePatchOperation

* add json-patch@v5.5.0 to go.mod

* remove unnecessary cancelFunc for WriteBytes

* remove default for -method

* use stable version of json-patch; go mod tidy

* more PR feedback

* temp go get vault-plugin-secrets-kv@master until official release

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
2021-10-13 15:24:31 -04:00
hghaf099 ad2ef412cc
Customizing HTTP headers in the config file (#12485)
* Customizing HTTP headers in the config file

* Add changelog, fix bad imports

* fixing some bugs

* fixing interaction of custom headers and /ui

* Defining a member in core to set custom response headers

* missing additional file

* Some refactoring

* Adding automated tests for the feature

* Changing some error messages based on some recommendations

* Incorporating custom response headers struct into the request context

* removing some unused references

* fixing a test

* changing some error messages, removing a default header value from /ui

* fixing a test

* wrapping ResponseWriter to set the custom headers

* adding a new test

* some cleanup

* removing some extra lines

* Addressing comments

* fixing some agent tests

* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before

* Removing default custom headers, and renaming some function varibles

* some refacotring

* Refactoring and addressing comments

* removing a function and fixing comments
2021-10-13 11:06:33 -04:00