Commit Graph

551 Commits

Author SHA1 Message Date
Alexander Scheel ef7dd8c1bb
Add fix for Go x/crypto/ocsp failure case (#20181)
* Add fix for Go x/crypto/ocsp failure case

When calling ocsp.ParseRequest(req, issue) with a non-nil issuer on a
ocsp request which _unknowingly_ contains an entry in the
BasicOCSPResponse's certs field, Go incorrectly assumes that the issuer
is a direct parent of the _first_ certificate in the certs field,
discarding the rest.

As documented in the Go issue, this is not a valid assumption and thus
causes OCSP verification to fail in Vault with an error like:

> bad OCSP signature: crypto/rsa: verification error

which ultimately leads to a cert auth login error of:

> no chain matching all constraints could be found for this login certificate

We address this by using the unsafe issuer=nil argument, taking on the
task of validating the OCSP response's signature as best we can in the
absence of full chain information on either side (both the trusted
certificate whose OCSP response we're verifying and the lack of any
additional certs the OCSP responder may have sent).

See also: https://github.com/golang/go/issues/59641

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test case with Vault PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-17 11:48:19 -04:00
Anton Averchenkov 624acfe62a
openapi: Better comments for OperationPrefix/Verb/Suffix (#20162) 2023-04-13 18:47:14 -04:00
Jason O'Donnell ec9e08c931
sdk/ldaputil: add connection_timeout configurable (#20144)
* sdk/ldaputil: add connection_timeout configurable

* changelog

* Update doc

* Fix test

* Change default to 30s
2023-04-13 12:43:28 -04:00
Tom Proctor 9aa9686c81
Simplify tracking of external plugins (#20009) 2023-04-12 18:34:35 +01:00
Mike Palmiotto 3ebe35b0d4
Fix RFC3339 comment typo (#20100) 2023-04-11 20:11:05 +00:00
Thy Ton fcf06d5874
feat: add plugin metadata to audit logging (#19814) 2023-04-06 00:41:07 -07:00
Anton Averchenkov 35d07e91d2
Fix OpenAPI-related tests (#19983) 2023-04-04 18:27:20 -04:00
Anton Averchenkov 4564a3534b
openapi: Improve operationId/request/response naming strategy (#19319) 2023-04-04 13:14:40 -04:00
Anton Averchenkov 833021759f
Fix response structure validation tests for non-2xx responses (#19726) 2023-03-23 16:33:44 -04:00
Alexander Scheel 2fcaec4b21
Add support for HEAD operations (#19520)
* Add header operation to sdk/logical

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support for routing HEAD operations

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-21 18:37:17 -04:00
Anton Averchenkov 4d10063cbd
openapi: Fix logic for labeling unauthenticated/sudo paths (#19600) 2023-03-20 13:25:09 -04:00
Tom Proctor f1f5c8444a
Suppress event broker not started log warning (#19593) 2023-03-20 11:14:14 +00:00
Hamid Ghaf 27bb03bbc0
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Anton Averchenkov e19dc98016
Ignore special HTTP fields in response validation tests (#19530) 2023-03-14 13:21:20 -04:00
Alexander Scheel ab3d6d61e0
Add support for importing RSA-PSS keys into Transit (#19519)
* Add support for importing RSA-PSS keys in Transit

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-13 17:03:01 +00:00
Tom Proctor 480c1dc85b
sdk: Fix fmt + add FieldType test (#19493)
* sdk: Fix fmt + add FieldType test

* Add test comment
2023-03-10 18:07:48 +00:00
valli_0x 8e9680223d
bug: correct sdk handling of the zero int64 value (#18729)
* bug: correct handling of the zero int64 value

* Update changelog/18729.txt

---------

Co-authored-by: valli_0x <personallune@mail.ru>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-03-09 11:25:45 +00:00
Austin Gebauer 6e18136365
ldaputil: adds comment on available text/template functions (#19469) 2023-03-06 19:38:42 -08:00
Jakob Beckmann 078a245939
Allow alias dereferencing in LDAP searches (#18230)
* impr(auth/ldap): allow to dereference aliases in searches

* docs: add documentation for LDAP alias dereferencing

* chore(auth/ldap): add changelog entry for PR 18230

* chore: run formatter

* fix: update default LDAP configuration with new default

* Update website/content/docs/auth/ldap.mdx

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>

* docs(ldap): add alias dereferencing to API docs for LDAP

---------

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
2023-02-24 13:49:17 -05:00
Alexander Scheel 809957aac0
Refactor OCSP client to support better retries (#19345)
Mirror NSS's GET-vs-POST selection criteria, wherein GET is preferred
over POST (as the former might be a response from a cached CDN entry,
whereas the latter might hit a live responder). However, only accept it
if it definitively says "Good" or "Revoked" -- trigger a POST request
when an unknown or failure status is seen.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-02-24 13:18:37 -05:00
Tom Proctor 5b52184766
Update x/net and x/crypto/ssh (#19282)
* Update x/net and x/crypto/ssh in api and sdk

* go mod tidy in root go module
2023-02-22 13:46:12 +00:00
Christopher Swenson 404d7a57bb
events: WS protobuf messages should be binary (#19232)
The [WebSockets spec](https://www.rfc-editor.org/rfc/rfc6455) states
that text messages must be valid UTF-8 encoded strings, which protobuf
messages virtually never are. This now correctly sends the protobuf events
as binary messages.

We change the format to correspond to CloudEvents, as originally intended,
and remove a redundant timestamp and newline.

We also bump the eventlogger to fix a race condition that this code triggers.
2023-02-17 11:38:03 -08:00
Scott Miller ccab6ab676
Update go-kms-wrapping and it's gcpc wrapper to pick up google-cloud-go migration (#19219) 2023-02-16 11:21:25 -06:00
Daniel Huckins 7fde5ecb83
Validate response schema for integration tests (#19043)
* add RequestResponseCallback to core/options

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* pass in router and apply function on requests

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add callback

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* cleanup

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Update vault/core.go

* bad typo...

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use pvt interface, can't downcast to child struct

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* finer grained errors

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* trim path for backend

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* remove entire mount point instead of just the first part of url

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/testing.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* add doc string

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* update docstring

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* reformat

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-02-15 14:57:57 -05:00
Tom Proctor bd592120e2
Convert events metadata type to google.protobuf.Struct (#19130) 2023-02-10 18:58:03 +00:00
Christopher Swenson 7d3d404ee2
events: Add websockets and command (#19057)
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.

This can be used by either `wscat` or the new
`vault events subscribe`:

e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

and

```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-09 13:18:58 -08:00
Scott Miller 78522ed923
Update specific wrappers to v2.0.7 (#19101)
* Update go-kms-wrapping to v2.0.7

* Update specific wrappers to v2.0.7

* Attempt to fix aead
2023-02-09 12:19:50 -06:00
Nick Cabatoff ec2af04ec6
Remove the last vestiges of sdk/version. (#19068) 2023-02-08 12:30:27 -05:00
Nick Cabatoff 53afd2627b
Make API not depend on SDK (#18962) 2023-02-06 09:41:56 -05:00
Christopher Swenson 5864075c30
Add events sending routed from plugins (#18834)
This isn't perfect for sure, but it's solidifying and becoming a useful
base to work off.

This routes events sent from auth and secrets plugins to the main
`EventBus` in the Vault Core. Events sent from plugins are automatically
tagged with the namespace and plugin information associated with them.
2023-02-03 13:24:16 -08:00
Anton Averchenkov 7b356f0366
openapi: Add default values to the thing_mount_path parameters (#18935) 2023-01-31 19:37:16 -05:00
Max Bowsher 9d863a92ce
Fix multiple OpenAPI generation issues with new AST-based generator (#18554)
* Regexp metacharacter `.` should be escaped when used literally

The paths including `/.well-known/` in the Vault API could currently
technically be invoked with any random character in place of the dot.

* Replace implementation of OpenAPI path translator with regexp AST-based one

* Add changelog

* Typo fix from PR review - thanks!

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Add comment based on review feedback

* Change style of error handling as suggested in code review

* Make a further tweak to the handling of the error case

* Add more tests, testing cases which fail with the previous implementation

* Resolve issue with a test, and improve comment

---------

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-01-31 16:27:39 -05:00
Steven Clark b737777f15
Address pki::TestAutoRebuild flakiness (#18903)
* Address pki::TestAutoRebuild flakiness

 - Wait for a CRL change before progressing to the next step after
   we change configuration. Prior to this we would be racing against
   the CRL reloading from the configuration change.
2023-01-30 21:38:38 +00:00
Matt Schultz 20c1645329
Add some managed key build tags to support incoming enterprise feature work. (#18902) 2023-01-30 14:57:01 -06:00
Rachel Culpepper c151d5fd58
Vault-11713: Add the ability for transit to use managed keys (#18844)
* add the ability for transit to use managed keys

* add fields for rotate

* reorganize managed keys for better error messaging

* add aad for managed keys

* change encrypting key interface

* fix import error

* fix fmt errors

* move managed key rotation to a separate function

* fix fmt issue

* fix import error

* fix go.mod issue

* fix go.mod

* remove extra rotate param

* fix locking issue in rotation

* fix errors in policy tests

* move managed key params to fit existing policy functions

* fix sign and verify

* fix policy tests

* fix test error
2023-01-27 13:39:58 -06:00
Nick Cabatoff 9ca78845b7
Proposed more streamlined approach to validating schema responses. (#18865) 2023-01-27 14:34:55 -05:00
Kit Haines 27be887bfd
Vault 9406 enablement certs need userid handling in role (#18397)
* The fields.

* UserID set, add to certificate

* Changelog.

* Fix test (set default).

* Add UserID constant to certutil, revert extension changes

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add user_ids as field for leaf signing

Presumably, this isn't necessary for CAs, given that CAs probably don't
have a user ID corresponding to them.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Support setting multiple user_ids in Subject

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow any User ID with sign-verbatim

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for User IDs in PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add docs about user_ids, allowed_user_ids

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-25 13:13:54 -05:00
Matt Schultz bf3e266929
Adds managed key usages for MAC generate/verify and RNG. (#18291)
* Adds managed key usages for MAC generate/verify and RNG.

* Remove MAC-related key usages from managed key in favor of sign/verify.

* Remove context from random source managed key interface.
2023-01-24 13:25:09 -06:00
Hamid Ghaf edbf093290
Allow Token Create Requests To Be Replicated (#18689)
* Allow Token Create Requests To Be Replicated

* adding a test

* revert a test
2023-01-24 14:00:27 -05:00
Anton Averchenkov 5a6092f8ab
Add approle's remaining response schema definitions (#18772) 2023-01-24 13:12:41 -05:00
Hamid Ghaf 65a41d4f08
named Login MFA methods (#18610)
* named MFA method configurations

* fix a test

* CL

* fix an issue with same config name different ID and add a test

* feedback

* feedback on test

* consistent use of passcode for all MFA methods (#18611)

* make use of passcode factor consistent for all MFA types

* improved type for MFA factors

* add method name to login CLI

* minor refactoring

* only accept MFA method name with its namespace path in the login request MFA header

* fix a bug

* fixing an ErrorOrNil return value

* more informative error message

* Apply suggestions from code review

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* feedback

* test refactor a bit

* adding godoc for a test

* feedback

* remove sanitize method name

* guard a possbile nil ref

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-23 15:51:22 -05:00
Alexander Scheel 2f5c7458b2
Update x/crypto + x/net (#18794)
* Update golang.org/x/crypto version

go get -u golang.org/x/crypto && go mod tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update golang.org/x/crypto version in api

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update golang.org/x/crypto version in sdk

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-23 19:11:04 +00:00
Alexander Scheel 6930568076
Add path based primary write forwarding (PBPWF) - OSS (#18735)
* Add WriteForwardedStorage to sdk's plugin, logical in OSS

This should allow backends to specify paths to forward write
(storage.Put(...) and storage.Delete(...)) operations for.

Notably, these semantics are subject to change and shouldn't yet be
relied on.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Collect paths for write forwarding in OSS

This adds a path manager to Core, allowing tracking across all Vault
versions of paths which could use write forwarding if available. In
particular, even on OSS offerings, we'll need to template {{clusterId}}
into the paths, in the event of later upgrading to Enterprise. If we
didn't, we'd end up writing paths which will no longer be accessible
post-migration, due to write forwarding now replacing the sentinel with
the actual cluster identifier.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add forwarded writer implementation to OSS

Here, for paths given to us, we determine if we need to do cluster
translation and perform local writing. This is the OSS variant.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Wire up mount-specific request forwarding in OSS

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clarify that state lock needs to be held to call HAState in OSS

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Move cluster sentinel constant to sdk/logical

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Expose ClusterID to Plugins via SystemView

This will let plugins learn what the Cluster's ID is, without having to
resort to hacks like writing a random string to its cluster-prefixed
namespace and then reading it once it has replicated.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add GRPC ClusterID implementation

For any external plugins which wish to use it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-20 16:36:18 -05:00
Christopher Swenson 4a93097895
Use schema for events in event broker (#18693)
For the new events schema. Based on the CloudEvents schema.
2023-01-20 10:18:23 -08:00
Max Bowsher 4c5f583f39
OpenAPI `generic_mount_paths` follow-up (#18663)
* OpenAPI `generic_mount_paths` follow-up

An incremental improvement within larger context discussed in #18560.

* Following the revert in #18617, re-introduce the change from
  `{mountPath}` to `{<path-of-mount>_mount_path}`; this is needed, as
  otherwise paths from multiple plugins would clash - e.g. almost every
  auth method would provide a conflicting definition for
  `auth/{mountPath}/login`, and the last one written into the map would
  win.

* Move the half of the functionality that was in `sdk/framework/` to
  `vault/logical_system.go` with the rest; this is needed, as
  `sdk/framework/` gets compiled in to externally built plugins, and
  therefore there may be version skew between it and the Vault main
  code. Implementing the `generic_mount_paths` feature entirely on one
  side of this boundary frees us from problems caused by this.

* Update the special exception that recognizes `system` and `identity`
  as singleton mounts to also include the other two singleton mounts,
  `cubbyhole` and `auth/token`.

* Include a comment that documents to restricted circumstances in which
  the `generic_mount_paths` option makes sense to use:

	    // Note that for this to actually be useful, you have to be using it with
	    // a Vault instance in which you have mounted one of each secrets engine
	    // and auth method of types you are interested in, at paths which identify
	    // their type, and for the KV secrets engine you will probably want to
	    // mount separate kv-v1 and kv-v2 mounts to include the documentation for
	    // each of those APIs.

* Fix tests

Also remove comment "// TODO update after kv repo update" which was
added 4 years ago in #5687 - the implied update has not happened.

* Add changelog

* Update 18663.txt
2023-01-17 23:07:11 -05:00
Christopher Swenson b95beeb675
Add basic event bus broker stub (#18640)
Creates a new `eventbus` package under `vault` with
an implementation of the `go-eventlogger` broker.

Also creates a stub of a common broker that will be accessible
in the core, and creates a simple event sending interface.
2023-01-17 13:34:37 -08:00
Max Bowsher 91855bce22
Fix a very old comment to include query-string parameters (#18557)
Vault has gradually had the ability to pass query-string parameters
added to GET, then DELETE, and now recently LIST requests. Update
a comment which seems to date back to when no query-string parameters
were used at all.
2023-01-17 12:29:30 -05:00
Anton Averchenkov 6ae09f3074
Add AppRole response schema validation tests (#18636)
This PR modifies every test in `builtin/credentials/approle/path_role_test.go` with new validation checks to ensure that approle/path_role  successful responses align with the declared response schema.

It also introduces a test helper in `sdk/helper/testhelpers`:

```go
func FindResponseSchema(t *testing.T, ...)
```

This test helper will be useful for all plugins that require similar response schema validation in tests.

### Background

This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
2023-01-13 15:23:36 -05:00
Anton Averchenkov 9696600e59
Add response schema validation methods & test helpers (#18635)
This pull request adds 3 functions (and corresponding tests):

`testhelpers/response_validation.go`:

  - `ValidateResponse`
  - `ValidateResponseData`
  
field_data.go:

  - `ValidateStrict` (has the "strict" validation logic)

The functions are primarily meant to be used in tests to ensure that the responses are consistent with the defined response schema. An example of how the functions can be used in tests can be found in #18636.

### Background

This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
2023-01-13 14:55:56 -05:00
Anton Averchenkov c7f40361eb
Revert "Add mount path into the default generated openapi.json spec (#17926)" (#18617)
* Revert "Add mount path into the default generated openapi.json spec (UI) (#17926)"

This reverts commit db8efac708e5385ec871be9558507eeaf54ac972.

* Revert "Remove `generic_mount_paths` field (#18558)"

This reverts commit 79c8f626c59ca11bb8e7f460d40b09f5e0cec76d.
2023-01-10 11:16:59 -05:00
Mike Palmiotto cb3406b1eb
plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Matt Schultz dcfb58e18c
Add ManagedKeyRandomSource interface to SDK. (#18269) 2022-12-08 14:17:41 -06:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Anton Averchenkov 493040d147
Add mount path into the default generated openapi.json spec (UI) (#17926) 2022-12-08 12:15:54 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
Matt Schultz b2a7cf158f
Add ManagedMACKey interface to SDK. (#18231) 2022-12-05 17:26:16 -06:00
Anton Averchenkov a54678fb6b
Add logic to generate openapi response structures (#18192) 2022-12-05 11:11:06 -05:00
Christopher Swenson eba490ccef
Check if sys view is missing in GRPC sys view (#18210)
And return an error instead of panicking.

This situation can occur if a plugin attempts to access the system
view during setup when Vault is checking the plugin metadata.

Fixes #17878.
2022-12-02 10:12:05 -08:00
Violet Hynes 3d7f9a402f
VAULT-6368 Metrics-only listener for Agent (#18101)
* VAULT-6368 Metrics-only listener for Agent

* VAULT-6368 changelog

* VAULT-6368 Update config to use string instead of bool

* VAULT-6368 Fix leftover code

* VAULT-6368 Fix changelog

* VAULT-6368 fix typo

* VAULT-6368 recommended doc update

* VAULT-6368 use != over !(==)
2022-11-25 16:00:56 -05:00
Scott Miller b51b2a7027
Add cached OCSP client support to Cert Auth (#17093)
* wip

* Add cached OCSP client support to Cert Auth

* ->pointer

* Code cleanup

* Fix unit tests

* Use an LRU cache, and only persist up to 1000 of the most recently used values to stay under the storage entry limit

* Fix caching, add fail open mode parameter to cert auth roles

* reduce logging

* Add the retry client and GET then POST logic

* Drop persisted cache, make cache size configurable, allow for parallel testing of multiple servers

* dead code

* Update builtin/credential/cert/path_certs.go

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Hook invalidate to reinit the ocsp cache size

* locking

* Conditionally init the ocsp client

* Remove cache size config from cert configs, it's a backend global

* Add field

* Remove strangely complex validity logic

* Address more feedback

* Rework error returning logic

* More edge cases

* MORE edge cases

* Add a test matrix with a builtin responder

* changelog

* Use an atomic for configUpdated

* Actually use ocsp_enabled, and bind to a random port for testing

* Update builtin/credential/cert/path_login.go

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Refactor unit tests

* Add status to cache

* Make some functions private

* Rename for testing, and attribute

* Up to date gofumpt

* remove hash from key, and disable the vault dependent unit test

* Comment out TestMultiOCSP

* imports

* more imports

* Address semgrep results

* Attempt to pass some sort of logging to test_responder

* fix overzealous search&replace

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-11-21 10:39:24 -06:00
Steven Clark 610a4ede82
certutil.ParseHexFormatted fails parsing 80 hex and above (#18018)
- Switch to using ParseUint of 8 bits to parse the hex values properly
   as ParseInt limited to 8 bits will only handle values up to 127
   decimal or 7F.
2022-11-17 15:04:37 -05:00
AnPucel b554a453e2
Make request objects required (#17909) 2022-11-11 14:05:12 -08:00
Anton Averchenkov f9fac68980
Revert "Add mount path into the default generated openapi.json spec (#17839)" (#17890)
This reverts commit 02064eccb42bb2ec1a3d12ec0d49c661312acd2d.
2022-11-10 15:39:53 -08:00
Anton Averchenkov f3aea876b9
Add mount path into the default generated openapi.json spec (#17839)
The current behaviour is to only add mount paths into the generated `opeanpi.json` spec if a `generic_mount_paths` flag is added to the request. This means that we would have to maintain two different `openapi.json` files, which is not ideal. The new solution in this PR is to add `{mount_path}` into every path with a default value specified:

```diff
--    "/auth/token/accessors/": {
++    "/auth/{mount_path}/accessors/": {
      "parameters": [
        {
          "name": "mount_path",
          "description": "....",
          "in": "path",
          "schema": {
            "type": "string",
++          "default": "token"
          }
        }
      ],
```

Additionally, fixed the logic to generate the `operationId` (used to generate method names in the code generated from OpenAPI spec). It had a bug where the ID had `mountPath` in it. The new ID will look like this:

```diff
-- "operationId": "listAuthMountpathAccessors",
++ "operationId": "listTokenAccessors",
```
2022-11-10 15:44:43 -05:00
Peter Wilson 469c102fd3
Removed the legacy env var: LOGXI_FORMAT (#17822)
* Removed the legacy env var: LOGXI_FORMAT

* Added changelog

* Actually filled in the CL

* Added the name of legacy env var
2022-11-04 22:16:18 +00:00
akshya96 2945924b2b
Vault 8305 Prevent Brute Forcing in Auth methods : Setting user lockout configuration (#17338)
* config file changes

* lockout config changes

* auth tune r/w and auth tune

* removing changes at enable

* removing q.Q

* go mod tidy

* removing comments

* changing struct name for config file

* fixing mount tune

* adding test file for user lockout

* fixing comments and add changelog

* addressing comments

* fixing mount table updates

* updating consts in auth_tune

* small fixes

* adding hcl parse test

* fixing config compare

* fixing github comments

* optimize userlockouts.go

* fixing test

* minor changes

* adding comments

* adding sort to flaky test

* fix flaky test
2022-11-01 11:02:07 -07:00
John-Michael Faircloth b92d42af2e
update vault sdk and api go-plugin version to v1.4.5 (#17734)
* update sdk's go-plugin version

* update api go.mod

* update api go.mod go version and go mod tidy
2022-10-31 08:52:04 -05:00
Alexander Scheel 1733d2a3d6
Add support for PKCSv1_5_NoOID signatures (#17636)
* Add support for PKCSv1_5_NoOID signatures

This assumes a pre-hashed input has been provided to Vault, but we do
not write the hash's OID into the signature stream. This allows us to
generate the alternative PKCSv1_5_NoOID signature type rather than the
existing PKCSv1_5_DERnull signature type we presently use.

These are specified in RFC 3447 Section 9.2.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Exclude new none type from PSS based tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for PKCS#1v1.5 signatures

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-27 08:26:20 -04:00
akshya96 1e189016e2
update protoc version to 3.21.7 oss (#17499)
* update protoc to 3.21.7

* adding changelog
2022-10-26 16:49:44 -07:00
Luis (LT) Carbonell 3425f8b36c
Add Paging Interface for LDAP Connection (#17640) 2022-10-26 14:05:53 -05:00
Alexander Scheel 09939f0ba9
Add AD mode to Transit's AEAD ciphers (#17638)
* Allow passing AssociatedData factories in keysutil

This allows the high-level, algorithm-agnostic Encrypt/Decrypt with
Factory to pass in AssociatedData, and potentially take multiple
factories (to allow KMS keys to work). On AEAD ciphers with a relevant
factory, an AssociatedData factory will be used to populate the
AdditionalData field of the SymmetricOpts struct, using it in the AEAD
Seal process.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add associated_data to Transit Encrypt/Decrypt API

This allows passing the associated_data (the last AD in AEAD) to
Transit's encrypt/decrypt when using an AEAD cipher (currently
aes128-gcm96, aes256-gcm96, and chacha20-poly1305). We err if this
parameter is passed on non-AEAD ciphers presently.

This associated data can be safely transited in plaintext, without risk
of modifications. In the event of tampering with either the ciphertext
or the associated data, decryption will fail.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add to documentation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-24 13:41:02 -04:00
Violet Hynes 5861c51e70
VAULT-8719 Support data array for alias clash error response so UI/machines can understand error (#17459)
* VAULT-8719 Support data array for alias clash error response so UI can understand error

* VAULT-8719 Changelog

* VAULT-8719 Update alias mount update logic

* VAULT-8719 Further restrict IsError()
2022-10-17 14:46:25 -04:00
Violet Hynes 4ba81e4174
Delete empty test (#17467) 2022-10-07 14:11:52 -04:00
Ruben De Visscher 1f0cf558a9
Fix for duplicate SANs in signed certificates (#16700)
* Fix for duplicate SANs in signed certificates when othernames are present in the CSR SAN extension and UseCSRValues is true.

When UseCSRValues is true (as is the case on the sign-verbatim endpoint), all extensions including Subject Alternative Names are copied from the CSR to the final certificate.
If the Subject Alternative Name in question contains any othernames (such as a Microsoft UPN) the SAN extension is added again as a workaround for an encoding issue (in function HandleOtherSANs).
Having duplicate x509v3 extensions is invalid and is rejected by openssl on Ubuntu 20.04, and also by Go since https://github.com/golang/go/issues/50988 (including in Go 1.19).

In this fix I do not add the extension from the CSR if it will be added during HandleOtherSANs.

* Added unittest and changelog entry.
2022-10-07 12:19:08 -04:00
Alexander Scheel 412603befd
Fix RevocationSigAlg provisioning in GCP (#17449)
* Fix RevocationSigAlg provisioning in GCP

GCP restricts keys to a certain type of signature, including hash
algorithm, so we must provision our RevocationSigAlg from the root
itself unconditionally in order for GCP to work.

This does change the default, but only for newly created certificates.

Additionally, we clarify that CRL building is not fatal to the import
process.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add inverse mapping for SignatureAlgorithm

By default we'd use .String() on x509.SignatureAlgorithm, but this
doesn't round-trip. Switch to a custom map that is round-trippable
and matches the constant name as there is no other way to get this info
presently.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add test to ensure root creation sets rev_sig_alg

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Test round-tripping of SigAlgoNames, InvSigAlgoNames

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix failing Default Update test

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-10-06 17:50:49 -04:00
Tom Proctor 3aa2fe8d8f
Plugins: Tighten requirements for multiplexing (#17403)
Change the multiplexing key to use all `PluginRunner` config (converted to a struct which is comparable), so that plugins with the same name but different env, args, types, versions etc are not incorrectly multiplexed together.

Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
2022-10-05 09:29:29 +01:00
Steven Clark 10ecf10248
PKI: Add support for signature_bits param to the intermediate/generate api (#17388)
* PKI: Add support for signature_bits param to the intermediate/generate api

 - Mainly to work properly with GCP backed managed keys, we need to
   issue signatures that would match the GCP key algorithm.
 - At this time due to https://github.com/golang/go/issues/45990 we
   can't issue PSS signed CSRs, as the libraries in Go always request
   a PKCS1v15.
 - Add an extra check in intermediate/generate that validates the CSR's
   signature before providing it back to the client in case we generated
   a bad signature such as if an end-user used a GCP backed managed key
   with a RSA PSS algorithm.
   - GCP ignores the requested signature type and always signs with the
     key's algorithm which can lead to a CSR that says it is signed with
     a PKCS1v15 algorithm but is actually a RSA PSS signature

* Add cl

* PR feedback
2022-10-03 12:39:54 -04:00
vinay-gopalan c88d53c708
fix: remove premature GRPC client connection close from vault (#17321) 2022-09-29 14:11:34 -07:00
Steven Clark 1f459a2df6
PKI: Fix managed key signatures when using specified signature_bits (#17328)
* PKI: Fix managed key signatures when using specified signature_bits

 - When calling sign-intermediate and other apis with signature_bits
   value overridden with a backing managed key we did not use that
   value as tests for the private key type were not working.

* Add cl
2022-09-28 09:08:23 -04:00
akshya96 542570c289
Return errInvalidCredentials when wrong credentials is provided for existent users (#17104)
* adding errInvalidCredentials

* fixing tests

* add changelog

* fixing fmt errors

* test if routeErr is seen externally and fixing error comment

* adding fmt changes

* adding comments
2022-09-27 16:49:14 -07:00
Violet Hynes 5bcd0c31cb
VAULT-6938 Remove license from being cache exempt (#17265)
* VAULT-6938 Remove license from being cache exempt

* VAULT-6938 Add changelog

* VAULT-6938 Typo in changelog name

* Update changelog/17265.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-09-26 10:26:07 -04:00
Alexander Scheel 3015689848
Update tlsutil to v0.1.2 for ecdsa algo naming (#17282)
* Update tlsutil in sdk

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update tlsutil in API

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update tlsutil to v0.1.2 for ecdsa algo naming

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2022-09-23 08:28:40 -04:00
Tom Proctor e21995fa27
Plugins: Update running version everywhere running sha256 is set (#17292) 2022-09-23 11:19:38 +01:00
Christopher Swenson 2c8e88ab67
Check if plugin version matches running version (#17182)
Check if plugin version matches running version

When registering a plugin, we check if the request version matches the
self-reported version from the plugin. If these do not match, we log a
warning.

This uncovered a few missing pieces for getting the database version
code fully working.

We added an environment variable that helps us unit test the running
version behavior as well, but only for approle, postgresql, and consul
plugins.

Return 400 on plugin not found or version mismatch

Populate the running SHA256 of plugins in the mount and auth tables (#17217)
2022-09-21 12:25:04 -07:00
Austin Gebauer 65b851bc2c
Fixes concurrent map writes in GRPC plugin server setup (#17247)
* Fixes concurrent map writes in GRPC plugin server setup

* move lock closer to critical section
2022-09-21 11:04:20 -07:00
Steven Clark e5319bcc3c
Bump version in SDK to 1.13 for next major release (#17233) 2022-09-20 15:40:06 -04:00
Steven Clark dae2ef535b
Update protos to match update of protobuf go library (#17215) 2022-09-19 16:45:44 -04:00
Steven Clark 05a5928b8d
Update missing go-kms-wrapping v2 dep and address some ENT->OSS drift (#17178)
* Update missing go-kms-wrapping v2 dep and address some ENT->OSS drift

* Bump go-kms-wrapping/wrappers/gcpckms/v2 to v2.0.1
2022-09-19 10:23:40 -04:00
Christopher Swenson b136a7ecd8
Add plugin version to GRPC interface (#17088)
Add plugin version to GRPC interface

Added a version interface in the sdk/logical so that it can be shared between all plugin types, and then wired it up to RunningVersion in the mounts, auth list, and database systems.

I've tested that this works with auth, database, and secrets plugin types, with the following logic to populate RunningVersion:

If a plugin has a PluginVersion() method implemented, then that is used
If not, and the plugin is built into the Vault binary, then the go.mod version is used
Otherwise, the it will be the empty string.
My apologies for the length of this PR.

* Placeholder backend should be external

We use a placeholder backend (previously a framework.Backend) before a
GRPC plugin is lazy-loaded. This makes us later think the plugin is a
builtin plugin.

So we added a `placeholderBackend` type that overrides the
`IsExternal()` method so that later we know that the plugin is external,
and don't give it a default builtin version.
2022-09-15 16:37:59 -07:00
Josh Black 1e6401a8eb
make proto (#17120) 2022-09-13 16:06:11 -04:00
Josh Black 6d94dd991d
merkle sync undo logs (#17103) 2022-09-13 10:03:19 -07:00
Hamid Ghaf ed0a9feb7f
running make proto (#17106) 2022-09-13 09:40:12 -04:00
Mike Palmiotto 9849af8663
Add deprecation status to plugin api and cli (#17077)
* api: Add deprecation status to plugin endpoints

* cli: Add -detailed flag to `plugin list`

* docs: Update plugin list/info docs
2022-09-09 16:03:07 -04:00
Tom Proctor 65adf42d48
Support running versioned plugins from the catalog (#17015) 2022-09-09 18:14:26 +01:00
Tom Proctor aa50e42fca
Support version selection for database plugins (#16982)
* Support version selection for database plugins
* Don't consider unversioned plugins for version selection algorithm
* Added version to 'plugin not found' error
* Add PluginFactoryVersion function to avoid changing sdk/ API
2022-09-09 17:32:28 +01:00
Max Coulombe f9b5d1a563
Multiplexing opt out flag (#16972)
* added mplexing opt-out flag
2022-09-08 11:32:46 -04:00
Josh Black d8e0a13aae
update gofumpt to 0.3.1 and reformat the repo (#17055)
* update gofumpt to 0.3.1 and reformat the repo

* output the version of the formatter we're using
2022-09-07 17:31:20 -07:00
Mike Palmiotto bf744e3bde
Handle deprecated builtins (#17005)
* core: Handle deprecated mounts on enable and unseal
* changelog: Deprecation Status handling
* core: Add Pending Removal override var
* core: Add some documentation for Pending Removal override
2022-09-06 15:49:35 -04:00
Steven Clark 98fffbe949
Address failures in FIPS builds around new transit RSA PSS tests (#17024)
- When we added new tests that validate the RSA PSS feature, they
   work properly on normal Go builds, but tests underneath the Boring
   Crypto fips implementations fail due to a lack of SHA3 support in
   FIPS 140-2.
2022-09-06 13:40:13 -04:00
Scott Miller 606edb66d6
Add support for a dedicated HMAC type in Transit. (#16668)
* Get import correct

* limits, docs

* changelog

* unit tests

* And fix import for hmac unit test

* typo

* Update website/content/api-docs/secret/transit.mdx

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

* Update builtin/logical/transit/path_keys.go

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

* Validate key sizes a bit more carefully

* Update sdk/helper/keysutil/policy.go

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>

Co-authored-by: Matt Schultz <975680+schultz-is@users.noreply.github.com>
2022-09-06 10:17:58 -05:00