Commit graph

496 commits

Author SHA1 Message Date
Thy Ton fcf06d5874
feat: add plugin metadata to audit logging (#19814) 2023-04-06 00:41:07 -07:00
Anton Averchenkov 35d07e91d2
Fix OpenAPI-related tests (#19983) 2023-04-04 18:27:20 -04:00
Anton Averchenkov 4564a3534b
openapi: Improve operationId/request/response naming strategy (#19319) 2023-04-04 13:14:40 -04:00
Anton Averchenkov 833021759f
Fix response structure validation tests for non-2xx responses (#19726) 2023-03-23 16:33:44 -04:00
Alexander Scheel 2fcaec4b21
Add support for HEAD operations (#19520)
* Add header operation to sdk/logical

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add support for routing HEAD operations

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-21 18:37:17 -04:00
Anton Averchenkov 4d10063cbd
openapi: Fix logic for labeling unauthenticated/sudo paths (#19600) 2023-03-20 13:25:09 -04:00
Tom Proctor f1f5c8444a
Suppress event broker not started log warning (#19593) 2023-03-20 11:14:14 +00:00
Hamid Ghaf 27bb03bbc0
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
Anton Averchenkov e19dc98016
Ignore special HTTP fields in response validation tests (#19530) 2023-03-14 13:21:20 -04:00
Alexander Scheel ab3d6d61e0
Add support for importing RSA-PSS keys into Transit (#19519)
* Add support for importing RSA-PSS keys in Transit

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add changelog entry

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

---------

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-13 17:03:01 +00:00
Tom Proctor 480c1dc85b
sdk: Fix fmt + add FieldType test (#19493)
* sdk: Fix fmt + add FieldType test

* Add test comment
2023-03-10 18:07:48 +00:00
valli_0x 8e9680223d
bug: correct sdk handling of the zero int64 value (#18729)
* bug: correct handling of the zero int64 value

* Update changelog/18729.txt

---------

Co-authored-by: valli_0x <personallune@mail.ru>
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-03-09 11:25:45 +00:00
Austin Gebauer 6e18136365
ldaputil: adds comment on available text/template functions (#19469) 2023-03-06 19:38:42 -08:00
Jakob Beckmann 078a245939
Allow alias dereferencing in LDAP searches (#18230)
* impr(auth/ldap): allow to dereference aliases in searches

* docs: add documentation for LDAP alias dereferencing

* chore(auth/ldap): add changelog entry for PR 18230

* chore: run formatter

* fix: update default LDAP configuration with new default

* Update website/content/docs/auth/ldap.mdx

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>

* docs(ldap): add alias dereferencing to API docs for LDAP

---------

Co-authored-by: tjperry07 <tjperry07@users.noreply.github.com>
2023-02-24 13:49:17 -05:00
Alexander Scheel 809957aac0
Refactor OCSP client to support better retries (#19345)
Mirror NSS's GET-vs-POST selection criteria, wherein GET is preferred
over POST (as the former might be a response from a cached CDN entry,
whereas the latter might hit a live responder). However, only accept it
if it definitively says "Good" or "Revoked" -- trigger a POST request
when an unknown or failure status is seen.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-02-24 13:18:37 -05:00
Tom Proctor 5b52184766
Update x/net and x/crypto/ssh (#19282)
* Update x/net and x/crypto/ssh in api and sdk

* go mod tidy in root go module
2023-02-22 13:46:12 +00:00
Christopher Swenson 404d7a57bb
events: WS protobuf messages should be binary (#19232)
The [WebSockets spec](https://www.rfc-editor.org/rfc/rfc6455) states
that text messages must be valid UTF-8 encoded strings, which protobuf
messages virtually never are. This now correctly sends the protobuf events
as binary messages.

We change the format to correspond to CloudEvents, as originally intended,
and remove a redundant timestamp and newline.

We also bump the eventlogger to fix a race condition that this code triggers.
2023-02-17 11:38:03 -08:00
Scott Miller ccab6ab676
Update go-kms-wrapping and it's gcpc wrapper to pick up google-cloud-go migration (#19219) 2023-02-16 11:21:25 -06:00
Daniel Huckins 7fde5ecb83
Validate response schema for integration tests (#19043)
* add RequestResponseCallback to core/options

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* pass in router and apply function on requests

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add callback

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* cleanup

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Update vault/core.go

* bad typo...

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use pvt interface, can't downcast to child struct

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* finer grained errors

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* trim path for backend

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* remove entire mount point instead of just the first part of url

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/testing.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* add doc string

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* update docstring

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* reformat

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-02-15 14:57:57 -05:00
Tom Proctor bd592120e2
Convert events metadata type to google.protobuf.Struct (#19130) 2023-02-10 18:58:03 +00:00
Christopher Swenson 7d3d404ee2
events: Add websockets and command (#19057)
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.

This can be used by either `wscat` or the new
`vault events subscribe`:

e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

and

```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-09 13:18:58 -08:00
Scott Miller 78522ed923
Update specific wrappers to v2.0.7 (#19101)
* Update go-kms-wrapping to v2.0.7

* Update specific wrappers to v2.0.7

* Attempt to fix aead
2023-02-09 12:19:50 -06:00
Nick Cabatoff ec2af04ec6
Remove the last vestiges of sdk/version. (#19068) 2023-02-08 12:30:27 -05:00
Nick Cabatoff 53afd2627b
Make API not depend on SDK (#18962) 2023-02-06 09:41:56 -05:00
Christopher Swenson 5864075c30
Add events sending routed from plugins (#18834)
This isn't perfect for sure, but it's solidifying and becoming a useful
base to work off.

This routes events sent from auth and secrets plugins to the main
`EventBus` in the Vault Core. Events sent from plugins are automatically
tagged with the namespace and plugin information associated with them.
2023-02-03 13:24:16 -08:00
Anton Averchenkov 7b356f0366
openapi: Add default values to the thing_mount_path parameters (#18935) 2023-01-31 19:37:16 -05:00
Max Bowsher 9d863a92ce
Fix multiple OpenAPI generation issues with new AST-based generator (#18554)
* Regexp metacharacter `.` should be escaped when used literally

The paths including `/.well-known/` in the Vault API could currently
technically be invoked with any random character in place of the dot.

* Replace implementation of OpenAPI path translator with regexp AST-based one

* Add changelog

* Typo fix from PR review - thanks!

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Add comment based on review feedback

* Change style of error handling as suggested in code review

* Make a further tweak to the handling of the error case

* Add more tests, testing cases which fail with the previous implementation

* Resolve issue with a test, and improve comment

---------

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-01-31 16:27:39 -05:00
Steven Clark b737777f15
Address pki::TestAutoRebuild flakiness (#18903)
* Address pki::TestAutoRebuild flakiness

 - Wait for a CRL change before progressing to the next step after
   we change configuration. Prior to this we would be racing against
   the CRL reloading from the configuration change.
2023-01-30 21:38:38 +00:00
Matt Schultz 20c1645329
Add some managed key build tags to support incoming enterprise feature work. (#18902) 2023-01-30 14:57:01 -06:00
Rachel Culpepper c151d5fd58
Vault-11713: Add the ability for transit to use managed keys (#18844)
* add the ability for transit to use managed keys

* add fields for rotate

* reorganize managed keys for better error messaging

* add aad for managed keys

* change encrypting key interface

* fix import error

* fix fmt errors

* move managed key rotation to a separate function

* fix fmt issue

* fix import error

* fix go.mod issue

* fix go.mod

* remove extra rotate param

* fix locking issue in rotation

* fix errors in policy tests

* move managed key params to fit existing policy functions

* fix sign and verify

* fix policy tests

* fix test error
2023-01-27 13:39:58 -06:00
Nick Cabatoff 9ca78845b7
Proposed more streamlined approach to validating schema responses. (#18865) 2023-01-27 14:34:55 -05:00
Kit Haines 27be887bfd
Vault 9406 enablement certs need userid handling in role (#18397)
* The fields.

* UserID set, add to certificate

* Changelog.

* Fix test (set default).

* Add UserID constant to certutil, revert extension changes

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add user_ids as field for leaf signing

Presumably, this isn't necessary for CAs, given that CAs probably don't
have a user ID corresponding to them.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Support setting multiple user_ids in Subject

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow any User ID with sign-verbatim

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add tests for User IDs in PKI

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add docs about user_ids, allowed_user_ids

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-25 13:13:54 -05:00
Matt Schultz bf3e266929
Adds managed key usages for MAC generate/verify and RNG. (#18291)
* Adds managed key usages for MAC generate/verify and RNG.

* Remove MAC-related key usages from managed key in favor of sign/verify.

* Remove context from random source managed key interface.
2023-01-24 13:25:09 -06:00
Hamid Ghaf edbf093290
Allow Token Create Requests To Be Replicated (#18689)
* Allow Token Create Requests To Be Replicated

* adding a test

* revert a test
2023-01-24 14:00:27 -05:00
Anton Averchenkov 5a6092f8ab
Add approle's remaining response schema definitions (#18772) 2023-01-24 13:12:41 -05:00
Hamid Ghaf 65a41d4f08
named Login MFA methods (#18610)
* named MFA method configurations

* fix a test

* CL

* fix an issue with same config name different ID and add a test

* feedback

* feedback on test

* consistent use of passcode for all MFA methods (#18611)

* make use of passcode factor consistent for all MFA types

* improved type for MFA factors

* add method name to login CLI

* minor refactoring

* only accept MFA method name with its namespace path in the login request MFA header

* fix a bug

* fixing an ErrorOrNil return value

* more informative error message

* Apply suggestions from code review

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* feedback

* test refactor a bit

* adding godoc for a test

* feedback

* remove sanitize method name

* guard a possbile nil ref

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-23 15:51:22 -05:00
Alexander Scheel 2f5c7458b2
Update x/crypto + x/net (#18794)
* Update golang.org/x/crypto version

go get -u golang.org/x/crypto && go mod tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update golang.org/x/crypto version in api

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Update golang.org/x/crypto version in sdk

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-23 19:11:04 +00:00
Alexander Scheel 6930568076
Add path based primary write forwarding (PBPWF) - OSS (#18735)
* Add WriteForwardedStorage to sdk's plugin, logical in OSS

This should allow backends to specify paths to forward write
(storage.Put(...) and storage.Delete(...)) operations for.

Notably, these semantics are subject to change and shouldn't yet be
relied on.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Collect paths for write forwarding in OSS

This adds a path manager to Core, allowing tracking across all Vault
versions of paths which could use write forwarding if available. In
particular, even on OSS offerings, we'll need to template {{clusterId}}
into the paths, in the event of later upgrading to Enterprise. If we
didn't, we'd end up writing paths which will no longer be accessible
post-migration, due to write forwarding now replacing the sentinel with
the actual cluster identifier.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add forwarded writer implementation to OSS

Here, for paths given to us, we determine if we need to do cluster
translation and perform local writing. This is the OSS variant.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Wire up mount-specific request forwarding in OSS

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clarify that state lock needs to be held to call HAState in OSS

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Move cluster sentinel constant to sdk/logical

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Expose ClusterID to Plugins via SystemView

This will let plugins learn what the Cluster's ID is, without having to
resort to hacks like writing a random string to its cluster-prefixed
namespace and then reading it once it has replicated.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add GRPC ClusterID implementation

For any external plugins which wish to use it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-20 16:36:18 -05:00
Christopher Swenson 4a93097895
Use schema for events in event broker (#18693)
For the new events schema. Based on the CloudEvents schema.
2023-01-20 10:18:23 -08:00
Max Bowsher 4c5f583f39
OpenAPI generic_mount_paths follow-up (#18663)
* OpenAPI `generic_mount_paths` follow-up

An incremental improvement within larger context discussed in #18560.

* Following the revert in #18617, re-introduce the change from
  `{mountPath}` to `{<path-of-mount>_mount_path}`; this is needed, as
  otherwise paths from multiple plugins would clash - e.g. almost every
  auth method would provide a conflicting definition for
  `auth/{mountPath}/login`, and the last one written into the map would
  win.

* Move the half of the functionality that was in `sdk/framework/` to
  `vault/logical_system.go` with the rest; this is needed, as
  `sdk/framework/` gets compiled in to externally built plugins, and
  therefore there may be version skew between it and the Vault main
  code. Implementing the `generic_mount_paths` feature entirely on one
  side of this boundary frees us from problems caused by this.

* Update the special exception that recognizes `system` and `identity`
  as singleton mounts to also include the other two singleton mounts,
  `cubbyhole` and `auth/token`.

* Include a comment that documents to restricted circumstances in which
  the `generic_mount_paths` option makes sense to use:

	    // Note that for this to actually be useful, you have to be using it with
	    // a Vault instance in which you have mounted one of each secrets engine
	    // and auth method of types you are interested in, at paths which identify
	    // their type, and for the KV secrets engine you will probably want to
	    // mount separate kv-v1 and kv-v2 mounts to include the documentation for
	    // each of those APIs.

* Fix tests

Also remove comment "// TODO update after kv repo update" which was
added 4 years ago in #5687 - the implied update has not happened.

* Add changelog

* Update 18663.txt
2023-01-17 23:07:11 -05:00
Christopher Swenson b95beeb675
Add basic event bus broker stub (#18640)
Creates a new `eventbus` package under `vault` with
an implementation of the `go-eventlogger` broker.

Also creates a stub of a common broker that will be accessible
in the core, and creates a simple event sending interface.
2023-01-17 13:34:37 -08:00
Max Bowsher 91855bce22
Fix a very old comment to include query-string parameters (#18557)
Vault has gradually had the ability to pass query-string parameters
added to GET, then DELETE, and now recently LIST requests. Update
a comment which seems to date back to when no query-string parameters
were used at all.
2023-01-17 12:29:30 -05:00
Anton Averchenkov 6ae09f3074
Add AppRole response schema validation tests (#18636)
This PR modifies every test in `builtin/credentials/approle/path_role_test.go` with new validation checks to ensure that approle/path_role  successful responses align with the declared response schema.

It also introduces a test helper in `sdk/helper/testhelpers`:

```go
func FindResponseSchema(t *testing.T, ...)
```

This test helper will be useful for all plugins that require similar response schema validation in tests.

### Background

This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
2023-01-13 15:23:36 -05:00
Anton Averchenkov 9696600e59
Add response schema validation methods & test helpers (#18635)
This pull request adds 3 functions (and corresponding tests):

`testhelpers/response_validation.go`:

  - `ValidateResponse`
  - `ValidateResponseData`
  
field_data.go:

  - `ValidateStrict` (has the "strict" validation logic)

The functions are primarily meant to be used in tests to ensure that the responses are consistent with the defined response schema. An example of how the functions can be used in tests can be found in #18636.

### Background

This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
2023-01-13 14:55:56 -05:00
Anton Averchenkov c7f40361eb
Revert "Add mount path into the default generated openapi.json spec (#17926)" (#18617)
* Revert "Add mount path into the default generated openapi.json spec (UI) (#17926)"

This reverts commit db8efac708e5385ec871be9558507eeaf54ac972.

* Revert "Remove `generic_mount_paths` field (#18558)"

This reverts commit 79c8f626c59ca11bb8e7f460d40b09f5e0cec76d.
2023-01-10 11:16:59 -05:00
Mike Palmiotto cb3406b1eb
plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Matt Schultz dcfb58e18c
Add ManagedKeyRandomSource interface to SDK. (#18269) 2022-12-08 14:17:41 -06:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Anton Averchenkov 493040d147
Add mount path into the default generated openapi.json spec (UI) (#17926) 2022-12-08 12:15:54 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00