The [WebSockets spec](https://www.rfc-editor.org/rfc/rfc6455) states
that text messages must be valid UTF-8 encoded strings, which protobuf
messages virtually never are. This now correctly sends the protobuf events
as binary messages.
We change the format to correspond to CloudEvents, as originally intended,
and remove a redundant timestamp and newline.
We also bump the eventlogger to fix a race condition that this code triggers.
* add RequestResponseCallback to core/options
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* pass in router and apply function on requests
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* add callback
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* cleanup
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Update vault/core.go
* bad typo...
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* use pvt interface, can't downcast to child struct
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* finer grained errors
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* trim path for backend
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* remove entire mount point instead of just the first part of url
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* Update vault/testing.go
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* add doc string
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* update docstring
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* reformat
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
* added changelog
---------
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.
This can be used by either `wscat` or the new
`vault events subscribe`:
e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
and
```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```
Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
This isn't perfect for sure, but it's solidifying and becoming a useful
base to work off.
This routes events sent from auth and secrets plugins to the main
`EventBus` in the Vault Core. Events sent from plugins are automatically
tagged with the namespace and plugin information associated with them.
* Regexp metacharacter `.` should be escaped when used literally
The paths including `/.well-known/` in the Vault API could currently
technically be invoked with any random character in place of the dot.
* Replace implementation of OpenAPI path translator with regexp AST-based one
* Add changelog
* Typo fix from PR review - thanks!
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Add comment based on review feedback
* Change style of error handling as suggested in code review
* Make a further tweak to the handling of the error case
* Add more tests, testing cases which fail with the previous implementation
* Resolve issue with a test, and improve comment
---------
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
* Address pki::TestAutoRebuild flakiness
- Wait for a CRL change before progressing to the next step after
we change configuration. Prior to this we would be racing against
the CRL reloading from the configuration change.
* The fields.
* UserID set, add to certificate
* Changelog.
* Fix test (set default).
* Add UserID constant to certutil, revert extension changes
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add user_ids as field for leaf signing
Presumably, this isn't necessary for CAs, given that CAs probably don't
have a user ID corresponding to them.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Support setting multiple user_ids in Subject
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Allow any User ID with sign-verbatim
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for User IDs in PKI
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add docs about user_ids, allowed_user_ids
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Adds managed key usages for MAC generate/verify and RNG.
* Remove MAC-related key usages from managed key in favor of sign/verify.
* Remove context from random source managed key interface.
* named MFA method configurations
* fix a test
* CL
* fix an issue with same config name different ID and add a test
* feedback
* feedback on test
* consistent use of passcode for all MFA methods (#18611)
* make use of passcode factor consistent for all MFA types
* improved type for MFA factors
* add method name to login CLI
* minor refactoring
* only accept MFA method name with its namespace path in the login request MFA header
* fix a bug
* fixing an ErrorOrNil return value
* more informative error message
* Apply suggestions from code review
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* feedback
* test refactor a bit
* adding godoc for a test
* feedback
* remove sanitize method name
* guard a possbile nil ref
Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
* Update golang.org/x/crypto version
go get -u golang.org/x/crypto && go mod tidy
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update golang.org/x/crypto version in api
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Update golang.org/x/crypto version in sdk
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add WriteForwardedStorage to sdk's plugin, logical in OSS
This should allow backends to specify paths to forward write
(storage.Put(...) and storage.Delete(...)) operations for.
Notably, these semantics are subject to change and shouldn't yet be
relied on.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Collect paths for write forwarding in OSS
This adds a path manager to Core, allowing tracking across all Vault
versions of paths which could use write forwarding if available. In
particular, even on OSS offerings, we'll need to template {{clusterId}}
into the paths, in the event of later upgrading to Enterprise. If we
didn't, we'd end up writing paths which will no longer be accessible
post-migration, due to write forwarding now replacing the sentinel with
the actual cluster identifier.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add forwarded writer implementation to OSS
Here, for paths given to us, we determine if we need to do cluster
translation and perform local writing. This is the OSS variant.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Wire up mount-specific request forwarding in OSS
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Clarify that state lock needs to be held to call HAState in OSS
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Move cluster sentinel constant to sdk/logical
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Expose ClusterID to Plugins via SystemView
This will let plugins learn what the Cluster's ID is, without having to
resort to hacks like writing a random string to its cluster-prefixed
namespace and then reading it once it has replicated.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add GRPC ClusterID implementation
For any external plugins which wish to use it.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* OpenAPI `generic_mount_paths` follow-up
An incremental improvement within larger context discussed in #18560.
* Following the revert in #18617, re-introduce the change from
`{mountPath}` to `{<path-of-mount>_mount_path}`; this is needed, as
otherwise paths from multiple plugins would clash - e.g. almost every
auth method would provide a conflicting definition for
`auth/{mountPath}/login`, and the last one written into the map would
win.
* Move the half of the functionality that was in `sdk/framework/` to
`vault/logical_system.go` with the rest; this is needed, as
`sdk/framework/` gets compiled in to externally built plugins, and
therefore there may be version skew between it and the Vault main
code. Implementing the `generic_mount_paths` feature entirely on one
side of this boundary frees us from problems caused by this.
* Update the special exception that recognizes `system` and `identity`
as singleton mounts to also include the other two singleton mounts,
`cubbyhole` and `auth/token`.
* Include a comment that documents to restricted circumstances in which
the `generic_mount_paths` option makes sense to use:
// Note that for this to actually be useful, you have to be using it with
// a Vault instance in which you have mounted one of each secrets engine
// and auth method of types you are interested in, at paths which identify
// their type, and for the KV secrets engine you will probably want to
// mount separate kv-v1 and kv-v2 mounts to include the documentation for
// each of those APIs.
* Fix tests
Also remove comment "// TODO update after kv repo update" which was
added 4 years ago in #5687 - the implied update has not happened.
* Add changelog
* Update 18663.txt
Creates a new `eventbus` package under `vault` with
an implementation of the `go-eventlogger` broker.
Also creates a stub of a common broker that will be accessible
in the core, and creates a simple event sending interface.
Vault has gradually had the ability to pass query-string parameters
added to GET, then DELETE, and now recently LIST requests. Update
a comment which seems to date back to when no query-string parameters
were used at all.
This PR modifies every test in `builtin/credentials/approle/path_role_test.go` with new validation checks to ensure that approle/path_role successful responses align with the declared response schema.
It also introduces a test helper in `sdk/helper/testhelpers`:
```go
func FindResponseSchema(t *testing.T, ...)
```
This test helper will be useful for all plugins that require similar response schema validation in tests.
### Background
This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
This pull request adds 3 functions (and corresponding tests):
`testhelpers/response_validation.go`:
- `ValidateResponse`
- `ValidateResponseData`
field_data.go:
- `ValidateStrict` (has the "strict" validation logic)
The functions are primarily meant to be used in tests to ensure that the responses are consistent with the defined response schema. An example of how the functions can be used in tests can be found in #18636.
### Background
This PR is part of the ongoing work to add structured responses in Vault OpenAPI (VLT-234)
* Revert "Add mount path into the default generated openapi.json spec (UI) (#17926)"
This reverts commit db8efac708e5385ec871be9558507eeaf54ac972.
* Revert "Remove `generic_mount_paths` field (#18558)"
This reverts commit 79c8f626c59ca11bb8e7f460d40b09f5e0cec76d.
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
* add Link config, init, and capabilities
* add node status proto
* bump protoc version to 3.21.9
* make proto
* adding link tests
* remove wrapped link
* add changelog entry
* update changelog entry
Move version out of SDK. For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString. That will have to wait until we've removed uses of useragent.String from all builtins.
And return an error instead of panicking.
This situation can occur if a plugin attempts to access the system
view during setup when Vault is checking the plugin metadata.
Fixes#17878.
* wip
* Add cached OCSP client support to Cert Auth
* ->pointer
* Code cleanup
* Fix unit tests
* Use an LRU cache, and only persist up to 1000 of the most recently used values to stay under the storage entry limit
* Fix caching, add fail open mode parameter to cert auth roles
* reduce logging
* Add the retry client and GET then POST logic
* Drop persisted cache, make cache size configurable, allow for parallel testing of multiple servers
* dead code
* Update builtin/credential/cert/path_certs.go
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Hook invalidate to reinit the ocsp cache size
* locking
* Conditionally init the ocsp client
* Remove cache size config from cert configs, it's a backend global
* Add field
* Remove strangely complex validity logic
* Address more feedback
* Rework error returning logic
* More edge cases
* MORE edge cases
* Add a test matrix with a builtin responder
* changelog
* Use an atomic for configUpdated
* Actually use ocsp_enabled, and bind to a random port for testing
* Update builtin/credential/cert/path_login.go
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Refactor unit tests
* Add status to cache
* Make some functions private
* Rename for testing, and attribute
* Up to date gofumpt
* remove hash from key, and disable the vault dependent unit test
* Comment out TestMultiOCSP
* imports
* more imports
* Address semgrep results
* Attempt to pass some sort of logging to test_responder
* fix overzealous search&replace
Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>
The current behaviour is to only add mount paths into the generated `opeanpi.json` spec if a `generic_mount_paths` flag is added to the request. This means that we would have to maintain two different `openapi.json` files, which is not ideal. The new solution in this PR is to add `{mount_path}` into every path with a default value specified:
```diff
-- "/auth/token/accessors/": {
++ "/auth/{mount_path}/accessors/": {
"parameters": [
{
"name": "mount_path",
"description": "....",
"in": "path",
"schema": {
"type": "string",
++ "default": "token"
}
}
],
```
Additionally, fixed the logic to generate the `operationId` (used to generate method names in the code generated from OpenAPI spec). It had a bug where the ID had `mountPath` in it. The new ID will look like this:
```diff
-- "operationId": "listAuthMountpathAccessors",
++ "operationId": "listTokenAccessors",
```
* Add support for PKCSv1_5_NoOID signatures
This assumes a pre-hashed input has been provided to Vault, but we do
not write the hash's OID into the signature stream. This allows us to
generate the alternative PKCSv1_5_NoOID signature type rather than the
existing PKCSv1_5_DERnull signature type we presently use.
These are specified in RFC 3447 Section 9.2.
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add changelog
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Exclude new none type from PSS based tests
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
* Add tests for PKCS#1v1.5 signatures
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
Tom Proctor