Commit graph

2851 commits

Author SHA1 Message Date
Christopher Swenson 020a3903ff
events: Remove subscriptions on timeout and cancel (#19185)
When subscriptions are too slow to accept messages on their channels,
then we should remove them from the fanout so that we don't have
dead subscriptions using up resources.

In addition, when a subscription is explicitly canceled, we should
also clean up after it remove the corresponding pipeline.

We also add a new metrics, `events.subscriptions`, to keep track
of the number of active subscriptions. This also helps us test.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-15 11:13:15 -08:00
Christopher Swenson 7a977fd6ea
events: Check token and ACLs on request (#19138)
This checks the request against the `read` permission for
`sys/events/subscribe/{eventType}` on the initial subscribe.

Future work includes moving this to its own verb (`subscribe`)
and periodically rechecking the request.

Tested locally by minting a token with the wrong permissions
and verifying that they are rejected as expected, and that
they work if the policy is adjusted to `sys/event/subscribe/*`
(or the specific topic name) with `read` permissions.

I had to change the `core.checkToken()` to be publicly accessible,
as it seems like the easiest way to check the token on the
`logical.Request` against all relevant policies, but without
going into all of the complex logic further in `handleLogical()`.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-10 20:56:00 +00:00
Christopher Swenson 7d3d404ee2
events: Add websockets and command (#19057)
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.

This can be used by either `wscat` or the new
`vault events subscribe`:

e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

and

```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-09 13:18:58 -08:00
Tom Proctor 78d83c9136
Make experiments API authenticated (#18966) 2023-02-09 20:18:14 +00:00
John-Michael Faircloth 34fd57ac08
test/plugin: test external plugin workflows (#19090)
* test/plugin: test external plugin workflows

* update secrets engine test
2023-02-09 10:16:16 -06:00
Nick Cabatoff ec2af04ec6
Remove the last vestiges of sdk/version. (#19068) 2023-02-08 12:30:27 -05:00
Scott Miller acee981753
Remove accidental addition of a hackweek file (#19016) 2023-02-06 16:45:55 -06:00
Scott Miller b43e4fbd9c
Add a stronger warning about the usage of recovery keys (#19011)
* Add a stronger warning about the usage of recovery keys

* Update website/content/docs/concepts/seal.mdx

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Keep the mitigation text in the warning box

---------

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-02-06 16:23:05 -06:00
miagilepner e873932bce
VAULT-8436 remove <-time.After statements in for loops (#18818)
* replace time.After with ticker in loops

* add semgrep rule

* update to use timers

* remove stop
2023-02-06 17:49:01 +01:00
miagilepner 9d09dba7ac
VAULT-13061: Fix mount path discrepancy in activity log (#18916)
* use single function to convert mount accessor to mount path

* add changelog

* more context and comments for the tests
2023-02-06 10:26:32 +01:00
John-Michael Faircloth 14e4d67026
test/plugin: refactor compilePlugin for reuse (#18952)
* test/plugin: refactor compilePlugin for reuse

- move compilePlugin to helper package
- make NewTestCluster use compilePlugin

* do not overwrite plugin directory in CoreConfig if set

* fix getting plugin directory path for go build
2023-02-03 16:27:11 -06:00
Christopher Swenson 5864075c30
Add events sending routed from plugins (#18834)
This isn't perfect for sure, but it's solidifying and becoming a useful
base to work off.

This routes events sent from auth and secrets plugins to the main
`EventBus` in the Vault Core. Events sent from plugins are automatically
tagged with the namespace and plugin information associated with them.
2023-02-03 13:24:16 -08:00
Chris Capurso c74c057bdb
fix sys/leases panic when lease_id is nil (#18951)
* fix sys/leases panic when lease_id is nil

* add changelog entry
2023-02-03 09:51:10 -05:00
Anton Averchenkov 619f5b2edf
openapi: Ensure mount_path parameters are marked as required (#18955) 2023-02-02 14:39:09 -05:00
Chris Capurso 53390eaddf
Add ClusteName to GetClusterStatus response (#18950)
* bump github.com/hashicorp/vault/vault/hcp_link/proto

* add ClusterName to GetClusterStatus response
2023-02-02 09:27:55 -05:00
Chris Capurso c8660ca2ea
add ClusterName to meta GetClusterStatusResponse (#18944)
* add ClusterName to meta GetClusterStatusResponse

* make proto
2023-02-01 15:15:04 -05:00
Scott Miller 20551261bd
Revert #18683 (#18942)
* Revert "Don't execute the seal recovery tests on ENT. (#18841)"

This reverts commit 990d3bacc203c229d0f6729929d7562e678a1ac2.

* Revert "Add the ability to unseal using recovery keys via an explicit seal option. (#18683)"

This reverts commit 2ffe49aab0fc1a527c5182637c8fa3ac39b08d45.
2023-02-01 13:34:53 -06:00
Hamid Ghaf 6672d3753f
check for nil references (#18871)
* check for nil references

* feedback

* feedback

* address vault-issue-18849

* Revert "address vault-issue-18849"

This reverts commit 081610c02398cf7ecdfb726cd3f622f3e69eafc4.
2023-02-01 09:51:52 -05:00
Nick Cabatoff c2b222a11a
Vault test cluster helper refactorings, mostly audit related (#18928)
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package.  Consolidate on a single "noop audit" implementation.
2023-02-01 08:33:16 -05:00
Anton Averchenkov 7b356f0366
openapi: Add default values to the thing_mount_path parameters (#18935) 2023-01-31 19:37:16 -05:00
Max Bowsher 9d863a92ce
Fix multiple OpenAPI generation issues with new AST-based generator (#18554)
* Regexp metacharacter `.` should be escaped when used literally

The paths including `/.well-known/` in the Vault API could currently
technically be invoked with any random character in place of the dot.

* Replace implementation of OpenAPI path translator with regexp AST-based one

* Add changelog

* Typo fix from PR review - thanks!

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Add comment based on review feedback

* Change style of error handling as suggested in code review

* Make a further tweak to the handling of the error case

* Add more tests, testing cases which fail with the previous implementation

* Resolve issue with a test, and improve comment

---------

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-01-31 16:27:39 -05:00
Hamid Ghaf 9c8fcaf5a5
prevent panic on mfa enforcement delete after a namespace is deleted (#18923)
* prevent panic on mfa enforcement delete after a namespace is deleted

* CL
2023-01-31 21:06:16 +00:00
Chris Capurso 6cb6157d37
return 403 for wrapping requests when no token provided (#18859)
* return 403 for wrapping requests when no token provided

* add changelog entry

* fix changelog

* use errors.As

* simplify error response string
2023-01-31 13:57:50 -05:00
Nick Cabatoff db41c5319d
TestClusterCore's TLSConfig becomes a method and does a Clone. (#18914) 2023-01-31 11:05:16 -05:00
akshya96 16ce923ddb
Brute forcing unlock user bug (#18890)
* brute forcing unlock user bug

* add changelog

* fix changelog
2023-01-30 13:06:10 -08:00
Kit Haines 5ece71109a
Vault 11798 vault cli issue intermediate (#18467)
* The verify-sign command in it's cleanest existing form.

* Working state

* Updates to proper verification syntax

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>

* make fmt

* Git CI caught some stuff.

* Base functionality.

* make fmt; changelog

* pki issue command.

* Make fmt. Changelog.

* Error Handling Is Almost A Tutorial

* What I thought empty issuers response fix would be.

* Some tests

* PR-review updates.

* make fmt.

* Fix null response data for listing empty issuers causing a crash.

* Update command/pki_list_children_command.go

Fix double specifier

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Add test for pki_list_children.

* Fix tests.

* Update descriptions for correctness based on PR reviews.

* make fmt.

* Updates based on PR feedback.

* Allow multiple arguements (space separated)

* Remove bad merge-thing.

* White-space hell fix change.

* Tests, and return information for issue ca

* Fix make fmt error introduced here: https://github.com/hashicorp/vault/pull/18876

* Update command/pki_issue_intermediate.go

Puncutation.

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove smart quotes for standard quotes.

* More information as part of the help text.

* Better help text.

* Add missing "/" into error message.

---------

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-01-27 16:41:16 -05:00
Jason O'Donnell e14f0216a0
Add timeout functionality to inmem (#18876)
* Add timeout functionality to inmem

* Update vault/cluster/inmem_layer.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Add comment about forceTimeout

* Add comment about time

---------

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-27 16:46:12 +00:00
Scott Miller af7df08921
Don't execute the seal recovery tests on ENT. (#18841)
* Don't execute the seal recovery tests on OSS.

* correct go:build syntax
2023-01-25 15:35:15 -06:00
Hamid Ghaf 2b5e5121e1
Provide IP Address in Duo Request (#18811)
* Provide IP Address in Duo Request

* CL
2023-01-24 17:28:59 -05:00
Scott Miller 25960fd034
Add the ability to unseal using recovery keys via an explicit seal option. (#18683)
* wip

* wip

* Got it 'working', but not happy about cleanliness yet

* Switch to a dedicated defaultSeal with recovery keys

This is simpler than trying to hijack SealAccess as before.  Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal.  Then everything proceeds
as if you had a shamir seal to begin with.

* Handle recovery rekeying

* changelog

* Revert go.mod redirect

* revert multi-blob info

* Dumb nil unmarshal target

* More comments

* Update vault/seal.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* pr feedback

* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split

* Better comment on recovery seal during adjustSealMigration

* Make it possible to migrate from an auto-seal in recovery mode to shamir

* Fix sealMigrated to account for a recovery seal

* comments

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Address PR feedback

* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate

* Don't shortcut the reast of seal migration

* get rid of redundant transit server cleanup

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-24 14:57:56 -06:00
Hamid Ghaf edbf093290
Allow Token Create Requests To Be Replicated (#18689)
* Allow Token Create Requests To Be Replicated

* adding a test

* revert a test
2023-01-24 14:00:27 -05:00
Hamid Ghaf 65a41d4f08
named Login MFA methods (#18610)
* named MFA method configurations

* fix a test

* CL

* fix an issue with same config name different ID and add a test

* feedback

* feedback on test

* consistent use of passcode for all MFA methods (#18611)

* make use of passcode factor consistent for all MFA types

* improved type for MFA factors

* add method name to login CLI

* minor refactoring

* only accept MFA method name with its namespace path in the login request MFA header

* fix a bug

* fixing an ErrorOrNil return value

* more informative error message

* Apply suggestions from code review

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* feedback

* test refactor a bit

* adding godoc for a test

* feedback

* remove sanitize method name

* guard a possbile nil ref

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-23 15:51:22 -05:00
Tom Proctor fc378c0908
Event system alpha experiment (#18795) 2023-01-23 19:26:49 +00:00
Chris Capurso 0eedcd979b
Fix link meta panics (#18774)
* return error for meta auth and mount listing if sealed

* some logging changes

* some more logging changes

* add panic recovery

* use ErrInternalError
2023-01-23 09:59:15 -05:00
Alexander Scheel b3dc380c82
Add cross-cluster revocation queues for PKI (#18784)
* Add global, cross-cluster revocation queue to PKI

This adds a global, cross-cluster replicated revocation queue, allowing
operators to revoke certificates by serial number across any cluster. We
don't support revoking with private key (PoP) in the initial
implementation.

In particular, building on the PBPWF work, we add a special storage
location for handling non-local revocations which gets replicated up to
the active, primary cluster node and back down to all secondary PR
clusters. These then check the pending revocation entry and revoke the
serial locally if it exists, writing a cross-cluster confirmation entry.

Listing capabilities are present under pki/certs/revocation-queue,
allowing operators to see which certs are present. However, a future
improvement to the tidy subsystem will allow automatic cleanup of stale
entries.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow tidying revocation queue entries

No manual operator control of revocation queue entries are allowed.
However, entries are stored with their request time, allowing tidy to,
after a suitable safety buffer, remove these unconfirmed and presumably
invalid requests.

Notably, when a cluster goes offline, it will be unable to process
cross-cluster revocations for certificates it holds. If tidy runs,
potentially valid revocations may be removed. However, it is up to the
administrator to ensure the tidy window is sufficiently long that any
required maintenance is done (or, prior to maintenance when an issue is
first noticed, tidy is temporarily disabled).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Only allow enabling global revocation queue on Vault Enterprise

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use a locking queue to handle revocation requests

This queue attempts to guarantee that PKI's invalidateFunc won't have
to wait long to execute: by locking only around access to the queue
proper, and internally using a list, we minimize the time spent locked,
waiting for queue accesses.

Previously, we held a lock during tidy and processing that would've
prevented us from processing invalidateFunc calls.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* use_global_queue->cross_cluster_revocation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Grab revocation storage lock when processing queue

We need to grab the storage lock as we'll actively be revoking new
certificates in the revocation queue. This ensures nobody else is
competing for storage access, across periodic funcs, new revocations,
and tidy operations.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix expected tidy status test

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow probing RollbackManager directly in tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address review feedback on revocationQueue

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add more cancel checks, fix starting manual tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-23 09:29:27 -05:00
Alexander Scheel 6930568076
Add path based primary write forwarding (PBPWF) - OSS (#18735)
* Add WriteForwardedStorage to sdk's plugin, logical in OSS

This should allow backends to specify paths to forward write
(storage.Put(...) and storage.Delete(...)) operations for.

Notably, these semantics are subject to change and shouldn't yet be
relied on.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Collect paths for write forwarding in OSS

This adds a path manager to Core, allowing tracking across all Vault
versions of paths which could use write forwarding if available. In
particular, even on OSS offerings, we'll need to template {{clusterId}}
into the paths, in the event of later upgrading to Enterprise. If we
didn't, we'd end up writing paths which will no longer be accessible
post-migration, due to write forwarding now replacing the sentinel with
the actual cluster identifier.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add forwarded writer implementation to OSS

Here, for paths given to us, we determine if we need to do cluster
translation and perform local writing. This is the OSS variant.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Wire up mount-specific request forwarding in OSS

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clarify that state lock needs to be held to call HAState in OSS

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Move cluster sentinel constant to sdk/logical

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Expose ClusterID to Plugins via SystemView

This will let plugins learn what the Cluster's ID is, without having to
resort to hacks like writing a random string to its cluster-prefixed
namespace and then reading it once it has replicated.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add GRPC ClusterID implementation

For any external plugins which wish to use it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-20 16:36:18 -05:00
Christopher Swenson 4a93097895
Use schema for events in event broker (#18693)
For the new events schema. Based on the CloudEvents schema.
2023-01-20 10:18:23 -08:00
Daniel Huckins fc6d13e29d
VAULT-12112: openapi response definitions: sys/audit (#18456)
* added audit-hash operations

* more audit paths

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added audit fields

* add changelog file

* dynamic fields should be nil

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* start to add test helper

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add tests for /sys/audit openapi paths

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <anton.averchenkov@hashicorp.com>
2023-01-20 11:09:33 -05:00
Josh Black fa1447cb3c
Add new clients into the monthly breakdown (#18766)
* Add new clients into the monthly breakdown

* add changelog
2023-01-19 09:12:17 -08:00
Christopher Swenson fcbce0effd
Start events when core starts if enabled (#18742)
For example, using:

```sh
vault server -dev -experiment events.beta1
```

Tested by checking that the events were enabled and disabled
when the `-experiment events.beta1` flag was present and absent.

Also added a small fix to pass the `hclog.Logger` in now so that
the logging hierarchy and levels are respected.
2023-01-18 10:46:01 -08:00
Max Bowsher 4c5f583f39
OpenAPI generic_mount_paths follow-up (#18663)
* OpenAPI `generic_mount_paths` follow-up

An incremental improvement within larger context discussed in #18560.

* Following the revert in #18617, re-introduce the change from
  `{mountPath}` to `{<path-of-mount>_mount_path}`; this is needed, as
  otherwise paths from multiple plugins would clash - e.g. almost every
  auth method would provide a conflicting definition for
  `auth/{mountPath}/login`, and the last one written into the map would
  win.

* Move the half of the functionality that was in `sdk/framework/` to
  `vault/logical_system.go` with the rest; this is needed, as
  `sdk/framework/` gets compiled in to externally built plugins, and
  therefore there may be version skew between it and the Vault main
  code. Implementing the `generic_mount_paths` feature entirely on one
  side of this boundary frees us from problems caused by this.

* Update the special exception that recognizes `system` and `identity`
  as singleton mounts to also include the other two singleton mounts,
  `cubbyhole` and `auth/token`.

* Include a comment that documents to restricted circumstances in which
  the `generic_mount_paths` option makes sense to use:

	    // Note that for this to actually be useful, you have to be using it with
	    // a Vault instance in which you have mounted one of each secrets engine
	    // and auth method of types you are interested in, at paths which identify
	    // their type, and for the KV secrets engine you will probably want to
	    // mount separate kv-v1 and kv-v2 mounts to include the documentation for
	    // each of those APIs.

* Fix tests

Also remove comment "// TODO update after kv repo update" which was
added 4 years ago in #5687 - the implied update has not happened.

* Add changelog

* Update 18663.txt
2023-01-17 23:07:11 -05:00
akshya96 6e04e4ede1
Prevent brute forcing : telemetry oss changes (#18718)
* Prevent brute forcing : telemetry oss changes

* adding changelog
2023-01-17 15:10:50 -08:00
akshya96 b2276a369a
Prevent Brute Forcing: Create an api endpoint to list locked users OSS changes (#18675)
* api to list lockedusers oss changes

* add changelog
2023-01-17 14:25:56 -08:00
Josh Black c9763996d4
Enable undo logs by default (#18692)
* Enable undo logs by default

* add consul test

* update go.mod/sum

* add a better non-existent key
2023-01-17 13:38:18 -08:00
Christopher Swenson b95beeb675
Add basic event bus broker stub (#18640)
Creates a new `eventbus` package under `vault` with
an implementation of the `go-eventlogger` broker.

Also creates a stub of a common broker that will be accessible
in the core, and creates a simple event sending interface.
2023-01-17 13:34:37 -08:00
Tom Proctor d5c35f39c3
Add experiment system + events experiment (#18682) 2023-01-16 16:07:18 +00:00
Peter Wilson 59450ecb82
Revert "Add new clients into the monthly breakdown (#18629)" (#18726)
This reverts commit d641bbc28e5e8cc12b81d409e5d5fc1f2cb7f66c.
2023-01-16 15:51:19 +00:00
akshya96 78546af8fc
Vault 8308 Background thread to update locked user entries (#18673)
* background thread changes

* adding changelog

* fix changelog typo
2023-01-12 14:09:33 -08:00
Chris Capurso 03b23a7970
VAULT-12548: Fix data races in Link status reporting (#18674)
* pin hcp-scada-provider

* fix link reportStatus data race

* go get hcp-scada-provider@v0.2.1
2023-01-12 15:10:34 -05:00
Ellie 6f7757e949
add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
Max Bowsher 6d6a726f9d
Fix HelpOperation on sudo-protected paths (#18568)
* Fix HelpOperation on sudo-protected paths

Fixes #18566

* Add changelog
2023-01-10 12:17:16 -06:00
Anton Averchenkov c7f40361eb
Revert "Add mount path into the default generated openapi.json spec (#17926)" (#18617)
* Revert "Add mount path into the default generated openapi.json spec (UI) (#17926)"

This reverts commit db8efac708e5385ec871be9558507eeaf54ac972.

* Revert "Remove `generic_mount_paths` field (#18558)"

This reverts commit 79c8f626c59ca11bb8e7f460d40b09f5e0cec76d.
2023-01-10 11:16:59 -05:00
Josh Black d3f822a938
Add new clients into the monthly breakdown (#18629)
* Add new clients into the monthly breakdown

* add changelog
2023-01-09 15:26:11 -08:00
Chris Capurso 25d0afae23
VAULT-11830: Expand NodeStatusReporter with new fields (#18302)
* expand NodeStatusReporter with new fields

* only call IsRaftVoter if using raft storage

* add changelog entry

* fix listeners

* return LogLevel as enum

* update github.com/hashicorp/vault/vault/hcp_link/proto

* add changelog entry

* bump github.com/hashicorp/vault/vault/hcp_link/proto

* go mod tidy
2023-01-06 20:53:09 -05:00
Chris Capurso bb0c92afe7
VAULT-11829: Add cluster status handler (#18351)
* go get link proto @vault-11829-meta-get-cluster-status

* add HA status

* add HAEnabled method

* add raft config

* allocate HA nodes based on actual count

* add raft autopilot status

* add raft quorum warnings

* add ClusterID method

* add StorageType

* add ClusterID

* update github.com/hashicorp/vault/vault/hcp_link/proto

* add changelog entry

* fix raft config panic

* remove "Warning" quorum message prefix

* add error wrapping

* add Core.HAStateWithLock method

* reduce quorum warnings to single string

* fix HCP_API_HOST test env var check

* Revert "fix HCP_API_HOST test env var check"

This reverts commit 97c73c4798b77b84aea84f341f2c63c4d657914d.
2023-01-06 17:06:54 -05:00
Chris Capurso 82eaecd745
change quorum warning to a singular string (#18619) 2023-01-06 15:31:27 -05:00
Chris Capurso de59e29bbc
VAULT-11829: Add GetClusterStatus rpc to meta capability (#18316)
* add GetClusterStatus rpc to meta capability

* consolidate HA-related info

* add storage type
2023-01-05 13:33:08 -05:00
Violet Hynes ae653a05a7
VAULT-12489 OSS Changes (#18607) 2023-01-05 13:00:55 -05:00
Josh Black c8a8c21cee
Account for mount counts when de-duplicating current and historical month data (#18598)
* Account for mount counts when de-duplicating current and historical month data

* add changelog
2023-01-05 09:34:05 -08:00
Chris Capurso e7787a94e0
VAULT-11827: Add new Link node level fields (#18203)
* add addition Link node-level status fields

* pin protoc to 3.21.9

* make proto

* change LogLevel to be a string

* add RaftStatus; IsActive -> Activetime

* use an enum for LogLevel
2023-01-05 11:16:48 -05:00
Max Bowsher 7d87548f4f
Fix duplicate definition of path sys/internal/specs/openapi (#18553)
This was accidentally duplicated in #5687.

Remove the second definition, which was shadowed by the first, and move
the documentation that was part of the second to the surviving version.
2023-01-04 22:48:40 -05:00
Ikko Eltociear Ashimine 6927478ab9
Fix typo in mount.go (#18575)
heirarchy -> hierarchy
2023-01-04 21:52:42 -05:00
vinay-gopalan bbd8ac9bbf
Upgrade go.opentelemetry.io/otel from v0.20.0 to v1.11.2 (#18589) 2023-01-04 11:31:30 -08:00
Chris Capurso 0635d304de
only update SCADA metadata if status changes (#18585)
* only update SCADA metadata if status changes

* add changelog entry
2023-01-04 11:09:51 -05:00
Max Bowsher c4e8e22963
Remove generic_mount_paths field (#18558)
PR #17926 already deleted the implementation of the
`generic_mount_paths` field so it needs to be removed from the declared
fields of the path too, so help and OpenAPI isn't misleading.
2023-01-03 19:14:29 -05:00
Max Bowsher 4052c785ea
Remove unreachable code (#18576)
I happened to spot that the `TemplateError` type is never instantiated.
Therefore delete it, and code referencing it.
2023-01-03 09:02:01 -05:00
akshya96 4126060d88
Prevent Brute Forcing: Create api endpoint to unlock users (#18279)
* code changes for unlock

* add test

* adding sys help

* adding sys help

* updating unlock user function

* edit test

* add changelog

* syshelp

* adding open api response definition

* removing response fields

* change path name
2022-12-19 14:24:42 -08:00
Josh Black cd7d6d5761
De-duplicate namespaces when historical and current month data are mixed (#18452)
* De-duplicate namespaces when historical and current month data are mixed

* add changelog
2022-12-16 16:02:42 -08:00
Scott Miller 53e73ceba2
Use a cleaner worker pattern (#18422) 2022-12-16 11:35:24 -06:00
divyaac f8ad8bc5a5
OSS PR for Config Changes PR (#18418)
* OSS PR for Config Changes PR

* Edited tests

* typo

* Added changelog

* Remove changelog
2022-12-15 12:19:19 -08:00
Nick Cabatoff 429916c135
Prevent panics in expiration invalidation, and make some changes for testing (#18401) 2022-12-15 18:09:36 +00:00
Mike Palmiotto cb3406b1eb
plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Mike Palmiotto 809a04c8b4
core: Make shutdownDoneCh atomic (#18358)
When issuing a core.Shutdown(), it is common to background the shutdown
request. This allows Vault to continue cleaning up, mainly to release
the stateLock. This allows the shutdown to complete, but is inherently
racy, so the core.shutdownDoneCh needs to be made atomic.
2022-12-14 15:59:11 +00:00
Scott Miller 25bff579ea
Use a small pool of workers to run postUnsealFuncs in parallel (#18244)
* Initial worker pool

* Run postUnsealFuncs in parallel

* Use the old logic for P=1

* changelog

* Use a CPU count relative worker pool

* Update vault/core.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Done must be called once per postUnsealFunc

* Defer is overkill

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-12-12 17:07:53 -06:00
Chris Capurso 42f36605c7
use meta and link_control from link proto (#18290) 2022-12-09 12:35:06 -05:00
Chris Capurso c81485d5cd
copy over link_control and meta to link proto module (#18285) 2022-12-09 11:57:35 -05:00
Violet Hynes 176c149a38
VAULT-8336 Fix default rate limit paths (#18273)
* VAULT-8336 Fix default rate limit paths

* VAULT-8336 changelog
2022-12-09 08:49:17 -05:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Anton Averchenkov 493040d147
Add mount path into the default generated openapi.json spec (UI) (#17926) 2022-12-08 12:15:54 -05:00
Nick Cabatoff 1b745aef58
Prevent autopilot from demoting voters when they join a 2nd time (#18263) 2022-12-07 14:17:45 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
akshya96 1801f09c6a
Vault 8307 user lockout workflow oss (#17951)
* adding oss file changes

* check disabled and read values from config

* isUserLocked, getUserLockout Configurations, check user lock before login and return error

* remove stale entry from storage during read

* added failed login process workflow

* success workflow updated

* user lockouts external tests

* changing update to support delete

* provide access to alias look ahead function

* adding path alias lookahead

* adding tests

* added changelog

* added comments

* adding changes from ent branch

* adding lock to UpdateUserFailedLoginInfo

* fix return default bug
2022-12-06 17:22:46 -08:00
Mike Palmiotto ea41e62e83
plugins: Mount missing plugin entries and skip loading (#18189)
* Skip plugin startup for missing plugins
* Skip secrets startup for missing plugins
* Add changelog for bugfix
* Make plugin handling on unseal version-aware
* Update plugin lazy-load logic/comments for readability
* Add register/mount/deregister/seal/unseal go test
* Consolidate lazy mount logic to prevent inconsistencies

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2022-12-02 13:16:31 -05:00
Tom Proctor 05aeab2752
Fix plugin list API when audit logging enabled (#18173)
* Add test that fails due to audit log panic
* Rebuild VersionedPlugin as map of primitive types before adding to response
* Changelog
* Fix casting in external plugin tests
2022-12-01 10:44:44 +00:00
Tom Proctor 48987ce052
Add stack trace to audit logging panic recovery (#18121) 2022-11-30 17:59:05 +00:00
Nick Cabatoff 12e1b609ac
Create global quotas of each type in every NewTestCluster. (#18038)
Create global quotas of each type in every NewTestCluster.  Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.

NewTestCluster also now starts the cluster, and the Start method becomes a no-op.  Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode.  This was needed because otherwise we can't safely make the quota api call.  We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start. 

The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint.  The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http.  The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
2022-11-29 14:38:33 -05:00
Violet Hynes 78efcb7d6a
VAULT-11786 OSS changes for this change (#18140) 2022-11-29 13:22:15 -05:00
nsimons ce90a6fa38
Make the error and http code clearer when supplying wrong unseal key (#17836)
* Fix typos

* Return http 400 when wrong unseal key is supplied

* Add changelog

* Add test cases and change one more return case to http 400

The new case is triggered when key length is within valid range
[16, 32], but it has uneven bytes, causing crypto/aes to return
invalid key size.

* remove expected in unit tests

* include error in the new error reason

* add multikey and autoseal test cases

* return invalid key for few more code paths
2022-11-28 16:01:47 -08:00
Chris Capurso 2843cfcdc1
VAULT-9427: Add read support to sys/loggers endpoints (#17979)
* add logger->log-level str func

* ensure SetLogLevelByName accounts for duplicates

* add read handlers for sys/loggers endpoints

* add changelog entry

* update docs

* ignore base logger

* fix docs formatting issue

* add ReadOperation support to TestSystemBackend_Loggers

* add more robust checks to TestSystemBackend_Loggers

* add more robust checks to TestSystemBackend_LoggersByName

* check for empty name in delete handler
2022-11-28 11:18:36 -05:00
Tom Proctor 853643d02b
Remove pinned builtin plugin versions from storage (#18051)
* Removes _builtin_ versions from mount storage where it already exists
* Stops new builtin versions being put into storage on mount creation/tuning
* Stops the plugin catalog from returning a builtin plugin that has been overridden, so it more accurately reflects the plugins that are available to actually run
2022-11-23 18:36:25 +00:00
Hamid Ghaf 22f51dc6d6
improve kv CLI to remove data or custom metadata using kv patch (#18067)
* improve kv CLI to remove data or custom metadata using kv patch

* CL

* adding a comment
2022-11-21 17:11:36 -05:00
Tom Proctor dc85e37cf4
storage/raft: Add retry_join_as_non_voter config option (#18030) 2022-11-18 17:58:16 +00:00
Violet Hynes bfeae1fe8e
VAULT-7707 OSS portion of changes (#18019)
* VAULT-7707 OSS portion of changes

* Revert "VAULT-7707 OSS portion of changes"

This reverts commit 5b8cf3882fb7e2427593d59e1439d46b3a5c20a7.

* VAULT-7707 smarter locking behaviour

* VAULT-7707 typo

* VAULT-7707 typo
2022-11-17 16:30:39 -05:00
davidadeleon 3394c28ce1
Deduplicate policies prior to generating ACL on request (#17914)
* Deduplicate policies prior to generating ACL on request

* add changelog

* edit changelog entry
2022-11-16 17:43:46 -05:00
Brian Kassouf 288b0567b1
Barrier: Fix potential locking issue (#17944)
* Barrier: Fix potential locking issue

* add changelog
2022-11-16 09:53:22 -08:00
akshya96 f3c9e98fd5
Vault-8306 User Lockout RPCs oss changes (#17765)
* adding oss file changes

* updating changes from ent
2022-11-15 15:07:52 -08:00
Josh Black 94739c1af6
Don't return a 204 if there's no historical data (#17935)
* don't return a 204 if there's no historical data

* add changelog
2022-11-15 12:15:51 -08:00
Mike Palmiotto 773f0d58ad
plugins: Filter builtins by RunningVersion (#17816)
This commit adds some logic to handle the case where a mount entry has a
non-builtin RunningVersion. This ensures that we only report deprecation
status for builtins.
2022-11-11 14:51:37 -05:00
Anton Averchenkov f9fac68980
Revert "Add mount path into the default generated openapi.json spec (#17839)" (#17890)
This reverts commit 02064eccb42bb2ec1a3d12ec0d49c661312acd2d.
2022-11-10 15:39:53 -08:00
Anton Averchenkov f3aea876b9
Add mount path into the default generated openapi.json spec (#17839)
The current behaviour is to only add mount paths into the generated `opeanpi.json` spec if a `generic_mount_paths` flag is added to the request. This means that we would have to maintain two different `openapi.json` files, which is not ideal. The new solution in this PR is to add `{mount_path}` into every path with a default value specified:

```diff
--    "/auth/token/accessors/": {
++    "/auth/{mount_path}/accessors/": {
      "parameters": [
        {
          "name": "mount_path",
          "description": "....",
          "in": "path",
          "schema": {
            "type": "string",
++          "default": "token"
          }
        }
      ],
```

Additionally, fixed the logic to generate the `operationId` (used to generate method names in the code generated from OpenAPI spec). It had a bug where the ID had `mountPath` in it. The new ID will look like this:

```diff
-- "operationId": "listAuthMountpathAccessors",
++ "operationId": "listTokenAccessors",
```
2022-11-10 15:44:43 -05:00
Josh Black a5c101d851
Fix activity log end time (#17856)
* Correct the end_time in the activity log output for partial counts

* use the real endTime not the passed in one

* add changelog
2022-11-10 12:11:23 -08:00
Violet Hynes 65e8eee0ba
VAULT-8703 Add warning for dangerous undocumented overrides, if used, in status response (#17855)
* VAULT-8703 Add warning for dangerous undocumented overrides, if used, in status response

* VAULT-8703 add changelog

* VAULT-8703 fix append
2022-11-09 11:04:36 -05:00
divyaac 2d3775a93b
Introspection API Implementation for Router Struct (#17789)
* OSS Commit from ENT for Introspection API

* Add changelog
2022-11-04 09:39:09 -07:00
Jason O'Donnell 4e122214f7
core: fix start up policy loading race condition on perf standbys (#17801)
* core: fix start up policy loading race condition on perf standbys

* Use correct bool for perf standby

* changelog
2022-11-03 13:01:39 -04:00
akshya96 2945924b2b
Vault 8305 Prevent Brute Forcing in Auth methods : Setting user lockout configuration (#17338)
* config file changes

* lockout config changes

* auth tune r/w and auth tune

* removing changes at enable

* removing q.Q

* go mod tidy

* removing comments

* changing struct name for config file

* fixing mount tune

* adding test file for user lockout

* fixing comments and add changelog

* addressing comments

* fixing mount table updates

* updating consts in auth_tune

* small fixes

* adding hcl parse test

* fixing config compare

* fixing github comments

* optimize userlockouts.go

* fixing test

* minor changes

* adding comments

* adding sort to flaky test

* fix flaky test
2022-11-01 11:02:07 -07:00
Mike Palmiotto a9dcc45f72
Tweak totp test to fix race failures (#17692) 2022-10-27 09:41:40 -04:00
akshya96 1e189016e2
update protoc version to 3.21.7 oss (#17499)
* update protoc to 3.21.7

* adding changelog
2022-10-26 16:49:44 -07:00
Mike Palmiotto cc96c6f470
Store login MFA secret with tokenhelper (#17040)
* Store login MFA secret with tokenhelper
* Clean up and refactor tokenhelper paths
* Refactor totp test code for re-use
* Add login MFA command tests
* Use longer sleep times and sha512 for totp test
* Add changelog
2022-10-26 17:02:26 -04:00
Violet Hynes 73f9b13762
VAULT-9451 Fix data race in entity merge (#17631) 2022-10-21 16:47:59 -04:00
Violet Hynes 5861c51e70
VAULT-8719 Support data array for alias clash error response so UI/machines can understand error (#17459)
* VAULT-8719 Support data array for alias clash error response so UI can understand error

* VAULT-8719 Changelog

* VAULT-8719 Update alias mount update logic

* VAULT-8719 Further restrict IsError()
2022-10-17 14:46:25 -04:00
Nick Cabatoff f94bd10540
Tolerate NamespaceByID returning (nil,nil) when looking up an mfa enforcement's ns (#17562) 2022-10-17 09:18:02 -04:00
Hamid Ghaf 8a624c1264
prevent memory leak when using control group factors in a policy (#17532)
* prevent a possible memory leak when using control group factors in a policy

* CL
2022-10-14 19:15:15 -04:00
Mike Palmiotto 81459de6fd
core: Move rollback period init to NewCore (#17547) 2022-10-13 18:39:00 -04:00
Nick Cabatoff 8e67651dcd
Fix a data race with rollbackPeriod. (#17387) 2022-10-13 09:59:07 -04:00
Chris Capurso 2c9b29ab42
fix off by one err in current month client count computation (#17457) 2022-10-07 12:37:09 -04:00
Nick Cabatoff 39c7e7c191
Add more raft metrics, emit more metrics on non-perf standbys (#12166)
Add some metrics helpful for monitoring raft cluster state.

Furthermore, we weren't emitting bolt metrics on regular (non-perf) standbys, and there were other metrics
in metricsLoop that would make sense to include in OSS but weren't.  We now have an active-node-only func,
emitMetricsActiveNode.  This runs metricsLoop on the active node.  Standbys and perf-standbys run metricsLoop
from a goroutine managed by the runStandby rungroup.
2022-10-07 09:09:08 -07:00
Josh Black ad1503ebcd
disable undo logs by default for 1.12.0 (#17453) 2022-10-07 08:47:40 -07:00
Tom Proctor df690ea414
Plugins: Add -version flag to 'vault plugin info' (#17454)
* Add -version flag to 'vault plugin info'
* Allow specifying a builtin tag when reading a single plugin from the catalog
2022-10-07 15:28:15 +01:00
Josh Black c45c6e51c0
only enable undo logs if all cluster members support it (#17378) 2022-10-06 11:24:16 -07:00
Christopher Swenson 06d843d3b0
Don't allow write perms to plugins in test (#17439)
We are specifically checking that the writes are now allowed for group
and other.

I don't know how this test was passing on non-arm64 platforms,
but hopefully this should fix it.
2022-10-06 10:09:27 -07:00
Tom Proctor 2cf29f93c7
Plugins: Add version info to CLI and server log output (#17430) 2022-10-06 12:54:27 +01:00
valli_0x 3ad4d3af6e
core: push entry table type-checking into for loop (#17220)
This commit refactors the `persistAudit`, `persistAuth`, and `persistMount` code paths to perform `entry.Table` type-checking within the same loop as the entry list appending. This saves cycles in the case of success; however, performs some unnecessary appends in the event that an entry has an incorrect table type/value combination.
2022-10-05 15:56:12 -04:00
Nick Cabatoff d5e0353696
Fix unsafe access to perf standby status from systemview (#17186)
Ensure that we don't try to access Core.perfStandby or Core.PerfStandby() from dynamicSystemView, which might be accessed with or without stateLock held.
2022-10-05 08:56:36 -04:00
Tom Proctor 3aa2fe8d8f
Plugins: Tighten requirements for multiplexing (#17403)
Change the multiplexing key to use all `PluginRunner` config (converted to a struct which is comparable), so that plugins with the same name but different env, args, types, versions etc are not incorrectly multiplexed together.

Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
2022-10-05 09:29:29 +01:00
Tom Proctor 4bd5af87f4
Plugins: Fix file permissions check to always use the correct path (#17340)
* Add failing test for when command != plugin name
* wrapFactoryCheckPerms uses pluginCatalog.Get to fetch the correct command
* Use filepath.Rel for consistency with plugin read API handler
2022-09-30 10:33:31 +01:00
Mike Palmiotto 0078822d1d
core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool (#17319)
* core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool

* docs: Update VAULT_ALLOW_PENDING_REMOVAL_MOUNTS doc
2022-09-29 14:35:52 -04:00
Tom Proctor 12ca81bc9b
cli/api: Update plugin listing to always include version info in the response (#17347) 2022-09-29 18:22:33 +01:00
Christopher Swenson 9d0e4986c2
Builtin plugins cannot be overridden in mounts (#17304)
Builtin plugins cannot be overridden in mounts

Before versioning, plugins with the same names as builtins (e.g., "kv")
could be registered, but trying to mount them would always use the
builtin versions.

With versioning, we still allow registering plugins with the same name
as a builtin. However, if the user tries to mount a versioned plugin
with the same name as a builtin, and specifies the non-builtin version
be used, we return an error.

Otherwise the user would see a confusing situation where the one or
both of the Version and RunningVersion would say they were using
the user-requested version, but in reality were using the builtin
version.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2022-09-23 13:14:37 -07:00
Tom Proctor c778f2d822
Plugins: Handle plugins that may be registered with a slash in their name (#17301) 2022-09-23 20:00:10 +01:00
Tom Proctor e21995fa27
Plugins: Update running version everywhere running sha256 is set (#17292) 2022-09-23 11:19:38 +01:00
Tom Proctor c77f009759
Plugins: Allow explicitly specifying the builtin version of a plugin (#17289) 2022-09-22 23:15:46 +01:00
Violet Hynes 2b8d8a3c6a
VAULT-8630 Fix goroutine leak from RLQ initialize (#17281)
* VAULT-8630 Fix goroutine leak from RLQ initialize

* VAULT-8630 Changelog

* VAULT-8630 additional nil check
2022-09-22 15:59:53 -04:00
Tom Proctor 7d09d5a653
CLI: Tune plugin version for auth/secret mounts (#17277)
* Add -plugin-version flag to vault auth/secrets tune
* CLI tests for auth/secrets tune
* CLI test for plugin register
* Plugin catalog listing bug where plugins of different type with the same name could be double counted
* Use constant for -plugin-version flag name
2022-09-22 20:55:46 +01:00
Austin Gebauer a6139cd5b2
Fixes multiplexed plugin initialization after manual plugin reload (#17248)
* Fixes initialize not called after v5 plugin reload

* use request context instead of core activeContext
2022-09-22 10:16:21 -07:00
Yoan Blanc fa8f7c793f
fix: PGP subkeys support (#16224)
* fix: PGP subkeys support

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

* fix: bump ProtonMail/go-crypto

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

* fix: bump ProtonMail/go-crypto

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>

Signed-off-by: Yoan Blanc <yoan@dosimple.ch>
2022-09-22 09:12:41 -04:00
Tom Proctor f920640db7
Plugins: Auto version selection for auth/secrets + tune version (#17167) 2022-09-22 13:53:52 +01:00
Christopher Swenson 895f2c9f3d
Change usages of RunningSha to RunningSha256 (#17266)
Some PRs got crossed and somehow these were missed in the
build checks for #17182.
2022-09-21 13:32:00 -07:00
Christopher Swenson 2c8e88ab67
Check if plugin version matches running version (#17182)
Check if plugin version matches running version

When registering a plugin, we check if the request version matches the
self-reported version from the plugin. If these do not match, we log a
warning.

This uncovered a few missing pieces for getting the database version
code fully working.

We added an environment variable that helps us unit test the running
version behavior as well, but only for approle, postgresql, and consul
plugins.

Return 400 on plugin not found or version mismatch

Populate the running SHA256 of plugins in the mount and auth tables (#17217)
2022-09-21 12:25:04 -07:00
Nick Cabatoff 559754d580
Break grabLockOrStop into two pieces to facilitate investigating deadlocks (#17187)
Break grabLockOrStop into two pieces to facilitate investigating deadlocks.  Without this change, the "grab" goroutine looks the same regardless of who was calling grabLockOrStop, so there's no way to identify one of the deadlock parties.
2022-09-20 11:03:16 -04:00
Nick Cabatoff d8101f82ee
Handle when pluginCatalog.Get returns (nil,nil) during cred backend creation (#17204) 2022-09-20 08:57:08 -04:00
Nick Cabatoff c7f4d79684
We don't need to test LifetimeWatcher's behaviour with database leases specifically. (#17208) 2022-09-20 08:23:51 -04:00
Tom Proctor f5655ae857
Plugins: Consistently use plugin_version (#17171)
* Delete Sha field, rename RunningSha -> RunningSha256
* Rename version -> plugin_version
2022-09-20 12:35:50 +01:00
Steven Clark dae2ef535b
Update protos to match update of protobuf go library (#17215) 2022-09-19 16:45:44 -04:00
Steven Clark 05a5928b8d
Update missing go-kms-wrapping v2 dep and address some ENT->OSS drift (#17178)
* Update missing go-kms-wrapping v2 dep and address some ENT->OSS drift

* Bump go-kms-wrapping/wrappers/gcpckms/v2 to v2.0.1
2022-09-19 10:23:40 -04:00
Mike Palmiotto 2bb11d2d4c
semgrep: Add replication-has-state and fix findings (#17179) 2022-09-19 08:15:27 -04:00
Nick Cabatoff b7c5dbd713
Reduce time taken to run the vault test package (#17157)
Factored out some plugin related tests into their own test package, and added a bunch of parallelism.  Moved some non-plugin tests that were in logical_system_integ_test into another file (keeping them in vault package) just for cohesion.
2022-09-16 09:53:16 -04:00
Christopher Swenson b136a7ecd8
Add plugin version to GRPC interface (#17088)
Add plugin version to GRPC interface

Added a version interface in the sdk/logical so that it can be shared between all plugin types, and then wired it up to RunningVersion in the mounts, auth list, and database systems.

I've tested that this works with auth, database, and secrets plugin types, with the following logic to populate RunningVersion:

If a plugin has a PluginVersion() method implemented, then that is used
If not, and the plugin is built into the Vault binary, then the go.mod version is used
Otherwise, the it will be the empty string.
My apologies for the length of this PR.

* Placeholder backend should be external

We use a placeholder backend (previously a framework.Backend) before a
GRPC plugin is lazy-loaded. This makes us later think the plugin is a
builtin plugin.

So we added a `placeholderBackend` type that overrides the
`IsExternal()` method so that later we know that the plugin is external,
and don't give it a default builtin version.
2022-09-15 16:37:59 -07:00
Josh Black 1e6401a8eb
make proto (#17120) 2022-09-13 16:06:11 -04:00
Josh Black 6d94dd991d
merkle sync undo logs (#17103) 2022-09-13 10:03:19 -07:00
georgethebeatle f9439a9c41
Make key completion work for both kv-v1 and kv-v2 (#16553)
Co-authored-by: Kieron Browne <kbrowne@vmware.com>
Co-authored-by: Georgi Sabev <georgethebeatle@gmail.com>
Co-authored-by: Danail Branekov <danailster@gmail.com>
2022-09-13 12:11:00 -04:00