Commit graph

2712 commits

Author SHA1 Message Date
Christopher Swenson fcbce0effd
Start events when core starts if enabled (#18742)
For example, using:

```sh
vault server -dev -experiment events.beta1
```

Tested by checking that the events were enabled and disabled
when the `-experiment events.beta1` flag was present and absent.

Also added a small fix to pass the `hclog.Logger` in now so that
the logging hierarchy and levels are respected.
2023-01-18 10:46:01 -08:00
Max Bowsher 4c5f583f39
OpenAPI generic_mount_paths follow-up (#18663)
* OpenAPI `generic_mount_paths` follow-up

An incremental improvement within larger context discussed in #18560.

* Following the revert in #18617, re-introduce the change from
  `{mountPath}` to `{<path-of-mount>_mount_path}`; this is needed, as
  otherwise paths from multiple plugins would clash - e.g. almost every
  auth method would provide a conflicting definition for
  `auth/{mountPath}/login`, and the last one written into the map would
  win.

* Move the half of the functionality that was in `sdk/framework/` to
  `vault/logical_system.go` with the rest; this is needed, as
  `sdk/framework/` gets compiled in to externally built plugins, and
  therefore there may be version skew between it and the Vault main
  code. Implementing the `generic_mount_paths` feature entirely on one
  side of this boundary frees us from problems caused by this.

* Update the special exception that recognizes `system` and `identity`
  as singleton mounts to also include the other two singleton mounts,
  `cubbyhole` and `auth/token`.

* Include a comment that documents to restricted circumstances in which
  the `generic_mount_paths` option makes sense to use:

	    // Note that for this to actually be useful, you have to be using it with
	    // a Vault instance in which you have mounted one of each secrets engine
	    // and auth method of types you are interested in, at paths which identify
	    // their type, and for the KV secrets engine you will probably want to
	    // mount separate kv-v1 and kv-v2 mounts to include the documentation for
	    // each of those APIs.

* Fix tests

Also remove comment "// TODO update after kv repo update" which was
added 4 years ago in #5687 - the implied update has not happened.

* Add changelog

* Update 18663.txt
2023-01-17 23:07:11 -05:00
akshya96 6e04e4ede1
Prevent brute forcing : telemetry oss changes (#18718)
* Prevent brute forcing : telemetry oss changes

* adding changelog
2023-01-17 15:10:50 -08:00
akshya96 b2276a369a
Prevent Brute Forcing: Create an api endpoint to list locked users OSS changes (#18675)
* api to list lockedusers oss changes

* add changelog
2023-01-17 14:25:56 -08:00
Josh Black c9763996d4
Enable undo logs by default (#18692)
* Enable undo logs by default

* add consul test

* update go.mod/sum

* add a better non-existent key
2023-01-17 13:38:18 -08:00
Christopher Swenson b95beeb675
Add basic event bus broker stub (#18640)
Creates a new `eventbus` package under `vault` with
an implementation of the `go-eventlogger` broker.

Also creates a stub of a common broker that will be accessible
in the core, and creates a simple event sending interface.
2023-01-17 13:34:37 -08:00
Tom Proctor d5c35f39c3
Add experiment system + events experiment (#18682) 2023-01-16 16:07:18 +00:00
Peter Wilson 59450ecb82
Revert "Add new clients into the monthly breakdown (#18629)" (#18726)
This reverts commit d641bbc28e5e8cc12b81d409e5d5fc1f2cb7f66c.
2023-01-16 15:51:19 +00:00
akshya96 78546af8fc
Vault 8308 Background thread to update locked user entries (#18673)
* background thread changes

* adding changelog

* fix changelog typo
2023-01-12 14:09:33 -08:00
Chris Capurso 03b23a7970
VAULT-12548: Fix data races in Link status reporting (#18674)
* pin hcp-scada-provider

* fix link reportStatus data race

* go get hcp-scada-provider@v0.2.1
2023-01-12 15:10:34 -05:00
Ellie 6f7757e949
add core state lock deadlock detection config option v2 (#18604)
* add core state lockd eadlock detection config option v2

* add changelog

* split out NewTestCluster function to maintain build flag

* replace long func with constant

* remove line

* rename file, and move where detect deadlock flag is set
2023-01-11 13:32:05 -06:00
Max Bowsher 6d6a726f9d
Fix HelpOperation on sudo-protected paths (#18568)
* Fix HelpOperation on sudo-protected paths

Fixes #18566

* Add changelog
2023-01-10 12:17:16 -06:00
Anton Averchenkov c7f40361eb
Revert "Add mount path into the default generated openapi.json spec (#17926)" (#18617)
* Revert "Add mount path into the default generated openapi.json spec (UI) (#17926)"

This reverts commit db8efac708e5385ec871be9558507eeaf54ac972.

* Revert "Remove `generic_mount_paths` field (#18558)"

This reverts commit 79c8f626c59ca11bb8e7f460d40b09f5e0cec76d.
2023-01-10 11:16:59 -05:00
Josh Black d3f822a938
Add new clients into the monthly breakdown (#18629)
* Add new clients into the monthly breakdown

* add changelog
2023-01-09 15:26:11 -08:00
Chris Capurso 25d0afae23
VAULT-11830: Expand NodeStatusReporter with new fields (#18302)
* expand NodeStatusReporter with new fields

* only call IsRaftVoter if using raft storage

* add changelog entry

* fix listeners

* return LogLevel as enum

* update github.com/hashicorp/vault/vault/hcp_link/proto

* add changelog entry

* bump github.com/hashicorp/vault/vault/hcp_link/proto

* go mod tidy
2023-01-06 20:53:09 -05:00
Chris Capurso bb0c92afe7
VAULT-11829: Add cluster status handler (#18351)
* go get link proto @vault-11829-meta-get-cluster-status

* add HA status

* add HAEnabled method

* add raft config

* allocate HA nodes based on actual count

* add raft autopilot status

* add raft quorum warnings

* add ClusterID method

* add StorageType

* add ClusterID

* update github.com/hashicorp/vault/vault/hcp_link/proto

* add changelog entry

* fix raft config panic

* remove "Warning" quorum message prefix

* add error wrapping

* add Core.HAStateWithLock method

* reduce quorum warnings to single string

* fix HCP_API_HOST test env var check

* Revert "fix HCP_API_HOST test env var check"

This reverts commit 97c73c4798b77b84aea84f341f2c63c4d657914d.
2023-01-06 17:06:54 -05:00
Chris Capurso 82eaecd745
change quorum warning to a singular string (#18619) 2023-01-06 15:31:27 -05:00
Chris Capurso de59e29bbc
VAULT-11829: Add GetClusterStatus rpc to meta capability (#18316)
* add GetClusterStatus rpc to meta capability

* consolidate HA-related info

* add storage type
2023-01-05 13:33:08 -05:00
Violet Hynes ae653a05a7
VAULT-12489 OSS Changes (#18607) 2023-01-05 13:00:55 -05:00
Josh Black c8a8c21cee
Account for mount counts when de-duplicating current and historical month data (#18598)
* Account for mount counts when de-duplicating current and historical month data

* add changelog
2023-01-05 09:34:05 -08:00
Chris Capurso e7787a94e0
VAULT-11827: Add new Link node level fields (#18203)
* add addition Link node-level status fields

* pin protoc to 3.21.9

* make proto

* change LogLevel to be a string

* add RaftStatus; IsActive -> Activetime

* use an enum for LogLevel
2023-01-05 11:16:48 -05:00
Max Bowsher 7d87548f4f
Fix duplicate definition of path sys/internal/specs/openapi (#18553)
This was accidentally duplicated in #5687.

Remove the second definition, which was shadowed by the first, and move
the documentation that was part of the second to the surviving version.
2023-01-04 22:48:40 -05:00
Ikko Eltociear Ashimine 6927478ab9
Fix typo in mount.go (#18575)
heirarchy -> hierarchy
2023-01-04 21:52:42 -05:00
vinay-gopalan bbd8ac9bbf
Upgrade go.opentelemetry.io/otel from v0.20.0 to v1.11.2 (#18589) 2023-01-04 11:31:30 -08:00
Chris Capurso 0635d304de
only update SCADA metadata if status changes (#18585)
* only update SCADA metadata if status changes

* add changelog entry
2023-01-04 11:09:51 -05:00
Max Bowsher c4e8e22963
Remove generic_mount_paths field (#18558)
PR #17926 already deleted the implementation of the
`generic_mount_paths` field so it needs to be removed from the declared
fields of the path too, so help and OpenAPI isn't misleading.
2023-01-03 19:14:29 -05:00
Max Bowsher 4052c785ea
Remove unreachable code (#18576)
I happened to spot that the `TemplateError` type is never instantiated.
Therefore delete it, and code referencing it.
2023-01-03 09:02:01 -05:00
akshya96 4126060d88
Prevent Brute Forcing: Create api endpoint to unlock users (#18279)
* code changes for unlock

* add test

* adding sys help

* adding sys help

* updating unlock user function

* edit test

* add changelog

* syshelp

* adding open api response definition

* removing response fields

* change path name
2022-12-19 14:24:42 -08:00
Josh Black cd7d6d5761
De-duplicate namespaces when historical and current month data are mixed (#18452)
* De-duplicate namespaces when historical and current month data are mixed

* add changelog
2022-12-16 16:02:42 -08:00
Scott Miller 53e73ceba2
Use a cleaner worker pattern (#18422) 2022-12-16 11:35:24 -06:00
divyaac f8ad8bc5a5
OSS PR for Config Changes PR (#18418)
* OSS PR for Config Changes PR

* Edited tests

* typo

* Added changelog

* Remove changelog
2022-12-15 12:19:19 -08:00
Nick Cabatoff 429916c135
Prevent panics in expiration invalidation, and make some changes for testing (#18401) 2022-12-15 18:09:36 +00:00
Mike Palmiotto cb3406b1eb
plugins: Handle mount/enable for shadowed builtins (#17879)
* Allow mounting external plugins with same name/type as deprecated builtins
* Add some go tests for deprecation status handling
* Move timestamp storage to post-unseal
* Add upgrade-aware deprecation shutdown and tests
2022-12-14 13:06:33 -05:00
Mike Palmiotto 809a04c8b4
core: Make shutdownDoneCh atomic (#18358)
When issuing a core.Shutdown(), it is common to background the shutdown
request. This allows Vault to continue cleaning up, mainly to release
the stateLock. This allows the shutdown to complete, but is inherently
racy, so the core.shutdownDoneCh needs to be made atomic.
2022-12-14 15:59:11 +00:00
Scott Miller 25bff579ea
Use a small pool of workers to run postUnsealFuncs in parallel (#18244)
* Initial worker pool

* Run postUnsealFuncs in parallel

* Use the old logic for P=1

* changelog

* Use a CPU count relative worker pool

* Update vault/core.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Done must be called once per postUnsealFunc

* Defer is overkill

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2022-12-12 17:07:53 -06:00
Chris Capurso 42f36605c7
use meta and link_control from link proto (#18290) 2022-12-09 12:35:06 -05:00
Chris Capurso c81485d5cd
copy over link_control and meta to link proto module (#18285) 2022-12-09 11:57:35 -05:00
Violet Hynes 176c149a38
VAULT-8336 Fix default rate limit paths (#18273)
* VAULT-8336 Fix default rate limit paths

* VAULT-8336 changelog
2022-12-09 08:49:17 -05:00
Chris Capurso 4dc5155c5f
Link OSS (#18228)
* add Link config, init, and capabilities

* add node status proto

* bump protoc version to 3.21.9

* make proto

* adding link tests

* remove wrapped link

* add changelog entry

* update changelog entry
2022-12-08 15:02:18 -05:00
Anton Averchenkov 493040d147
Add mount path into the default generated openapi.json spec (UI) (#17926) 2022-12-08 12:15:54 -05:00
Nick Cabatoff 1b745aef58
Prevent autopilot from demoting voters when they join a 2nd time (#18263) 2022-12-07 14:17:45 -05:00
Nick Cabatoff 342b61984a
Move version out of SDK. (#14229)
Move version out of SDK.  For now it's a copy rather than move: the part not addressed by this change is sdk/helper/useragent.String, which we'll want to remove in favour of PluginString.  That will have to wait until we've removed uses of useragent.String from all builtins.
2022-12-07 13:29:51 -05:00
akshya96 1801f09c6a
Vault 8307 user lockout workflow oss (#17951)
* adding oss file changes

* check disabled and read values from config

* isUserLocked, getUserLockout Configurations, check user lock before login and return error

* remove stale entry from storage during read

* added failed login process workflow

* success workflow updated

* user lockouts external tests

* changing update to support delete

* provide access to alias look ahead function

* adding path alias lookahead

* adding tests

* added changelog

* added comments

* adding changes from ent branch

* adding lock to UpdateUserFailedLoginInfo

* fix return default bug
2022-12-06 17:22:46 -08:00
Mike Palmiotto ea41e62e83
plugins: Mount missing plugin entries and skip loading (#18189)
* Skip plugin startup for missing plugins
* Skip secrets startup for missing plugins
* Add changelog for bugfix
* Make plugin handling on unseal version-aware
* Update plugin lazy-load logic/comments for readability
* Add register/mount/deregister/seal/unseal go test
* Consolidate lazy mount logic to prevent inconsistencies

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2022-12-02 13:16:31 -05:00
Tom Proctor 05aeab2752
Fix plugin list API when audit logging enabled (#18173)
* Add test that fails due to audit log panic
* Rebuild VersionedPlugin as map of primitive types before adding to response
* Changelog
* Fix casting in external plugin tests
2022-12-01 10:44:44 +00:00
Tom Proctor 48987ce052
Add stack trace to audit logging panic recovery (#18121) 2022-11-30 17:59:05 +00:00
Nick Cabatoff 12e1b609ac
Create global quotas of each type in every NewTestCluster. (#18038)
Create global quotas of each type in every NewTestCluster.  Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.

NewTestCluster also now starts the cluster, and the Start method becomes a no-op.  Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode.  This was needed because otherwise we can't safely make the quota api call.  We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start. 

The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint.  The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http.  The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
2022-11-29 14:38:33 -05:00
Violet Hynes 78efcb7d6a
VAULT-11786 OSS changes for this change (#18140) 2022-11-29 13:22:15 -05:00
nsimons ce90a6fa38
Make the error and http code clearer when supplying wrong unseal key (#17836)
* Fix typos

* Return http 400 when wrong unseal key is supplied

* Add changelog

* Add test cases and change one more return case to http 400

The new case is triggered when key length is within valid range
[16, 32], but it has uneven bytes, causing crypto/aes to return
invalid key size.

* remove expected in unit tests

* include error in the new error reason

* add multikey and autoseal test cases

* return invalid key for few more code paths
2022-11-28 16:01:47 -08:00
Chris Capurso 2843cfcdc1
VAULT-9427: Add read support to sys/loggers endpoints (#17979)
* add logger->log-level str func

* ensure SetLogLevelByName accounts for duplicates

* add read handlers for sys/loggers endpoints

* add changelog entry

* update docs

* ignore base logger

* fix docs formatting issue

* add ReadOperation support to TestSystemBackend_Loggers

* add more robust checks to TestSystemBackend_Loggers

* add more robust checks to TestSystemBackend_LoggersByName

* check for empty name in delete handler
2022-11-28 11:18:36 -05:00