Commit graph

2667 commits

Author SHA1 Message Date
Tom Proctor 48987ce052
Add stack trace to audit logging panic recovery (#18121) 2022-11-30 17:59:05 +00:00
Nick Cabatoff 12e1b609ac
Create global quotas of each type in every NewTestCluster. (#18038)
Create global quotas of each type in every NewTestCluster.  Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.

NewTestCluster also now starts the cluster, and the Start method becomes a no-op.  Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode.  This was needed because otherwise we can't safely make the quota api call.  We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start. 

The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint.  The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http.  The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
2022-11-29 14:38:33 -05:00
Violet Hynes 78efcb7d6a
VAULT-11786 OSS changes for this change (#18140) 2022-11-29 13:22:15 -05:00
nsimons ce90a6fa38
Make the error and http code clearer when supplying wrong unseal key (#17836)
* Fix typos

* Return http 400 when wrong unseal key is supplied

* Add changelog

* Add test cases and change one more return case to http 400

The new case is triggered when key length is within valid range
[16, 32], but it has uneven bytes, causing crypto/aes to return
invalid key size.

* remove expected in unit tests

* include error in the new error reason

* add multikey and autoseal test cases

* return invalid key for few more code paths
2022-11-28 16:01:47 -08:00
Chris Capurso 2843cfcdc1
VAULT-9427: Add read support to sys/loggers endpoints (#17979)
* add logger->log-level str func

* ensure SetLogLevelByName accounts for duplicates

* add read handlers for sys/loggers endpoints

* add changelog entry

* update docs

* ignore base logger

* fix docs formatting issue

* add ReadOperation support to TestSystemBackend_Loggers

* add more robust checks to TestSystemBackend_Loggers

* add more robust checks to TestSystemBackend_LoggersByName

* check for empty name in delete handler
2022-11-28 11:18:36 -05:00
Tom Proctor 853643d02b
Remove pinned builtin plugin versions from storage (#18051)
* Removes _builtin_ versions from mount storage where it already exists
* Stops new builtin versions being put into storage on mount creation/tuning
* Stops the plugin catalog from returning a builtin plugin that has been overridden, so it more accurately reflects the plugins that are available to actually run
2022-11-23 18:36:25 +00:00
Hamid Ghaf 22f51dc6d6
improve kv CLI to remove data or custom metadata using kv patch (#18067)
* improve kv CLI to remove data or custom metadata using kv patch

* CL

* adding a comment
2022-11-21 17:11:36 -05:00
Tom Proctor dc85e37cf4
storage/raft: Add retry_join_as_non_voter config option (#18030) 2022-11-18 17:58:16 +00:00
Violet Hynes bfeae1fe8e
VAULT-7707 OSS portion of changes (#18019)
* VAULT-7707 OSS portion of changes

* Revert "VAULT-7707 OSS portion of changes"

This reverts commit 5b8cf3882fb7e2427593d59e1439d46b3a5c20a7.

* VAULT-7707 smarter locking behaviour

* VAULT-7707 typo

* VAULT-7707 typo
2022-11-17 16:30:39 -05:00
davidadeleon 3394c28ce1
Deduplicate policies prior to generating ACL on request (#17914)
* Deduplicate policies prior to generating ACL on request

* add changelog

* edit changelog entry
2022-11-16 17:43:46 -05:00
Brian Kassouf 288b0567b1
Barrier: Fix potential locking issue (#17944)
* Barrier: Fix potential locking issue

* add changelog
2022-11-16 09:53:22 -08:00
akshya96 f3c9e98fd5
Vault-8306 User Lockout RPCs oss changes (#17765)
* adding oss file changes

* updating changes from ent
2022-11-15 15:07:52 -08:00
Josh Black 94739c1af6
Don't return a 204 if there's no historical data (#17935)
* don't return a 204 if there's no historical data

* add changelog
2022-11-15 12:15:51 -08:00
Mike Palmiotto 773f0d58ad
plugins: Filter builtins by RunningVersion (#17816)
This commit adds some logic to handle the case where a mount entry has a
non-builtin RunningVersion. This ensures that we only report deprecation
status for builtins.
2022-11-11 14:51:37 -05:00
Anton Averchenkov f9fac68980
Revert "Add mount path into the default generated openapi.json spec (#17839)" (#17890)
This reverts commit 02064eccb42bb2ec1a3d12ec0d49c661312acd2d.
2022-11-10 15:39:53 -08:00
Anton Averchenkov f3aea876b9
Add mount path into the default generated openapi.json spec (#17839)
The current behaviour is to only add mount paths into the generated `opeanpi.json` spec if a `generic_mount_paths` flag is added to the request. This means that we would have to maintain two different `openapi.json` files, which is not ideal. The new solution in this PR is to add `{mount_path}` into every path with a default value specified:

```diff
--    "/auth/token/accessors/": {
++    "/auth/{mount_path}/accessors/": {
      "parameters": [
        {
          "name": "mount_path",
          "description": "....",
          "in": "path",
          "schema": {
            "type": "string",
++          "default": "token"
          }
        }
      ],
```

Additionally, fixed the logic to generate the `operationId` (used to generate method names in the code generated from OpenAPI spec). It had a bug where the ID had `mountPath` in it. The new ID will look like this:

```diff
-- "operationId": "listAuthMountpathAccessors",
++ "operationId": "listTokenAccessors",
```
2022-11-10 15:44:43 -05:00
Josh Black a5c101d851
Fix activity log end time (#17856)
* Correct the end_time in the activity log output for partial counts

* use the real endTime not the passed in one

* add changelog
2022-11-10 12:11:23 -08:00
Violet Hynes 65e8eee0ba
VAULT-8703 Add warning for dangerous undocumented overrides, if used, in status response (#17855)
* VAULT-8703 Add warning for dangerous undocumented overrides, if used, in status response

* VAULT-8703 add changelog

* VAULT-8703 fix append
2022-11-09 11:04:36 -05:00
divyaac 2d3775a93b
Introspection API Implementation for Router Struct (#17789)
* OSS Commit from ENT for Introspection API

* Add changelog
2022-11-04 09:39:09 -07:00
Jason O'Donnell 4e122214f7
core: fix start up policy loading race condition on perf standbys (#17801)
* core: fix start up policy loading race condition on perf standbys

* Use correct bool for perf standby

* changelog
2022-11-03 13:01:39 -04:00
akshya96 2945924b2b
Vault 8305 Prevent Brute Forcing in Auth methods : Setting user lockout configuration (#17338)
* config file changes

* lockout config changes

* auth tune r/w and auth tune

* removing changes at enable

* removing q.Q

* go mod tidy

* removing comments

* changing struct name for config file

* fixing mount tune

* adding test file for user lockout

* fixing comments and add changelog

* addressing comments

* fixing mount table updates

* updating consts in auth_tune

* small fixes

* adding hcl parse test

* fixing config compare

* fixing github comments

* optimize userlockouts.go

* fixing test

* minor changes

* adding comments

* adding sort to flaky test

* fix flaky test
2022-11-01 11:02:07 -07:00
Mike Palmiotto a9dcc45f72
Tweak totp test to fix race failures (#17692) 2022-10-27 09:41:40 -04:00
akshya96 1e189016e2
update protoc version to 3.21.7 oss (#17499)
* update protoc to 3.21.7

* adding changelog
2022-10-26 16:49:44 -07:00
Mike Palmiotto cc96c6f470
Store login MFA secret with tokenhelper (#17040)
* Store login MFA secret with tokenhelper
* Clean up and refactor tokenhelper paths
* Refactor totp test code for re-use
* Add login MFA command tests
* Use longer sleep times and sha512 for totp test
* Add changelog
2022-10-26 17:02:26 -04:00
Violet Hynes 73f9b13762
VAULT-9451 Fix data race in entity merge (#17631) 2022-10-21 16:47:59 -04:00
Violet Hynes 5861c51e70
VAULT-8719 Support data array for alias clash error response so UI/machines can understand error (#17459)
* VAULT-8719 Support data array for alias clash error response so UI can understand error

* VAULT-8719 Changelog

* VAULT-8719 Update alias mount update logic

* VAULT-8719 Further restrict IsError()
2022-10-17 14:46:25 -04:00
Nick Cabatoff f94bd10540
Tolerate NamespaceByID returning (nil,nil) when looking up an mfa enforcement's ns (#17562) 2022-10-17 09:18:02 -04:00
Hamid Ghaf 8a624c1264
prevent memory leak when using control group factors in a policy (#17532)
* prevent a possible memory leak when using control group factors in a policy

* CL
2022-10-14 19:15:15 -04:00
Mike Palmiotto 81459de6fd
core: Move rollback period init to NewCore (#17547) 2022-10-13 18:39:00 -04:00
Nick Cabatoff 8e67651dcd
Fix a data race with rollbackPeriod. (#17387) 2022-10-13 09:59:07 -04:00
Chris Capurso 2c9b29ab42
fix off by one err in current month client count computation (#17457) 2022-10-07 12:37:09 -04:00
Nick Cabatoff 39c7e7c191
Add more raft metrics, emit more metrics on non-perf standbys (#12166)
Add some metrics helpful for monitoring raft cluster state.

Furthermore, we weren't emitting bolt metrics on regular (non-perf) standbys, and there were other metrics
in metricsLoop that would make sense to include in OSS but weren't.  We now have an active-node-only func,
emitMetricsActiveNode.  This runs metricsLoop on the active node.  Standbys and perf-standbys run metricsLoop
from a goroutine managed by the runStandby rungroup.
2022-10-07 09:09:08 -07:00
Josh Black ad1503ebcd
disable undo logs by default for 1.12.0 (#17453) 2022-10-07 08:47:40 -07:00
Tom Proctor df690ea414
Plugins: Add -version flag to 'vault plugin info' (#17454)
* Add -version flag to 'vault plugin info'
* Allow specifying a builtin tag when reading a single plugin from the catalog
2022-10-07 15:28:15 +01:00
Josh Black c45c6e51c0
only enable undo logs if all cluster members support it (#17378) 2022-10-06 11:24:16 -07:00
Christopher Swenson 06d843d3b0
Don't allow write perms to plugins in test (#17439)
We are specifically checking that the writes are now allowed for group
and other.

I don't know how this test was passing on non-arm64 platforms,
but hopefully this should fix it.
2022-10-06 10:09:27 -07:00
Tom Proctor 2cf29f93c7
Plugins: Add version info to CLI and server log output (#17430) 2022-10-06 12:54:27 +01:00
valli_0x 3ad4d3af6e
core: push entry table type-checking into for loop (#17220)
This commit refactors the `persistAudit`, `persistAuth`, and `persistMount` code paths to perform `entry.Table` type-checking within the same loop as the entry list appending. This saves cycles in the case of success; however, performs some unnecessary appends in the event that an entry has an incorrect table type/value combination.
2022-10-05 15:56:12 -04:00
Nick Cabatoff d5e0353696
Fix unsafe access to perf standby status from systemview (#17186)
Ensure that we don't try to access Core.perfStandby or Core.PerfStandby() from dynamicSystemView, which might be accessed with or without stateLock held.
2022-10-05 08:56:36 -04:00
Tom Proctor 3aa2fe8d8f
Plugins: Tighten requirements for multiplexing (#17403)
Change the multiplexing key to use all `PluginRunner` config (converted to a struct which is comparable), so that plugins with the same name but different env, args, types, versions etc are not incorrectly multiplexed together.

Co-authored-by: Christopher Swenson <christopher.swenson@hashicorp.com>
2022-10-05 09:29:29 +01:00
Tom Proctor 4bd5af87f4
Plugins: Fix file permissions check to always use the correct path (#17340)
* Add failing test for when command != plugin name
* wrapFactoryCheckPerms uses pluginCatalog.Get to fetch the correct command
* Use filepath.Rel for consistency with plugin read API handler
2022-09-30 10:33:31 +01:00
Mike Palmiotto 0078822d1d
core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool (#17319)
* core: Parse VAULT_ALLOW_PENDING_REMOVAL_MOUNTS as bool

* docs: Update VAULT_ALLOW_PENDING_REMOVAL_MOUNTS doc
2022-09-29 14:35:52 -04:00
Tom Proctor 12ca81bc9b
cli/api: Update plugin listing to always include version info in the response (#17347) 2022-09-29 18:22:33 +01:00
Christopher Swenson 9d0e4986c2
Builtin plugins cannot be overridden in mounts (#17304)
Builtin plugins cannot be overridden in mounts

Before versioning, plugins with the same names as builtins (e.g., "kv")
could be registered, but trying to mount them would always use the
builtin versions.

With versioning, we still allow registering plugins with the same name
as a builtin. However, if the user tries to mount a versioned plugin
with the same name as a builtin, and specifies the non-builtin version
be used, we return an error.

Otherwise the user would see a confusing situation where the one or
both of the Version and RunningVersion would say they were using
the user-requested version, but in reality were using the builtin
version.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2022-09-23 13:14:37 -07:00
Tom Proctor c778f2d822
Plugins: Handle plugins that may be registered with a slash in their name (#17301) 2022-09-23 20:00:10 +01:00
Tom Proctor e21995fa27
Plugins: Update running version everywhere running sha256 is set (#17292) 2022-09-23 11:19:38 +01:00
Tom Proctor c77f009759
Plugins: Allow explicitly specifying the builtin version of a plugin (#17289) 2022-09-22 23:15:46 +01:00
Violet Hynes 2b8d8a3c6a
VAULT-8630 Fix goroutine leak from RLQ initialize (#17281)
* VAULT-8630 Fix goroutine leak from RLQ initialize

* VAULT-8630 Changelog

* VAULT-8630 additional nil check
2022-09-22 15:59:53 -04:00
Tom Proctor 7d09d5a653
CLI: Tune plugin version for auth/secret mounts (#17277)
* Add -plugin-version flag to vault auth/secrets tune
* CLI tests for auth/secrets tune
* CLI test for plugin register
* Plugin catalog listing bug where plugins of different type with the same name could be double counted
* Use constant for -plugin-version flag name
2022-09-22 20:55:46 +01:00
Austin Gebauer a6139cd5b2
Fixes multiplexed plugin initialization after manual plugin reload (#17248)
* Fixes initialize not called after v5 plugin reload

* use request context instead of core activeContext
2022-09-22 10:16:21 -07:00