Commit Graph

2863 Commits

Author SHA1 Message Date
Anton Averchenkov 254c5e2568
openapi: Add display attributes for /sys (p1) (#19706) 2023-04-13 11:32:26 -04:00
Anton Averchenkov d0cc7bc71a
openapi: Add display attributes for identity/ (remaining) (#19763) 2023-04-12 15:46:01 -04:00
Anton Averchenkov 31e123f7a0
openapi: Add display attributes for identity/group (#19762) 2023-04-12 15:45:12 -04:00
Anton Averchenkov 4b94669779
openapi: Add display attributes for identity/entity (#19760) 2023-04-12 15:44:43 -04:00
Anton Averchenkov f69bea9161
openapi: Add display attributes for identity/oidc (#19758) 2023-04-12 15:44:07 -04:00
Tom Proctor 9aa9686c81
Simplify tracking of external plugins (#20009) 2023-04-12 18:34:35 +01:00
miagilepner 1ea85c56d7
VAULT-14734: activity log write endpoint (#20019)
* add noop endpoint with testonly build flag

* add tests for endpoint

* cleanup

* fix test name

* add changelog

* pr fixes
2023-04-12 18:26:26 +02:00
Mike Palmiotto 1b5d527521
api: Add reporting fields to activitylog config endpoint (#20086)
This PR adds the internal reporting state to the
`internal/counters/config` read endpoint:
* reporting_enabled
* billing_start_timestamp
2023-04-12 12:02:28 -04:00
Hamid Ghaf 976ab9c87f
Revert "Remove dead licensing code and extra core setup (#20080)" (#20112)
This reverts commit 60d8bff89c490c5489c97b98b168de9a50239815.
2023-04-12 13:46:15 +00:00
Anton Averchenkov 618e79af82
openapi: Add display attributes for identity/mfa (#19757) 2023-04-11 16:00:35 -04:00
Christopher Swenson df8d0444dd
Remove dead licensing code and extra core setup (#20080)
The extra core setup is no longer needed in Vault Enterprise, and the
licensing code here has no effect here or in Vault Enterprise.

I pulled this commit into Vault Enterprise and it still compiled fine,
and all tests pass. (Though a few functions can be deleted there as
well after this is merged.)
2023-04-11 10:30:05 -07:00
miagilepner c8b4afd189
Require activity log retention months at least the minimum (#20078)
* reject retention month updates that are less than min retention months

* add changelog

* reword error

* switch to retention_months
2023-04-11 15:09:01 +00:00
Alexander Scheel f0fc48c788
Only log stopping rollback manager once (#20041)
When testing the Rollback Manager's one-time invocation in Enterprise,
it was noticed that due to the channel being closed, we'd always hit
this case and thus spam logs rather quickly with this message.

Switch to a boolean flip to log this once, as it is not executed in
parallel and thus doesn't need a sync.Once.

This only affected anyone calling the test core's
StopAutomaticRollbacks() helper.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-04-10 13:24:03 -04:00
Anton Averchenkov 56698d6d15
openapi: Add display attributes for token/ (#19399) 2023-04-06 11:11:49 -04:00
miagilepner 3b91b9ebbf
VAULT-14733: SegmentReader interface for reading activity log segments (#19934)
* create a segment reader for activity log segment

* fix imports

* updates based on comments
2023-04-06 16:23:41 +02:00
Thy Ton fcf06d5874
feat: add plugin metadata to audit logging (#19814) 2023-04-06 00:41:07 -07:00
Jason O'Donnell e4e34c0f96
Add extra ssct gen counter debug logs (#19980) 2023-04-05 13:06:57 -04:00
miagilepner 09adb9ee12
VAULT-14733: Refactor processClientRecord in activity log (#19933) 2023-04-04 14:50:19 +02:00
Peter Wilson a2bdf7250b
VAULT-14048: raft-autopilot appears to refuse to remove a node which has left and wouldn't impact stability (#19472)
* ensure we supply the node type when it's for a voter
* bumped autopilot version back to v0.2.0 and ran go mod tidy
* changed condition in knownservers and added some comments
* Export GetRaftBackend
* Updated tests for autopilot (related to dead server cleanup)
* Export Raft NewDelegate

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-04-03 11:58:57 -04:00
Steven Clark b1c557bd73
Extend expiration of diagnose test-fixture certificate (#19868)
- Regenerated with
    - openssl x509 -x509toreq -in xxx.crt -signkey xxx -out xxx.csr
    - openssl x509 -req -in xxx.csr -signkey xxx.pem -days 18250 -out xxx.crt
2023-04-03 09:34:58 -04:00
miagilepner de56c728a1
VAULT-13191: OSS changes (#19891)
* add open source changes for reporting

* fix function signature

* add changelog
2023-03-31 15:05:16 +00:00
Alexander Scheel e95fadd8f0
Add mechanism to disable automatic rollbacks (#19748)
When testing the rollback mechanism, there's two categories of tests
typically written:

 1. Ones in which the rollback manager is entirely left alone, which
    usually are a bit slower and less predictable. However, it is still
    sufficient in many scenarios.
 2. Ones in which the rollback manager is explicitly probed by tests
    and "stepped" to achieve the next rollback.

Here, without a mechanism to fully disable the rollback manager's
periodic ticker (without affecting its ability to work!) we'll continue
to see races of the sort:

>     --- FAIL: TestRevocationQueue (50.95s)
>     panic: sync: WaitGroup is reused before previous Wait has returned [recovered]
>         panic: sync: WaitGroup is reused before previous Wait has returned

This allows us to disable the ticker, returning control to the test
suite entirely.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-03-29 08:59:01 -04:00
Daniel Huckins 243c86b2c5
VAULT-12144: add openapi responses for /sys/rotate endpoints (#18624)
* responses for rotate endpoints

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

* add test for rotate config

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* update to use newer function

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use new func

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-03-28 15:40:48 -04:00
Daniel Huckins 4b52cea28c
VAULT-12144: add openapi responses for /sys/seal endpoints (#18625)
* added responses to seal/unseal endpoints

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add response for /seal-status

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added change log

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-03-28 15:39:08 -04:00
Daniel Huckins e33b87a2c3
VAULT-12144: add openapi responses for assorted /sys endpoints (#18628)
* added response struct for version-history

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add response struct for leader

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add response struct for ha-status

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add response struct for host-info

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add response struct for in-flight-req

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* make fmt

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-03-28 15:38:35 -04:00
Daniel Huckins d8191e0a4b
mark internal/counters/requests as deprecated (#19359)
Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-03-28 15:31:20 -04:00
Daniel Huckins f34313e611
VAULT-12144: add openapi responses for /sys/wrapping endpoints (#18627)
* add response structures for /sys/wrapping endpoints

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

* dynamic tests should be nil

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-03-28 11:12:34 -04:00
Violet Hynes 44894253fc
VAULT-8337 OSS changes part 2 (#19698) 2023-03-28 10:32:30 -04:00
Daniel Huckins e3d3d6e528
VAULT-12144: add openapi responses for /sys/tools endpoints (#18626)
* add struct for /sys/tools/hash

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added responses for /sys/tools paths

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add changelog

* verify respose structure for hash

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* verify respose structure for hash/random

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use newer testing funct

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use new test method

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-03-24 23:11:39 +00:00
Nick Cabatoff fae3e31fda
Address regression introduced by #15493 for non-raft storage backends. (#19721) 2023-03-24 10:15:25 -04:00
Violet Hynes db31cf2da2
Revert "VAULT-8337 OSS changes" (#19700)
This reverts commit 160196f4bc270ea0698b894d4f5f3e84426b77c9.
2023-03-23 11:47:10 -04:00
Nick Cabatoff 7de257de46
Make sure that we specify Backoff in conjunction with MinConnectTimeout, else we get a zero value. (#19701) 2023-03-23 10:21:28 -04:00
Nick Cabatoff 06e3f971ef
Allow overriding gRPC's connection timeout with VAULT_GRPC_MIN_CONNECT_TIMEOUT (#19676) 2023-03-22 18:51:37 +00:00
Chris Capurso 12cf80fafd
user Platform from host info for node status OS (#19611) 2023-03-21 09:53:31 -04:00
Mike Palmiotto d6a1cc63ac
activitylog: Fix pq.Get trace logger output (#19650) 2023-03-20 19:35:40 -04:00
Mike Palmiotto 2381e6be66
Add no-op CensusAgent (#19625)
* Add no-op CensusAgent

* Changelog for Census Agent background worker
2023-03-20 10:51:35 -04:00
Violet Hynes e627737fe8
VAULT-8337 OSS changes (#19580) 2023-03-20 09:04:55 -04:00
Tom Proctor f1f5c8444a
Suppress event broker not started log warning (#19593) 2023-03-20 11:14:14 +00:00
Chris Capurso 078d4eef7c
update link policy fetch URL (#19371)
* update link policy fetch URL

* fix fmt
2023-03-17 09:09:25 -04:00
Violet Hynes 943678e359
Fix remount for mounts with spaces in the name (#19585)
* Fix remount for mounts with spaces in the name

* Git mishap

* Git mishap

* Changelog

* Godocs for tests
2023-03-16 15:26:55 -04:00
Hamid Ghaf 27bb03bbc0
adding copyright header (#19555)
* adding copyright header

* fix fmt and a test
2023-03-15 09:00:52 -07:00
John-Michael Faircloth 1553c310c4
Fix a possible data race with rollback manager and plugin reload (#19468)
* fix data race on plugin reload

* add changelog

* add comment for posterity

* revert comment and return assignment in router.go

* rework plugin continue on error tests to use compilePlugin

* fix race condition on route entry

* add test for plugin reload and rollback race detection

* add go doc for test
2023-03-14 09:36:37 -05:00
Marc Boudreau f286ee5b3c
Fix failing TestHCPLinkConnected Test (#19474)
* replace use of os.Unsetenv in test with t.Setenv and remove t.Parallel from test that rely on env being modified.

* experiment with using fromJSON function

* revert previous experiment

* including double quotes in the output value for the string ubuntu-latest

* use go run to launch gofumpt
2023-03-09 13:46:54 -05:00
nsimons d91d2ceaf8
Fix cubbyhole and token revocation for legacy service tokens (#19416)
* Fix cubbyhole and revocation for legacy service tokens

Legacy service tokens generated in Vault 1.10+ with env var
VAULT_DISABLE_SERVER_SIDE_CONSISTENT_TOKENS=true are not assigned
a cubbyhole ID. The implication is that cubbyhole/ cannot be
used, nor can the tokens be revoked.

This commit assigns a cubbyhole ID to these tokens and adds
a new test case to see that cubbyhole and revocation works correctly.

* add changelog

* add godoc to test cases
2023-03-06 15:09:45 -05:00
miagilepner ac36b31846
VAULT-13729 activity log test godocs (#19433)
* add godocs to activity log tests

* format

* add trailing periods
2023-03-06 13:08:22 +01:00
Daniel Huckins d9229a5fba
VAULT-12112: add openapi responses for /sys/internal endpoints (#18542)
* added responses for sys/internal/ui/mounts

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* responses for internal paths

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

* add schema validation for internal/ui/mounts

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add counters test

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* update test to use new method

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use new method in TestSystemBackend_InternalUIMounts

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* :rage4: fixed test, diff between core.HandleRequest and backend.HandleRequest

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* test feature flags

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-02-24 15:03:21 -05:00
miagilepner c31a10b90a
VAULT-13763 normalize activity log mount paths (#19343)
* add slashes to mount paths in activity log

* cleanup test

* fix test
2023-02-24 16:57:41 +01:00
Christopher Swenson 6b36cc7587
When copying test binary, delete first (#19331)
For plugin tests, we copy the test binary. On macOS, if the
destination binary already exists, then copying over it will result
in an invalid signature.

The easiest workaround is to delete the file before copying.
2023-02-23 15:10:13 -08:00
Jason O'Donnell f69297e0b3
Fix inmem layer unlock bug (#19323) 2023-02-23 20:16:49 +00:00
John-Michael Faircloth 0a7656ae5c
test: Fix bug in TestAddTestPlugin test helper (#19313)
* fix external plugin test failing locally

* Ensure file is closed and written in TestAddTestPlugin
2023-02-23 17:07:48 +00:00
Leland Ursu 432fad12b1
added in the missing test cases to validate response structures (#19277)
* added in the missing test cases to validate response structures

* added changelog file

* remove unneeded changelog file

* removed comment to update when indentity/entity is implemented

---------

Co-authored-by: lursu <leland.ursu@hashicorp.com>
2023-02-22 12:46:46 -05:00
Chris Capurso 3a361e1e83
add error consistency in link node status resp (#19279) 2023-02-22 11:53:29 -05:00
John-Michael Faircloth 2cc6117f28
test/plugin: add more test scenarios for external plugins (#19257) 2023-02-21 09:44:54 -06:00
Christopher Swenson 404d7a57bb
events: WS protobuf messages should be binary (#19232)
The [WebSockets spec](https://www.rfc-editor.org/rfc/rfc6455) states
that text messages must be valid UTF-8 encoded strings, which protobuf
messages virtually never are. This now correctly sends the protobuf events
as binary messages.

We change the format to correspond to CloudEvents, as originally intended,
and remove a redundant timestamp and newline.

We also bump the eventlogger to fix a race condition that this code triggers.
2023-02-17 11:38:03 -08:00
John-Michael Faircloth eca810d06e
test/plugin: test external database plugin workflows (#19191)
* test/plugin: test external db plugin

* use test helper to get cluster and plugins

* create test helper to create a vault admin user

* add step to revoke lease

* make tests parallel and add reload test

* use more descriptive name for test group; check response
2023-02-16 15:52:24 -06:00
Daniel Huckins 448f5dd33e
VAULT-12112: add openapi response structures for /sys/config and /sys/generate-root endpoints (#18472)
* some config responses

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added response structs

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

* add test for config/cors

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add (failing) tests

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* copy-pasta err

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* update tests for /sys/config/ui/headers/{header}

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-02-16 15:06:26 -05:00
Daniel Huckins 60488687ad
VAULT-12112: add openapi response structures for /sys/capabilities* endpoints (#18468)
* add capabilities

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added change log

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add test

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use nil for dynamic fields

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
2023-02-16 15:04:37 -05:00
Daniel Huckins a9d15f1252
VAULT-12112: add openapi response structures for `/sys/auth/*` endpoints (#18465)
* added responses to /sys/auth/.../tune

* add response structure for auth/...

* added changelog

* Update vault/logical_system_paths.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* its TypeString

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use nil for dynamic fields

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* test auth endpoint schema

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* kicking off ci

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-02-16 15:03:19 -05:00
Tom Proctor 2472029a0a
Support event subscriptions with glob wildcards (#19205) 2023-02-16 17:22:56 +00:00
Leland Ursu 0704127020
added OpenAPI response objects for sys endpoints (#18633)
* added response objects for sys 3 section

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_raw.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_quotas.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_quotas.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_quotas.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add tests and update based on reviews

* added changelog file

* finally got make fmt to work...

* fixed copy pasta test case

* updated based on review

* Update vault/logical_system_quotas.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Update vault/logical_system_test.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_test.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

---------

Co-authored-by: lursu <leland.ursu@hashicorp.com>
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-02-15 15:02:21 -05:00
Leland Ursu 6425130605
Added OpenAPI response structures for sys endpoints (#18515)
* added response objects to all of the endpoints laid out by the ticket linked

* added changelog file and updated based on review

* added the required bool to the correct fields

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* updated based on review

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* updated based on review and added test cases for validating response structures

* fix copy pasta issues breaking tests

* Update vault/logical_system_paths.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* fix test failures

* fixed issue with refrencing the wrong req var name

* fixed another test case and double checked the rest

* updated based on review

* updated in all locations

* Update vault/logical_system_paths.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* fixed my brain fart

* Update vault/logical_system_paths.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* address fmt error

---------

Co-authored-by: lursu <leland.ursu@hashicorp.com>
Co-authored-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-02-15 15:00:06 -05:00
Daniel Huckins 7fde5ecb83
Validate response schema for integration tests (#19043)
* add RequestResponseCallback to core/options

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* pass in router and apply function on requests

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add callback

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* cleanup

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Update vault/core.go

* bad typo...

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* use pvt interface, can't downcast to child struct

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* finer grained errors

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* trim path for backend

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* remove entire mount point instead of just the first part of url

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* Update vault/testing.go

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* add doc string

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* update docstring

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* reformat

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added changelog

---------

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-02-15 14:57:57 -05:00
Christopher Swenson 020a3903ff
events: Remove subscriptions on timeout and cancel (#19185)
When subscriptions are too slow to accept messages on their channels,
then we should remove them from the fanout so that we don't have
dead subscriptions using up resources.

In addition, when a subscription is explicitly canceled, we should
also clean up after it remove the corresponding pipeline.

We also add a new metrics, `events.subscriptions`, to keep track
of the number of active subscriptions. This also helps us test.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-15 11:13:15 -08:00
Christopher Swenson 7a977fd6ea
events: Check token and ACLs on request (#19138)
This checks the request against the `read` permission for
`sys/events/subscribe/{eventType}` on the initial subscribe.

Future work includes moving this to its own verb (`subscribe`)
and periodically rechecking the request.

Tested locally by minting a token with the wrong permissions
and verifying that they are rejected as expected, and that
they work if the policy is adjusted to `sys/event/subscribe/*`
(or the specific topic name) with `read` permissions.

I had to change the `core.checkToken()` to be publicly accessible,
as it seems like the easiest way to check the token on the
`logical.Request` against all relevant policies, but without
going into all of the complex logic further in `handleLogical()`.

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-10 20:56:00 +00:00
Christopher Swenson 7d3d404ee2
events: Add websockets and command (#19057)
Also updates the event receieved to include a timestamp.
Websockets support both JSON and protobuf binary formats.

This can be used by either `wscat` or the new
`vault events subscribe`:

e.g.,
```sh
$ wscat -H "X-Vault-Token: $(vault print token)" --connect ws://127.0.0.1:8200/v1/sys/events/subscribe/abc?json=true
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

and

```sh
$ vault events subscribe abc
{"event":{"id":"5c5c8c83-bf43-7da5-fe88-fc3cac814b2e", "note":"testing"}, "eventType":"abc", "timestamp":"2023-02-07T18:40:50.598408Z"}
...
```

Co-authored-by: Tom Proctor <tomhjp@users.noreply.github.com>
2023-02-09 13:18:58 -08:00
Tom Proctor 78d83c9136
Make experiments API authenticated (#18966) 2023-02-09 20:18:14 +00:00
John-Michael Faircloth 34fd57ac08
test/plugin: test external plugin workflows (#19090)
* test/plugin: test external plugin workflows

* update secrets engine test
2023-02-09 10:16:16 -06:00
Nick Cabatoff ec2af04ec6
Remove the last vestiges of sdk/version. (#19068) 2023-02-08 12:30:27 -05:00
Scott Miller acee981753
Remove accidental addition of a hackweek file (#19016) 2023-02-06 16:45:55 -06:00
Scott Miller b43e4fbd9c
Add a stronger warning about the usage of recovery keys (#19011)
* Add a stronger warning about the usage of recovery keys

* Update website/content/docs/concepts/seal.mdx

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Keep the mitigation text in the warning box

---------

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-02-06 16:23:05 -06:00
miagilepner e873932bce
VAULT-8436 remove <-time.After statements in for loops (#18818)
* replace time.After with ticker in loops

* add semgrep rule

* update to use timers

* remove stop
2023-02-06 17:49:01 +01:00
miagilepner 9d09dba7ac
VAULT-13061: Fix mount path discrepancy in activity log (#18916)
* use single function to convert mount accessor to mount path

* add changelog

* more context and comments for the tests
2023-02-06 10:26:32 +01:00
John-Michael Faircloth 14e4d67026
test/plugin: refactor compilePlugin for reuse (#18952)
* test/plugin: refactor compilePlugin for reuse

- move compilePlugin to helper package
- make NewTestCluster use compilePlugin

* do not overwrite plugin directory in CoreConfig if set

* fix getting plugin directory path for go build
2023-02-03 16:27:11 -06:00
Christopher Swenson 5864075c30
Add events sending routed from plugins (#18834)
This isn't perfect for sure, but it's solidifying and becoming a useful
base to work off.

This routes events sent from auth and secrets plugins to the main
`EventBus` in the Vault Core. Events sent from plugins are automatically
tagged with the namespace and plugin information associated with them.
2023-02-03 13:24:16 -08:00
Chris Capurso c74c057bdb
fix sys/leases panic when lease_id is nil (#18951)
* fix sys/leases panic when lease_id is nil

* add changelog entry
2023-02-03 09:51:10 -05:00
Anton Averchenkov 619f5b2edf
openapi: Ensure mount_path parameters are marked as required (#18955) 2023-02-02 14:39:09 -05:00
Chris Capurso 53390eaddf
Add ClusteName to GetClusterStatus response (#18950)
* bump github.com/hashicorp/vault/vault/hcp_link/proto

* add ClusterName to GetClusterStatus response
2023-02-02 09:27:55 -05:00
Chris Capurso c8660ca2ea
add ClusterName to meta GetClusterStatusResponse (#18944)
* add ClusterName to meta GetClusterStatusResponse

* make proto
2023-02-01 15:15:04 -05:00
Scott Miller 20551261bd
Revert #18683 (#18942)
* Revert "Don't execute the seal recovery tests on ENT. (#18841)"

This reverts commit 990d3bacc203c229d0f6729929d7562e678a1ac2.

* Revert "Add the ability to unseal using recovery keys via an explicit seal option. (#18683)"

This reverts commit 2ffe49aab0fc1a527c5182637c8fa3ac39b08d45.
2023-02-01 13:34:53 -06:00
Hamid Ghaf 6672d3753f
check for nil references (#18871)
* check for nil references

* feedback

* feedback

* address vault-issue-18849

* Revert "address vault-issue-18849"

This reverts commit 081610c02398cf7ecdfb726cd3f622f3e69eafc4.
2023-02-01 09:51:52 -05:00
Nick Cabatoff c2b222a11a
Vault test cluster helper refactorings, mostly audit related (#18928)
* Move some test helper stuff from the vault package to a new helper/testhelpers/corehelpers package.  Consolidate on a single "noop audit" implementation.
2023-02-01 08:33:16 -05:00
Anton Averchenkov 7b356f0366
openapi: Add default values to the thing_mount_path parameters (#18935) 2023-01-31 19:37:16 -05:00
Max Bowsher 9d863a92ce
Fix multiple OpenAPI generation issues with new AST-based generator (#18554)
* Regexp metacharacter `.` should be escaped when used literally

The paths including `/.well-known/` in the Vault API could currently
technically be invoked with any random character in place of the dot.

* Replace implementation of OpenAPI path translator with regexp AST-based one

* Add changelog

* Typo fix from PR review - thanks!

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>

* Add comment based on review feedback

* Change style of error handling as suggested in code review

* Make a further tweak to the handling of the error case

* Add more tests, testing cases which fail with the previous implementation

* Resolve issue with a test, and improve comment

---------

Co-authored-by: Anton Averchenkov <84287187+averche@users.noreply.github.com>
2023-01-31 16:27:39 -05:00
Hamid Ghaf 9c8fcaf5a5
prevent panic on mfa enforcement delete after a namespace is deleted (#18923)
* prevent panic on mfa enforcement delete after a namespace is deleted

* CL
2023-01-31 21:06:16 +00:00
Chris Capurso 6cb6157d37
return 403 for wrapping requests when no token provided (#18859)
* return 403 for wrapping requests when no token provided

* add changelog entry

* fix changelog

* use errors.As

* simplify error response string
2023-01-31 13:57:50 -05:00
Nick Cabatoff db41c5319d
TestClusterCore's TLSConfig becomes a method and does a Clone. (#18914) 2023-01-31 11:05:16 -05:00
akshya96 16ce923ddb
Brute forcing unlock user bug (#18890)
* brute forcing unlock user bug

* add changelog

* fix changelog
2023-01-30 13:06:10 -08:00
Kit Haines 5ece71109a
Vault 11798 vault cli issue intermediate (#18467)
* The verify-sign command in it's cleanest existing form.

* Working state

* Updates to proper verification syntax

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>

* make fmt

* Git CI caught some stuff.

* Base functionality.

* make fmt; changelog

* pki issue command.

* Make fmt. Changelog.

* Error Handling Is Almost A Tutorial

* What I thought empty issuers response fix would be.

* Some tests

* PR-review updates.

* make fmt.

* Fix null response data for listing empty issuers causing a crash.

* Update command/pki_list_children_command.go

Fix double specifier

Co-authored-by: Steven Clark <steven.clark@hashicorp.com>

* Add test for pki_list_children.

* Fix tests.

* Update descriptions for correctness based on PR reviews.

* make fmt.

* Updates based on PR feedback.

* Allow multiple arguements (space separated)

* Remove bad merge-thing.

* White-space hell fix change.

* Tests, and return information for issue ca

* Fix make fmt error introduced here: https://github.com/hashicorp/vault/pull/18876

* Update command/pki_issue_intermediate.go

Puncutation.

Co-authored-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Remove smart quotes for standard quotes.

* More information as part of the help text.

* Better help text.

* Add missing "/" into error message.

---------

Co-authored-by: 'Alex Scheel' <alex.scheel@hashicorp.com>
Co-authored-by: Steven Clark <steven.clark@hashicorp.com>
2023-01-27 16:41:16 -05:00
Jason O'Donnell e14f0216a0
Add timeout functionality to inmem (#18876)
* Add timeout functionality to inmem

* Update vault/cluster/inmem_layer.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Add comment about forceTimeout

* Add comment about time

---------

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-27 16:46:12 +00:00
Scott Miller af7df08921
Don't execute the seal recovery tests on ENT. (#18841)
* Don't execute the seal recovery tests on OSS.

* correct go:build syntax
2023-01-25 15:35:15 -06:00
Hamid Ghaf 2b5e5121e1
Provide IP Address in Duo Request (#18811)
* Provide IP Address in Duo Request

* CL
2023-01-24 17:28:59 -05:00
Scott Miller 25960fd034
Add the ability to unseal using recovery keys via an explicit seal option. (#18683)
* wip

* wip

* Got it 'working', but not happy about cleanliness yet

* Switch to a dedicated defaultSeal with recovery keys

This is simpler than trying to hijack SealAccess as before.  Instead, if the operator
has requested recovery unseal mode (via a flag in the seal stanza), we new up a shamir
seal with the recovery unseal key path instead of the auto seal.  Then everything proceeds
as if you had a shamir seal to begin with.

* Handle recovery rekeying

* changelog

* Revert go.mod redirect

* revert multi-blob info

* Dumb nil unmarshal target

* More comments

* Update vault/seal.go

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* pr feedback

* Fix recovery rekey, which needs to fetch root keys and restore them under the new recovery split

* Better comment on recovery seal during adjustSealMigration

* Make it possible to migrate from an auto-seal in recovery mode to shamir

* Fix sealMigrated to account for a recovery seal

* comments

* Update changelog/18683.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* Address PR feedback

* Refactor duplicated migration code into helpers, using UnsealRecoveryKey/RecoveryKey where appropriate

* Don't shortcut the reast of seal migration

* get rid of redundant transit server cleanup

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-24 14:57:56 -06:00
Hamid Ghaf edbf093290
Allow Token Create Requests To Be Replicated (#18689)
* Allow Token Create Requests To Be Replicated

* adding a test

* revert a test
2023-01-24 14:00:27 -05:00
Hamid Ghaf 65a41d4f08
named Login MFA methods (#18610)
* named MFA method configurations

* fix a test

* CL

* fix an issue with same config name different ID and add a test

* feedback

* feedback on test

* consistent use of passcode for all MFA methods (#18611)

* make use of passcode factor consistent for all MFA types

* improved type for MFA factors

* add method name to login CLI

* minor refactoring

* only accept MFA method name with its namespace path in the login request MFA header

* fix a bug

* fixing an ErrorOrNil return value

* more informative error message

* Apply suggestions from code review

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

* feedback

* test refactor a bit

* adding godoc for a test

* feedback

* remove sanitize method name

* guard a possbile nil ref

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2023-01-23 15:51:22 -05:00
Tom Proctor fc378c0908
Event system alpha experiment (#18795) 2023-01-23 19:26:49 +00:00
Chris Capurso 0eedcd979b
Fix link meta panics (#18774)
* return error for meta auth and mount listing if sealed

* some logging changes

* some more logging changes

* add panic recovery

* use ErrInternalError
2023-01-23 09:59:15 -05:00
Alexander Scheel b3dc380c82
Add cross-cluster revocation queues for PKI (#18784)
* Add global, cross-cluster revocation queue to PKI

This adds a global, cross-cluster replicated revocation queue, allowing
operators to revoke certificates by serial number across any cluster. We
don't support revoking with private key (PoP) in the initial
implementation.

In particular, building on the PBPWF work, we add a special storage
location for handling non-local revocations which gets replicated up to
the active, primary cluster node and back down to all secondary PR
clusters. These then check the pending revocation entry and revoke the
serial locally if it exists, writing a cross-cluster confirmation entry.

Listing capabilities are present under pki/certs/revocation-queue,
allowing operators to see which certs are present. However, a future
improvement to the tidy subsystem will allow automatic cleanup of stale
entries.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow tidying revocation queue entries

No manual operator control of revocation queue entries are allowed.
However, entries are stored with their request time, allowing tidy to,
after a suitable safety buffer, remove these unconfirmed and presumably
invalid requests.

Notably, when a cluster goes offline, it will be unable to process
cross-cluster revocations for certificates it holds. If tidy runs,
potentially valid revocations may be removed. However, it is up to the
administrator to ensure the tidy window is sufficiently long that any
required maintenance is done (or, prior to maintenance when an issue is
first noticed, tidy is temporarily disabled).

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Only allow enabling global revocation queue on Vault Enterprise

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Use a locking queue to handle revocation requests

This queue attempts to guarantee that PKI's invalidateFunc won't have
to wait long to execute: by locking only around access to the queue
proper, and internally using a list, we minimize the time spent locked,
waiting for queue accesses.

Previously, we held a lock during tidy and processing that would've
prevented us from processing invalidateFunc calls.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* use_global_queue->cross_cluster_revocation

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Grab revocation storage lock when processing queue

We need to grab the storage lock as we'll actively be revoking new
certificates in the revocation queue. This ensures nobody else is
competing for storage access, across periodic funcs, new revocations,
and tidy operations.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Fix expected tidy status test

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Allow probing RollbackManager directly in tests

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Address review feedback on revocationQueue

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add more cancel checks, fix starting manual tidy

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-23 09:29:27 -05:00
Alexander Scheel 6930568076
Add path based primary write forwarding (PBPWF) - OSS (#18735)
* Add WriteForwardedStorage to sdk's plugin, logical in OSS

This should allow backends to specify paths to forward write
(storage.Put(...) and storage.Delete(...)) operations for.

Notably, these semantics are subject to change and shouldn't yet be
relied on.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Collect paths for write forwarding in OSS

This adds a path manager to Core, allowing tracking across all Vault
versions of paths which could use write forwarding if available. In
particular, even on OSS offerings, we'll need to template {{clusterId}}
into the paths, in the event of later upgrading to Enterprise. If we
didn't, we'd end up writing paths which will no longer be accessible
post-migration, due to write forwarding now replacing the sentinel with
the actual cluster identifier.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add forwarded writer implementation to OSS

Here, for paths given to us, we determine if we need to do cluster
translation and perform local writing. This is the OSS variant.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Wire up mount-specific request forwarding in OSS

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Clarify that state lock needs to be held to call HAState in OSS

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Move cluster sentinel constant to sdk/logical

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Expose ClusterID to Plugins via SystemView

This will let plugins learn what the Cluster's ID is, without having to
resort to hacks like writing a random string to its cluster-prefixed
namespace and then reading it once it has replicated.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

* Add GRPC ClusterID implementation

For any external plugins which wish to use it.

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>

Signed-off-by: Alexander Scheel <alex.scheel@hashicorp.com>
2023-01-20 16:36:18 -05:00
Christopher Swenson 4a93097895
Use schema for events in event broker (#18693)
For the new events schema. Based on the CloudEvents schema.
2023-01-20 10:18:23 -08:00
Daniel Huckins fc6d13e29d
VAULT-12112: openapi response definitions: sys/audit (#18456)
* added audit-hash operations

* more audit paths

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* added audit fields

* add changelog file

* dynamic fields should be nil

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* start to add test helper

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>

* add tests for /sys/audit openapi paths

Signed-off-by: Daniel Huckins <dhuckins@users.noreply.github.com>
Co-authored-by: Anton Averchenkov <anton.averchenkov@hashicorp.com>
2023-01-20 11:09:33 -05:00