Commit graph

2493 commits

Author SHA1 Message Date
Brian Kassouf 0e1b2c1b7b
Fix some more error shadowing issues (#12990) 2021-11-01 11:43:00 -07:00
John-Michael Faircloth 792400a24c
ensure errors are checked (#12989) 2021-11-01 13:34:51 -05:00
Austin Gebauer 83dcbc0cd5
Deny access to UserInfo endpoint if client no longer allowed by provider (#12949) 2021-10-28 09:46:07 -07:00
Austin Gebauer e5b45b5416
Adds check to ensure authorization code can only be exchanged by the issuing provider (#12948) 2021-10-28 09:45:41 -07:00
Brian Kassouf 350ecd6bc5
Fix leak of restore worker go routines after an error (#12933) 2021-10-27 09:08:10 -07:00
Austin Gebauer b3fab954fe
Use mutex in OIDC configuration handlers (#12932) 2021-10-27 08:23:05 -07:00
swayne275 92e6972b86
oss LastDRWAL (#12931) 2021-10-26 15:17:20 -06:00
Hridoy Roy 1d29b93eb5
fix adding clientID to request in audit log (#12919)
* fix adding clientID to request in audit log

* fix boolean statement

* use standard encoding for client ID instead of urlEncoding

* change encoding in tests

* add in client counts to request handling

* remove redundant client ID generation in request handling

* directly add clientID to req after handling token usage
2021-10-26 09:02:13 -07:00
Austin Gebauer a25cfe822e
Adds missing unlock of RWMutex in OIDC delete key (#12916) 2021-10-25 09:59:26 -07:00
swayne275 95e5cdd000
VAULT 2844: remove legacy lease revocation strategy (#12888)
* remove legacy lease revocation strategy

* add deprecation change log note

* remove VAULT_16_REVOKE_PERMITPOOL

* update changelog
2021-10-22 17:37:01 -06:00
hghaf099 442b5920e2
Entities may have duplicate policies (#12812)
* Entities may have duplicate plicies

* Adding changelog

* removing duplicates on reading entity policies

* fix changelog
2021-10-22 19:28:31 -04:00
Nick Cabatoff ff74f49047
Move to go 1.17 (#12868)
Also ensure that the go 1.17 breaking changes to net.ParseCIDR don't make us choke on stored CIDRs that were acceptable to older Go versions.
2021-10-21 09:32:03 -04:00
Vishal Nayak 6eead9f09b
Fix entity alias deletion (#12834)
* Fix entity alias deletion

* Fix tests

* Add CL
2021-10-19 15:05:06 -04:00
Brian Kassouf c705adc79c
Fix some linting errors (#12860) 2021-10-18 17:29:47 -07:00
Austin Gebauer 4e5b865c4f
Rename scopes to scopes_supported for OIDC providers (#12851) 2021-10-15 19:33:32 -07:00
Hridoy Roy 19822781cc
use IsRoot helper before clientID generation in activity log (#12846)
* use IsRoot helper in activity log

* use IsRoot helper in activity log
2021-10-15 14:04:55 -07:00
Vishal Nayak 476fb08e0d
Local aliases OSS patch (#12848)
* Local aliases OSS patch

* build fix
2021-10-15 15:20:00 -04:00
saltperfect 66369469d9
Removed unpublished:true for sys/internal/* endpoints (#12713)
* removed unpublished:true for sys/internal/* endpoints

* added changelog file

* updated change log and added placeholder summary as these endpoints are not mentioned in docs.

* added documentation for internal/ui/namspaces and resultant-acl

* updated log configs
2021-10-15 14:50:14 -04:00
Pratyoy Mukhopadhyay 148109b8ed
[VAULT-3252] Disallow alias creation if entity/accessor combination exists (#12747)
* Disallow alias creation if entity/accessor combination exists

* Add changelog

* Address review comments

* Add handling to aliasUpdate, some field renaming

* Update tests to work under new entity-alias constraint

* Add check to entity merge, other review fixes

* Log duplicated accessors only once

* Fix flaky test

* Add note about new constraint to docs

* Update entity merge warn log
2021-10-14 09:52:07 -07:00
Hridoy Roy 1c427d3286
Port: add client ID to TWEs in activity log [vault-3136] (#12820)
* port for tracking twes as clients

* comment clean up

* changelog

* change changelog entry phrasing
2021-10-14 09:10:59 -07:00
Austin Gebauer 0551f91068
Adds OIDC Token and UserInfo endpoints (#12711) 2021-10-13 18:59:36 -07:00
swayne275 76556c867b
port relevant ent changes (#12823) 2021-10-13 18:15:54 -06:00
Chris Capurso bbb4ab4a41
Add HTTP PATCH support to KV (#12687)
* handle HTTP PATCH requests as logical.PatchOperation

* update go.mod, go.sum

* a nil response for logical.PatchOperation should result in 404

* respond with 415 for incorrect MIME type in PATCH Content-Type header

* add abstraction to handle PatchOperation requests

* add ACLs for patch

* Adding JSON Merge support to the API client

* add HTTP PATCH tests to check high level response logic

* add permission-based 'kv patch' tests in prep to add HTTP PATCH

* adding more 'kv patch' CLI command tests

* fix TestHandler_Patch_NotFound

* Fix TestKvPatchCommand_StdinValue

* add audit log test for HTTP PATCH

* patch CLI changes

* add patch CLI tests

* change JSONMergePatch func to accept a ctx

* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag

* go fmt

* add a test to verify patching works by default with the root token

* add changelog entry

* get vault-plugin-secrets-kv@add-patch-support

* PR feedback

* reorder some imports; go fmt

* add doc comment for HandlePatchOperation

* add json-patch@v5.5.0 to go.mod

* remove unnecessary cancelFunc for WriteBytes

* remove default for -method

* use stable version of json-patch; go mod tidy

* more PR feedback

* temp go get vault-plugin-secrets-kv@master until official release

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
2021-10-13 15:24:31 -04:00
John-Michael Faircloth 2abf916ddb
Add support to parameterize unauthenticated paths (#12668)
* store unauthenticated path wildcards in map

* working unauthenticated paths with basic unit tests

* refactor wildcard logic

* add parseUnauthenticatedPaths unit tests

* use parseUnauthenticatedPaths when reloading backend

* add more wildcard test cases

* update special paths doc; add changelog

* remove buggy prefix check; add test cases

* prevent false positives for prefix matches

If we ever encounter a mismatched segment, break and set a flag to
prevent false positives for prefix matches.

If it is a match we need to do a prefix check. But we should not return
unless HasPrefix also evaluates to true. Otherwise we should let the for
loop continue to check other possibilities and only return false once
all wildcard paths have been evaluated.

* refactor switch and add more test cases

* remove comment leftover from debug session

* add more wildcard path validation and test cases

* update changelong; feature -> improvement

* simplify wildcard segment matching logic

* refactor wildcard matching into func

* fix glob matching, add more wildcard validation, refactor

* refactor common wildcard errors to func

* move doc comment to logical.Paths

* optimize wildcard paths storage with pre-split slices

* fix comment typo

* fix test case after changing wildcard paths storage type

* move prefix check to parseUnauthenticatedPaths

* tweak regex, remove unneeded array copy, refactor

* add test case around wildcard and glob matching
2021-10-13 11:51:20 -05:00
hghaf099 ad2ef412cc
Customizing HTTP headers in the config file (#12485)
* Customizing HTTP headers in the config file

* Add changelog, fix bad imports

* fixing some bugs

* fixing interaction of custom headers and /ui

* Defining a member in core to set custom response headers

* missing additional file

* Some refactoring

* Adding automated tests for the feature

* Changing some error messages based on some recommendations

* Incorporating custom response headers struct into the request context

* removing some unused references

* fixing a test

* changing some error messages, removing a default header value from /ui

* fixing a test

* wrapping ResponseWriter to set the custom headers

* adding a new test

* some cleanup

* removing some extra lines

* Addressing comments

* fixing some agent tests

* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before

* Removing default custom headers, and renaming some function varibles

* some refacotring

* Refactoring and addressing comments

* removing a function and fixing comments
2021-10-13 11:06:33 -04:00
Victor Rodriguez ae40d1c2c9
Wait for expiration manager to be out of restore mode while testing. (#12779) 2021-10-13 08:58:02 -04:00
Matt Greenfield 0b3eea4441
Dedup from_entity_ids when merging two entities (#10101)
Fixes #10100
2021-10-12 15:35:19 -04:00
Pratyoy Mukhopadhyay c52ff27e9d
Update naptime for retrytoken test (#12799) 2021-10-12 11:12:06 -07:00
John-Michael Faircloth f30c3ac621
Filter identity token keys (#12780)
* filter identity token keys

* Update test cases to associate keys with roles

* use getOIDCRole helper

* add func comment and test assertion

* add changelog

* remove unnecessary code

* build list of keys to return by starting with a list of roles

* move comment

* update changelog
2021-10-12 11:14:03 -05:00
swayne275 2edac287ae
update function signature and call (#12806) 2021-10-11 18:21:38 -06:00
John-Michael Faircloth e72c15426b
OIDC: Unique prefix to client_secret (#12745)
* Unique prefix to client_id and client_secret

* only prefix client secret and use constants

* split client secret checks
2021-10-08 16:02:20 -05:00
Scott Miller 1097f356af
Add a TRACE log with TLS connection details on replication connections (#12754)
* remove cruft
use helper
Add a helper for getting public key sizes
wip

* error names

* Fix ecdsa

* only if trace is on

* Log listener side as well

* rename

* Add remote address

* Make the log level configurable via the env var, and a member of the Listener and thus modifiable by tests

* Fix certutil_test
2021-10-07 14:17:31 -05:00
Austin Gebauer e09657e1f3
Adds ability to define an inline policy and internal metadata on tokens (#12682)
* Adds ability to define an inline policy and internal metadata to tokens

* Update comment on fetchEntityAndDerivedPolicies

* Simplify handling of inline policy

* Update comment on InternalMeta

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Improve argument name

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Use explicit SkipIdentityInheritance token field instead of implicit InlinePolicy behavior

* Add SkipIdentityInheritance to pb struct in token store create method

* Rename SkipIdentityInheritance to NoIdentityPolicies

* Merge latest from main and make proto

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-10-07 10:36:22 -07:00
swayne275 b9fde1dd6f
oss port (#12755) 2021-10-07 11:25:16 -06:00
vinay-gopalan 458927c2ed
[VAULT-3157] Move mergeStates utils from Agent to api module (#12731)
* move merge and compare states to vault core

* move MergeState, CompareStates and ParseRequiredStates to api package

* fix merge state reference in API Proxy

* move mergeStates test to api package

* add changelog

* ghost commit to trigger CI

* rename CompareStates to CompareReplicationStates

* rename MergeStates and make compareStates and parseStates private methods

* improved error messaging in parseReplicationState

* export ParseReplicationState for enterprise files
2021-10-06 10:57:06 -07:00
hghaf099 a3796997d9
Fix a Deadlock on HA leadership transfer (#12691)
* Fix a Deadlock on HA leadership transfer when standby
was actively forwarding a request
fixes GH #12601

* adding the changelog
2021-10-04 13:55:15 -04:00
vinay-gopalan 680e8515fa
[VAULT-3472] Cap Client id_token_ttl field to associated Key's verification_ttl (#12677) 2021-10-01 10:47:40 -07:00
Matt Greenfield 8577602395
Fix entity group associations (#10085)
- When two entities are merged, remove the from entity ID in any
  associated groups.
- When two entities are merged, also merge their associated group
  memberships.

Fixes #10084
2021-10-01 10:22:52 -04:00
Brian Kassouf 39a9727c8b
Update protobuf & grpc libraries and protoc plugins (#12679) 2021-09-29 18:25:15 -07:00
Tero Saarni 944332d12d
Update Go client libraries for etcd (#11980)
* Update Go client libraries for etcd

* Added etcd server container to run etcd3 tests automatically.

* Removed etcd2 test case: it fails the backend tests but the failure is
  unrelated to the uplift.  The etcd2 backend implementation does not
  remove empty nested nodes when removing leaf (see comments in #11980).
2021-09-29 14:28:13 -04:00
swayne275 af3136d393
move from static sleep to timed loop waiting for lease revocation (#12627) 2021-09-27 14:23:17 -06:00
Pratyoy Mukhopadhyay 73ae1fcfff
Update flaky expiration test (#12628)
* Update flaky expiration test

* Remove extraneous comment

Co-authored-by: swayne275 <swayne275@gmail.com>

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-09-27 11:51:27 -07:00
Austin Gebauer b58913ad9f
Adds OIDC Authorization Endpoint to OIDC providers (#12538) 2021-09-27 10:55:29 -07:00
Austin Gebauer da394f34b1
Adds additional OIDC discovery metadata (#12623) 2021-09-27 10:05:55 -07:00
Austin Gebauer 93f8d248d3
Use durations for ID and access token TTLs (#12632) 2021-09-27 10:05:28 -07:00
Calvin Leung Huang 7ad62f5be4
core: set namespace within GeneratePasswordFromPolicy (#12635)
* core: set namespace from the sysview's mount entry on GeneratePasswordFromPolicy

* test: update TestDynamicSystemView to be ns-aware, update tests

* add changelog entry
2021-09-27 09:08:07 -07:00
Pratyoy Mukhopadhyay 8e6698fb4a
[VAULT-3519] Return no_default_policy on token role read (#12565)
* [VAULT-3519] Return no_default_policy on token role read if set

* [VAULT-3519] Add changelog

* [VAULT-3519] Always return token_no_default_policy on role read

* Fix broken test

* Update role read response in docs
2021-09-21 09:53:08 -07:00
Tiernan a538936367
Allow globbing dis/allowed_policies_glob in token roles (#7277)
* Add allowed_policies_glob and disallowed_policies_glob that are the same as allowed_policies and disallowed_policies but allow glob matching.

* Update changelog, docs, tests, and comments for (dis)allowed_token_glob token role feature.

* Improve docs and unit tests for auth/token role policy globbing.
2021-09-21 08:25:06 -07:00
Nick Cabatoff 2bd95232cf
Fail alias rename if the resulting (name,accessor) exists already (#12473) 2021-09-21 08:19:44 -04:00
akshya96 c643dc1d53
Add Custom metadata field to alias (#12502)
* adding changes

* removing q.Q

* removing empty lines

* testing

* checking tests

* fixing tests

* adding changes

* added requested changes

* added requested changes

* added policy templating changes and fixed tests

* adding proto changes

* making changes

* adding unit tests

* using suggested function
2021-09-17 11:03:47 -07:00
John-Michael Faircloth bee9f25277
OIDC provider: show success message in CLI (#12574) 2021-09-17 11:41:08 -05:00
Michael Ward c15ac1053f
Expose secret_id_accessor as WrappedAccessor when wrapping secret-id creation. (#12425)
* Expose secret_id_accessor as WrappedAccessor when wrapping secret-id creation.

* Add changelog.

* Minor updates as suggested.

* Adding external test for wrapped accessor.

* Add check that mounttype is approle.

* Update changelog text to use improvement
2021-09-16 10:47:49 -07:00
John-Michael Faircloth 4ed0eb4493
identity: fix bug and increase logging for jwks cache control max age test (#12561)
* identity: increase logging for jwks cache control max age test

* clarify comment

* add more logging for jwks cache control max age in test
2021-09-15 14:38:29 -05:00
John-Michael Faircloth c42c9993a0
feature: OIDC keys endpoint (#12525)
* add keys path and initial handler

* read provider public keys

* add test cases

* remove some debug logs

* update tests after merging main

* refactor list all clients

* refactor logic to collect Key IDs
2021-09-14 15:37:53 -05:00
hghaf099 2576be23d0
Fixing a flaky test TestQuotas_RateLimitQuota_ExemptPaths (#12532)
* Fixing a flaky test TestQuotas_RateLimitQuota_ExemptPaths

* fixing a string formatting by removing an extra parameter
2021-09-10 17:41:14 -04:00
John-Michael Faircloth c42bbb369c
Identity: prepublish jwt signing keys (#12414)
* pre-publish new signing keys for `rotation_period` of time before using

* Work In Progress: Prepublish JWKS and even cache control

* remove comments

* use math/rand instead of math/big

* update tests

* remove debug comment

* refactor cache control logic into func

* don't set expiry when create/update key

* update cachecontrol name in oidccache for test

* fix bug in periodicfunc test case

* add changelog

* remove confusing comment

* add logging and comments

* update change log from bug to improvement

Co-authored-by: Ian Ferguson <ian.ferguson@datadoghq.com>
2021-09-09 13:47:42 -05:00
John-Michael Faircloth 0d04a9892a
identity: enforce key param and key existence on role creation (#12208)
* identity: handle creation of role without a key parameter

* update docs to not require key parameter for creation of a role

* add changelog

* require key param when creating a role

* lock create/update role; remove now redundant key check

* update changelog and UTs

* update change log to refelct actual implementation

* remove deprecated test case
2021-09-08 10:46:58 -05:00
Daniel Kimsey b4b61efc75
Auto-join support for IPv6 discovery (#12366)
* Auto-join support for IPv6 discovery

The go-discover library returns IP addresses and not URLs. It just so
happens net.URL parses "127.0.0.1", which isn't a valid URL.

Instead, we construct the URL ourselves. Being careful to check if it's
an ipv6 address and making sure it's in explicit form if so.

Fixes #12323

* feedback: addrs & ipv6 test

Rename addrs to clusterIPs to improve clarity and intent

Tighten up our IPv6 address detection to be more correct and to ensure
it's actually in implicit form
2021-09-07 11:55:07 -07:00
John-Michael Faircloth 01011973a3
feature: OIDC discovery endpoint (#12481)
* OIDC Provider: implement discovery endpoint

* handle case when provider does not exist

* refactor providerDiscover struct and add scopes_supported

* fix authz endpoint
2021-09-07 13:35:23 -05:00
akshya96 f4bd14ed3f
Vault 2823 cc namespace (#12393)
* vault-2823 adding changes

* VAULT-2823 adding alias

* Vault-2823 addressing comments

* Vault-2823 removing comments

* Vault-2823 removing comments

* vault-2823 removing q debug

* adding changelog

* Vault-2823 updating external test

* adding approved changes

* fixing returns

* fixing returns
2021-09-07 09:16:12 -07:00
Nick Cabatoff 45a83d8e0f
Add code to api.RaftSnapshot to detect incomplete snapshots (#12388) 2021-09-07 11:16:37 -04:00
Scott Miller 8b20cae44f
Make necessary OSS changes for contexts in quotas (#12480)
* Make necessary OSS changes for contexts in quotas

* missed one
2021-09-02 09:54:21 -05:00
Scott Miller 0f6543fb41
Upgrade go-limiter to fix building on 1.17 (#12358)
* Upgrade go-limiter

* Modify quota system to pass contexts to upgraded go-limiter

* One more spot

* Add context vars to unit tests

* missed one
2021-09-01 16:28:47 -05:00
vinay-gopalan c99cf35b6a
[VAULT-3347] Ensure Deduplication in Provider and Client APIs in OIDC Provider (#12460)
* add deduplication for Provider

* add deduplication to provider client API

* add changelog

* delete changelog

* update comments

* update test names
2021-08-30 13:57:28 -07:00
Nick Cabatoff 0762f9003d
Refactor usages of Core in IdentityStore so they can be decoupled. (#12461) 2021-08-30 15:31:11 -04:00
vinay-gopalan 6338f3dc62
feature: OIDC Provider API (#12279) 2021-08-26 10:13:51 -07:00
John-Michael Faircloth 924cab3b06
OIDC Client API: add more test coverage (#12392)
* initial commit

* add read and delete operations

* fix bug in delete and add list unit test

* func doc typo fix

* add existence check for assignment

* remove locking on the assignment resource

It is not needed at this time.

* convert Callbacks to Operations

- convert Callbacks to Operations
- add test case for update operations

* add CRUD operations and test cases

* add client api and tests

* remove use of oidcCache

* remove use of oidcCache

* add template validation and update tests

* remove usage of oidcCache

* refactor struct and var names

* harmonize test name conventions

* refactor struct and var names

* add changelog and refactor

- add changelog
- be more explicit in the case where we do not recieve a path field

* refactor

be more explicit in the case where a field is not provided

* remove extra period from changelog

* update scope path to be OIDC provider specific

* refactor naming conventions

* update assignment path

* update scope path

* enforce key existence on client creation

* removed unused name field

* removed unused name field

* removed unused name field

* prevent assignment deletion when ref'ed by a client

* enfoce assignment existence on client create/update

* update scope template description

* error when attempting to created scope with openid reserved name

* fix UT failures after requiring assignment existence

* disallow key deletion when ref'ed by existing client

* generate client_id and client_secret on CreateOp

* do not allow key modification on client update

* return client_id and client_secret on read ops

* small refactor

* fix bug in delete assignment op

* remove client secret get call

* OIDC Client API: add more test coverage

* change name convention in tests
2021-08-23 14:08:03 -05:00
John-Michael Faircloth fec8e8b21a
feature: OIDC provider client API (#12272)
* initial commit

* add read and delete operations

* fix bug in delete and add list unit test

* func doc typo fix

* add existence check for assignment

* remove locking on the assignment resource

It is not needed at this time.

* convert Callbacks to Operations

- convert Callbacks to Operations
- add test case for update operations

* add CRUD operations and test cases

* add client api and tests

* remove use of oidcCache

* remove use of oidcCache

* add template validation and update tests

* remove usage of oidcCache

* refactor struct and var names

* harmonize test name conventions

* refactor struct and var names

* add changelog and refactor

- add changelog
- be more explicit in the case where we do not recieve a path field

* refactor

be more explicit in the case where a field is not provided

* remove extra period from changelog

* update scope path to be OIDC provider specific

* refactor naming conventions

* update assignment path

* update scope path

* enforce key existence on client creation

* removed unused name field

* removed unused name field

* removed unused name field

* prevent assignment deletion when ref'ed by a client

* enfoce assignment existence on client create/update

* update scope template description

* error when attempting to created scope with openid reserved name

* fix UT failures after requiring assignment existence

* disallow key deletion when ref'ed by existing client

* generate client_id and client_secret on CreateOp

* do not allow key modification on client update

* return client_id and client_secret on read ops

* small refactor

* fix bug in delete assignment op

* remove client secret get call
2021-08-23 08:42:31 -05:00
John-Michael Faircloth 2b3da872b3
feature: OIDC provider scope API (#12266)
* initial commit

* add read and delete operations

* fix bug in delete and add list unit test

* func doc typo fix

* add existence check for assignment

* remove locking on the assignment resource

It is not needed at this time.

* convert Callbacks to Operations

- convert Callbacks to Operations
- add test case for update operations

* add CRUD operations and test cases

* remove use of oidcCache

* remove use of oidcCache

* add template validation and update tests

* refactor struct and var names

* harmonize test name conventions

* refactor struct and var names

* add changelog and refactor

- add changelog
- be more explicit in the case where we do not recieve a path field

* refactor

be more explicit in the case where a field is not provided

* remove extra period from changelog

* update scope path to be OIDC provider specific

* update assignment path

* update scope path

* removed unused name field

* removed unused name field

* update scope template description

* error when attempting to created scope with openid reserved name
2021-08-18 13:20:27 -05:00
Hridoy Roy 2554563268
(OSS Port) Restrict Quota Deletion to Primary Cluster [vault-2399] (#12339)
* oss part of vault 2399

* Update vault/quotas/quotas.go

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* use OSS PR number as changelog entry as indicated by the changelog guide

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2021-08-17 15:34:43 -07:00
John-Michael Faircloth 40fd60342a
feature: OIDC provider assignment API (#12198)
* initial commit

* add read and delete operations

* fix bug in delete and add list unit test

* func doc typo fix

* add existence check for assignment

* remove locking on the assignment resource

It is not needed at this time.

* convert Callbacks to Operations

- convert Callbacks to Operations
- add test case for update operations

* remove use of oidcCache

* refactor struct and var names

* harmonize test name conventions

* add changelog and refactor

- add changelog
- be more explicit in the case where we do not recieve a path field

* remove extra period from changelog

* update assignment path

* removed unused name field
2021-08-17 15:55:06 -05:00
Lars Lehtonen 3bb1c68996
vault: fix dropped test errors (#12242)
* vault: fix dropped test error

* vault/activity: fix dropped test error
2021-08-16 09:52:08 -04:00
Pratyoy Mukhopadhyay 691a43030d
[VAULT-2822] Sort client count response (#12308) 2021-08-12 09:35:15 -07:00
hghaf099 f885d97774
VAULT-2285 adding capability to accept comma separated entries for au… (#12126)
* VAULT-2285 adding capability to accept comma separated entries for auth enable/tune

* Adding changelog

* Adding logic to detect invalid input parameter for auth enable config

* Updating tune.mdx

* Updating secret enable/tune for comma separated parameters

* Adding further parameter checks for auth/secret tests
Fixing changelog
using builtin type for a switch statement
Fixing a possible panic scenario

* Changing a function name, using deep.Equal instead of what reflect package provides

* Fixing auth/secret enable/tune mdx files

* One more mdx file fix

* Only when users provide a single comma separated string in a curl command, split the entries by commas

* Fixing API docs for auth/mount enable/tune for comma separated entries

* updating docs, removing an unnecessary switch case
2021-08-09 15:37:03 -04:00
John-Michael Faircloth 0d94a6530f
identity: allow creating a role with a non-existent key (#12251)
* identity: allow creating a role with a non-existent key

* remove whitespace

* add changelog
2021-08-04 11:01:13 -07:00
Hridoy Roy eaa88155d1
change advice to be used with spot warning (#12237) 2021-08-02 14:41:12 -07:00
Hridoy Roy a3fefdca35
oss part of license diagnose test fix (#12234)
* oss part of license diagnose test fix

* cl
2021-08-02 10:50:49 -07:00
Hridoy Roy 28f33d2384
Fix Diagnose Formatting In Disk Usage Checks (#12229)
* save

* fix diagnose formatting errors

* fix diagnose formatting errors

* change powers

* change powers

* use humanize instead of doing the conversion to mb manually

* cl
2021-08-02 10:06:04 -07:00
Pratyoy Mukhopadhyay 113b6885c3
[VAULT-2852] deprecate req counters in oss (#12197) 2021-07-29 10:21:40 -07:00
John-Michael Faircloth 39c744ca4e
identity: do not allow a role's token_ttl to be longer than verification_ttl (#12151)
* do not allow token_ttl to be longer than verification_ttl

* add verification when updating an existing key

When updating a key, ensure any roles referencing the key do not already
have a token_ttl greater than the key's verification_ttl

* add changelog

* remove unneeded UT check and comment

* refactor based on PR comments

- remove make slice in favor of var delcaration
- remove unneeded if check
- validate expiry value during token generation
- update changelog as bug

* refactor get roles referencing target key names logic

* add note about thread safety to helper func

* update func comment

* sort array and refactor func names

* add warning to return response

* remove unnecessary code from unit test

* Update vault/identity_store_oidc.go

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
2021-07-28 20:34:52 -05:00
Pratyoy Mukhopadhyay fa29e780f0
[NO-TICKET] Upgrade protoc-gen-go to 1.26, upgrade protoc to 3.17.3 (#12171)
* [NO-TICKET] Set protoc-gen-go to 1.23, upgrade protoc to 3.17.3

* [NO-TICKET] Upgrade version of protoc-gen-go to 1.26
2021-07-28 14:51:36 -07:00
Pratyoy Mukhopadhyay 11a3c3d72d
[VAULT-2807] Count entity tokens on use (#12153) 2021-07-22 14:01:49 -07:00
Romain Aviolat c76a4e8bd1
fix: typo in function documentation (#11852) 2021-07-21 13:21:45 -07:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090)
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
Scott Miller 555eb2ae0b
Add advice as a trace option to spot checks (#12105)
* Add advice as a trace option to spot checks

* typo

* Collect advice when forming the tree
2021-07-15 18:01:51 -05:00
Nick Cabatoff f027a1b1ff
Revert #12061 due to failures in TestLogical_RequestSizeLimit (#12093) 2021-07-15 12:55:09 -04:00
hghaf099 f7635ec1b8
Add namespace in error (#12061)
* hghaf099-VAULT-1303-Adding namespace in error when it is set

* casting ResponseWriter in handleMonitor to logical.NamespaceResponseWriter

* Casting ResponseWriter conditionally for http.Flusher
Adding changelog

* Improving changlog message
2021-07-14 15:55:55 -04:00
Scott Miller 1aa98e81f7
Make any section skippable via string contains skip checking. (#12055)
* Make any section skippable via string contains skip checking

* Fix unit test
2021-07-13 18:25:04 -05:00
Hridoy Roy d3034f2851
Diagnose Language Pass (#11909)
* save

* save

* save

* first round of the diagnose language pass

* capitalization

* first round of feedback

* fix bug in advise

* a few more nouns to verbs
2021-07-11 15:44:19 -07:00
Chris Capurso 349378454a
[VAULT-1836] Support kv-v1 generic mounts for vault.kv.secret.count metric (#12020)
* [VAULT-1836] Support kv-v1 generic mounts for vault.kv.secret.count metric

* [VAULT-1836] Add changelog entry
2021-07-09 11:05:05 -04:00
Vishal Nayak eecb39a57f
OSS parts of Autopilot in DR secondaries (#12014) 2021-07-08 12:30:01 -04:00
Pratyoy Mukhopadhyay adbaad17db
[VAULT-708] Zero out request counter on preSeal (#11970)
* [VAULT-708] Zero out request counter on preSeal

* [VAULT-708] Added changelog entry

* [VAULT-708] Add comment clarifying request counter zeroing
2021-07-07 14:03:39 -05:00
Nick Cabatoff e49f125fe5
Return a more meaningful error when trying to use a partially-replicated token. (#11993) 2021-07-07 08:43:58 -04:00
Pratyoy Mukhopadhyay a8b62bd93e
[VAULT-1981] Add OSS changes (#11999) 2021-07-06 17:12:24 -05:00
swayne275 6664846a65
Vault-2713: Retry Irrevocable Lease Revocation Periodically (#11977)
* prototype function to retry irrevocable lease revocation, clean up comment

* setup irrevocable lease revoke retry on expiration manager creation

* reduce expiration lag to 1 hour, use core context and timeout on retry attempts

* add lease namespace to revocation call

* start irrevocable revocation attempts in setupExpiration

* grab reference to channel to avoid race test failures
2021-07-06 15:05:04 -06:00
Pratyoy Mukhopadhyay c7e6a07939
Augment lease revocation metrics (#11954)
* [VAULT-1982] Add metric for irrevocable leases

* [VAULT-1982] Add metric for tracking worker count per queue

* [VAULT-1982] Update external docs

* [VAULT-1982] Add nil check on metrics sink

* [VAULT-1982] Update metric name, fix access patterns

* [VAULT-1982] Add locking when marking leases as irrevocable

* [VAULT-1982] Remove workers per mount metric

* [vault-1982] Misc review fixes

* [VAULT-1982] minor refactor

* [VAULT-1982] Add/update some comments
2021-06-30 14:37:36 -07:00
swayne275 fd517e6114
remove trailing / on leases API endpoint (#11959) 2021-06-30 13:02:26 -06:00
swayne275 f882c31856
update oss files with changes from ent PR #1928 (#11965) 2021-06-29 15:02:30 -06:00
Lars Lehtonen 27679e4fda
fix dropped errors in external_tests (#11947)
* vault/external_tests/token: fix dropped test error

* vault/external_tests/identity: fix dropped test errors
2021-06-29 09:04:26 -04:00
swayne275 67bc4fdabd
Require special privileges to list irrevocable leases (#11888)
* add leases path to sudo required set

* update TestSystemBackend_RootPaths with new special privilege paths

* note that list-leases requires sudo

* minor typo fixes
2021-06-28 16:51:47 -06:00
Hridoy Roy f3895f7cce
oss part of license diagnose (#11939) 2021-06-25 14:18:34 -07:00
hghaf099 6d7dbc85b5
diagnose: Add seal transit tls check (#11922)
* diagnose: Add seal transit tls check

* Fixing the path to the config file and the path to the cert files

* Addressing comment

* Addressing seal transit tls check comments
2021-06-24 18:30:42 -04:00
Hridoy Roy a8cc5e4557
TLS Verification Bugfixes (#11910)
* tls verification bugfix

* tls verification bugfix

* allow diagnose fail to report status when there are also warnings

* allow diagnose fail to report status when there are also warnings

* Update vault/diagnose/helpers_test.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* comments

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-06-24 10:43:49 -07:00
hghaf099 658a4ea276
Checking Validity of all Certs in the chain [VAULT-2114] (#11883)
* Checking Validity of all Certs in the chain

* Addressing Comments for TLS cert validation

* Fixing tls_verification tests

* Fixing minor issue in tls_verification tests

* Addressing Comments, Rebasing with main

* Adding comment on top of a test
2021-06-22 16:23:55 -04:00
Hridoy Roy bbfd450319
remove prints (#11918) 2021-06-22 09:40:53 -07:00
hghaf099 38b753046b
Tls check client certs [VAULT-2117] (#11860)
* add expiration warning to certificate checking for diagnose

* Adding TLS Client CA checks to diagnose
Adding checks for tls_disable_client_certs and tls_require_and_verify_client_cert flags

* Check validity of TLSClientCAFile
Adding related tests

* Addressing comments

* Fixing some tls tests

* Addressing comments

Co-authored-by: HridoyRoy <roy@hashicorp.com>
2021-06-18 19:35:38 -04:00
Scott Miller ee0d6603f3
Wire configuration checks into diagnose and fix resulting bugs. (#11854)
* Actually call config.Validate in diagnose

* Wire configuration checks into diagnose and fix resulting bugs.

* go mod vendor

* Merge to vendorless version

* Remove sentinel section to allow diagnose_ok to pass

* Fix unit tests
2021-06-17 13:09:37 -05:00
Hridoy Roy e38f991054
Diagnose checks for raft quorum status and file backend permissions (#11771)
* raft file and quorum checks

* raft checks

* backup

* raft file checks test

* address comments and add more raft and file and process checks

* syntax issues

* modularize functions to compile differently on different os

* compile raft checks everywhere

* more build tag issues

* raft-diagnose

* correct file permission checks

* upgrade tests and add a getConfigOffline test that currently does not work

* comment

* update file checks method signature on windows

* Update physical/raft/raft_test.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* raft tests

* add todo comment for windows root ownership

* voter count message

* raft checks test fixes

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-17 10:04:21 -07:00
Luke Jolly 0779d260ec
Fix error log typo for failing to decrypt keys (#11873) 2021-06-17 10:12:13 -04:00
Lars Lehtonen f390cad6e4
vault: fix dropped identity store test errors (#11867) 2021-06-16 11:32:20 -04:00
Hridoy Roy f46764abbf
Diagnose TLS Cert Expiry Warning Check: change 1 week expiry to 1 month expiry (#11871)
* change 1 week expiry to 1 month expiry

* report time left to expiry

* report time left to expiry
2021-06-15 12:31:28 -07:00
Hridoy Roy e58c39033b
Add expiration warning to certificate checking for diagnose [VAULT-1018] (#11850)
* add expiration warning to certificate checking for diagnose

* Update serviceregistration/consul/consul_service_registration.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* review comments

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-06-15 09:53:29 -07:00
Nick Cabatoff 2ce2617a8a
Reorganize request handling code so that we don't touch storage until we have the stateLock. (#11835) 2021-06-11 13:18:16 -04:00
Lars Lehtonen 5c8b3db0e5
vault: token store dropped test errors (#11799) 2021-06-11 11:41:31 -04:00
Brian Kassouf b42529dd17
Omit wrapping tokens and control groups from client counts (#11826)
* Omit wrapping tokens and control groups from client counts

* add changelog note
2021-06-10 15:57:51 -07:00
Hridoy Roy 65e3489c45
Diagnose resource creation checks (#11627)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* merge main branch

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* backup while I fix something else

* fix storage latency test errors

* runtime checks

* diagnose with timeout on seal
2021-06-10 12:29:32 -07:00
swayne275 7c5c9c830e
fix irrevocable leases API deadlock on m.coreStateLock (#11815)
* fix m.coreStateLock deadlock for API handler code

* improve lock comment
2021-06-10 11:01:14 -06:00
Mark Gritter 3e8a4bad1a
Acquire a per-lock lease to make renew and revoke atomic wrt each other. (#11122)
* Acquire a per-lock lease to make renew and revoke atomic wrt each other.
This means we don't have to hold pendingLock during I/O.

* Attempted fix for deadlock in token revocation.

* Comment fix.

* Fix error checking in loadEntry.

* Add benchmark

* Add a few additional locking locations

* Improve benchmark slightly

* Update vault/expiration.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Update vault/expiration.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add a lease lock into tidy

Co-authored-by: Scott Miller <smiller@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-06-09 17:28:41 -07:00
Nick Cabatoff c07711b488
Add some paths that use nonstandard auth. (#11804) 2021-06-09 17:04:06 -04:00
Nick Cabatoff abdf54c77e
Use the perfStandby field instead of the method because we don't want to try to grab the readlock when we're servicing a request (#11793) 2021-06-08 11:55:38 -04:00
Scott Miller b795643146
Third time trying to get the os_openbsd_arm.go right. (#11786)
* Fix the third dumb thing in this file

* ...
2021-06-07 17:19:47 -05:00
Scott Miller 69d0242db9
Add Advice and Advise functions for adding an advice section to any span. (#11760)
* wip

* wip

* Finish implementing advice handling and word wrapping

* Properly word wrap messages and warnings

* Remove debugging

* Remove debugging

* Remove unnecessary test

* unit test bug

* go vendor
2021-06-07 11:29:36 -05:00
Scott Miller c896c3a6aa
Fix SpotSkipped on openbsd/arm (#11781) 2021-06-07 11:29:15 -05:00
Hridoy Roy 1782b4e880
oss part of control groups upgrade (#11772)
* oss part of control groups upgrade

* changelog and docs

* formatting

* formatting
2021-06-07 09:15:35 -07:00
swayne275 62a92a77d7
bug fixes and improvements from ent namespace work (#11774) 2021-06-04 13:25:41 -06:00
Josh Black c8cfcd9514
OSS parts of sighup license reload (#11767) 2021-06-04 10:24:35 -07:00
Scott Miller fe8f0476fd
Fix build boolean logic causing os_common to be skipped wrongly (#11766) 2021-06-03 17:29:34 -05:00
Josh Black c6c0424a8e
OSS parts of sys/config/reload/license (#11695) 2021-06-03 10:30:30 -07:00
Scott Miller 9db384e0f7
Edge cases in the skipping gathering and output logic (#11752)
* Edge cases in the skipping gathering and output logic

* Fix unit test
2021-06-03 11:01:14 -05:00
Scott Miller d151b0b55b
Segment out disk checks to disable on openbsd/arm (#11749)
* Segment out disk checks to disable on openbsd/arm

Also add a spot skipped helper.

* Expected results may be fewer than actual because of variable length tests like disk usage

* Move to os_common and build on windows
2021-06-02 12:17:52 -05:00
swayne275 35aad1df4a
Vault 2256: fix lease count quotas causing panics on dr secondaries (#11742)
* lift relevant changes from ent to oss

* fix silent error bug in quotas
2021-06-02 10:12:05 -06:00
swayne275 9724f59180
Vault 1979: Query API for Irrevocable Leases (#11607)
* build out lease count (not fully working), start lease list

* build out irrevocable lease list

* bookkeeping

* test irrevocable lease counts for API/CLI

* fix listIrrevocableLeases, test listIrrevocableLeases, cleanup

* test expiration API limit

* namespace tweaks, test force flag on lease list

* integration test leases/count API, plenty of fixes and improvements

* test lease list API, fixes and improvements

* test force flag for irrevocable lease list API

* i guess this wasn't saved on the last refactor...

* fixes and improvements found during my review

* better test error msg

* Update vault/logical_system_paths.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* return warning with data if more than default leases to list without force flag

* make api doc more generalized

* list leases in general, not by mount point

* change force flag to include_large_results

* sort leases by LeaseID for consistent API response

* switch from bool flag for API limit to string value

* sort first by leaseID, then stable sort by expiration

* move some utils to be in oss and ent

* improve sort efficiency for API response

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-02 10:11:30 -06:00
Scott Miller 59bea5f6fd
Add ulimit check, and tidy unit test cases (#11678)
* Add ulimit check, and tidy unit test cases to avoid needing to have all results and perfect ordering

* Make order independent check recursive

* Fix unit tests

* Try a 5s request timeout
2021-06-01 12:43:51 -05:00
Cory Bolar 6c726bde67
Only fail auth plugin backend mounting for custom plugins (#11690)
Checking if a plugin is a builtin by comparing it's type to those in the builtin
registry allows for a custom plugin loaded with the same name to be considered a
builtin during error handling of the mounting process.  This can cause the vault
installation to brick itself because it cannot be unsealed to register a new
sha256 or file path for a previously loaded custom plugin.  Improve this logic
by checking the plugin catalog rather than the builtin registry.

Fixes #11687
2021-05-27 13:37:45 -05:00
Scott Miller 46da8ebae6
Diagnose Disk usage checks (#11673)
* Disk usage checks

* Move disk free earlier

* Move logic to helpers

* Bring over test logic from the ulimit PR

* imports

* Report error

* Get unit tests working
2021-05-27 10:17:52 -07:00
Hridoy Roy b6e1d62d33
Modularize Run Command (#11573)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* fix storage latency test errors

* review comments

* use random uuid for latency checks instead of static id
2021-05-25 15:23:20 -07:00
Scott Miller e305f78e9c
Add a Success helper to set successful spans' messages (#11621) 2021-05-21 19:21:23 -07:00
Scott Miller 941d01eee3
Initial Diagnose CLI output (#11583)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* wip

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* wip

* wip

* wip

* Revendor otel

* Fix merge related problems

* imports

* Fix unit tests

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-21 19:21:11 -07:00
Nick Cabatoff 2adef1f878
OSS parts of #1891 (sys/health license addition) (#11676) 2021-05-20 13:32:15 -04:00
Nick Cabatoff 01f96f18ce
VAULT-2439: OSS parts of #1889 (raft licensing init) (#11665) 2021-05-19 16:07:58 -04:00
Brian Kassouf eabfda0c79
Make sure we are checking the final peerset (#11664) 2021-05-19 12:01:52 -07:00
Brian Kassouf f498d0d389
Reload raft TLS keys on active startup (#11660) 2021-05-19 10:03:32 -07:00
Scott Miller 6b8d7fe2e6
Patch expiration fix over from ENT (#11650)
* Patch expiration fix over from ENT

* Rename changelog
2021-05-18 14:55:38 -07:00
Nick Cabatoff e212ec5d8e
OSS parts of ent PR #1857: license autoloading init changes. (#11623) 2021-05-17 14:10:26 -04:00
Scott Miller 7b05fcf9bf
Don't backoff if a listener error was a timeout (#11594) 2021-05-13 17:02:25 -05:00
Jason O'Donnell 502cf3b212
core: updates to password policy generator (#11596)
* core: fix bug in password policies not using namespaces

* Add changelog
2021-05-13 09:55:46 -04:00
Josh Black 641a81b74b
Provide a new API endpoint for retrieving signed licenses (#11543) 2021-05-12 12:19:25 -07:00
Scott Miller 9dbf1a7dba
Add infrastructure and helpers for skipping diagnose checks (#11593)
* Add infrastructure for skipping tests

* Add infrastructure for skipping tests

* Set it

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Implement type alias for test functions

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-12 12:54:40 -05:00
Scott Miller f0c3192f20
Add an exponential backoff to TCP listeners to avoid fast loops in error scenarios (#11588)
* Add an exponential backoff to TCP listeners to avoid fast loops in error scenarios

* reset loop delay

* changelog
2021-05-12 10:47:38 -05:00
swayne275 0229787a26
Vault 2303: Count irrevocable leases in quotas/metrics and other improvements (#11542)
* shrink generic error message

* move zombie loading to updatePendingInternal from loadEntryInternal

* probably the right metric/lease behavior for irrevocable leases...

* comment improvements

* test total lease count with valid and irrevocable leases
2021-05-11 14:04:06 -06:00
Lars Lehtonen 53dd619d2f
vault: deprecate errwrap.Wrapf() (#11577) 2021-05-11 13:12:54 -04:00
swayne275 f467d86805
don't panic if no data, improve log (#11581) 2021-05-11 10:41:57 -06:00
Scott Miller 40e0806f29
Add timeout helper (#11554)
* Add timeout helper

* cleanup-imports

* didnt need mainsection

Co-authored-by: HridoyRoy <roy@hashicorp.com>
2021-05-10 17:55:43 -07:00
swayne275 ee53fc5afb
Fix activity log fragment race test (#11565)
* grab reference to done channel while locked

* add clarifying comment

* lock and grab channel reference again for test races
2021-05-07 17:02:11 -06:00
Nick Cabatoff 53c7d1de7d
config for autoloading license (oss parts) 2021-05-07 08:55:41 -04:00
Nick Cabatoff 0926e302c5
Fix some panics resulting from calling ExpirationManager.Stop directly. (#11547) 2021-05-06 12:39:01 -04:00
Nick Cabatoff 663ad150a7
Make TestActivityLog_MultipleFragmentsAndSegments timeout on its own (#11490)
* The main driver for this change was to make the read from a.newFragmentCh timeout quickly rather than waiting for the test timeout (much longer).  While testing the change I observed a panic during shutdown, but it was swallowed and moreover there was no stack trace so it wasn't obvious.  I'm hoping we can get rid of the recover, so I fixed the issue in the activitylog tests that needed it.
2021-05-06 10:19:53 -04:00
Scott Miller 4fc6e8b366
Fix barrier key autoration config edge cases (#11541)
* Add an Int64 type

* Use the new Int64 type so that even 32 bit builds can specify max_operations above 2^31

* Missed a spot

* go mod vendor

* fix cast

* changelog

* Update unit test to ensure this works on both 32 and 64-bit archs
2021-05-05 14:39:04 -05:00
swayne275 56c57ffddd
change zombie terminology to irrevocable (#11525) 2021-05-04 15:48:02 -06:00
swayne275 335e4c3711
Introduce Logical Unrecoverable Error, Use it in Expiration Manager (#11477)
* build out zombie lease system

* add typo for CI

* undo test CI commit

* time equality test isn't working on CI, so let's see what this does...

* add unrecoverable proto error, make proto, go mod vendor

* zombify leases if unrecoverable error, tests

* test fix: somehow pointer in pointer rx is null after pointer rx called

* tweaks based on roy feedback

* improve zombie errors

* update which errors are unrecoverable

* combine zombie logic

* keep subset of zombie lease in memory
2021-05-03 17:56:06 -06:00
Vishal Nayak 1e61f799ca
Use correct mount accessor when refreshing external group memberships (#11506)
* Use correct mount accessor when refreshing external group memberships

* Add CL

* Handle the renew case properly
2021-05-03 08:23:59 -04:00
Hridoy Roy 28aa9b9175
Diagnoses Consul Direct Access Check (#11505)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* run direct access checks in diagnose command

* review comments

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 16:21:06 -07:00
Hridoy Roy 5b63066c2c
Diagnose V0: Storage End to End Checks (#11468)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* cosmetic changes

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 13:33:13 -07:00
Scott Miller fcb9bab51b
Fix rotate/config unit test for 32 bit yet again (#11491) 2021-04-30 10:31:11 -05:00
Scott Miller 85fbd45e1c
Create helpers which integrate with OpenTelemetry for diagnose collection (#11454)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-04-29 13:32:41 -05:00
swayne275 8db70c9e58
Expiration Manager: Handle Presumed Irrevocable Leases Separately (#11452)
* build out zombie lease system

* add typo for CI

* undo test CI commit

* time equality test isn't working on CI, so let's see what this does...

* protect against nil pointer receiver calls
2021-04-29 09:12:02 -06:00
Josh Black ec105f288f
Switch to shared raft-boltdb library and add metrics (#11269) 2021-04-26 16:01:26 -07:00
Nick Cabatoff e3a88c59e9
Add metrics for perf standby and replication node type. (#11472) 2021-04-26 13:54:19 -04:00
Nick Cabatoff 5a4151873c
Cast constant to int64 so it doesn't overflow. (#11466) 2021-04-26 10:04:39 -04:00
Hridoy Roy 22cab6185d
[VAULT-1441] Fix race that allowed remounting on path used by another mount (#11453)
* remount concurrent test fix

* changelog

* Update changelog/11453.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-04-24 09:05:41 -07:00
Angel Garbarino 2e35e9578c
UI/obscure secret on input (#11284)
* new font and add as font-family to be used in masked-input

* clean up logic

* refactor for displayOnly

* start cert masking

* work on certificates

* upload cert work

* fix global styling

* fix styling for class no longer used

* make mask by default and remove option

* glimmerize start and certificate on LDAP a file field

* glimmerize actions

* first part of glimmerizing text-file still need to do some clean up

* not doing awesome over here

* getting ready to un-glimmer

* unglimmerize

* remove placeholder based on conversations with design

* clean up text-file

* cleanup

* fix class bindings

* handle class binding

* set up for test

* fix elementId

* track down index

* update masked-input test

* add more to the masked-input test

* test-file test

* fix broken test

* clear old style

* clean up

* remove pgp key masked font, this really needs to be refactored to text-file component

* changelog

* cover other certificate view

* add allowCopy

* address some pr styling comments

* improve test coverage

* fix some issues

* add attr.options.masked
2021-04-22 08:58:37 -06:00
Josh Black 06809930a3
Add HTTP response headers for hostname and raft node ID (if applicable) (#11289) 2021-04-20 15:25:04 -07:00
Nick Cabatoff b7ed30afed
Poll until token disappears from storage. (#11424) 2021-04-20 14:55:10 -04:00
Nick Cabatoff 75ec35421d
Refactor TestTokenStore_CubbyholeTidy so that it can be tested on ent with a non-root namespace. (#11422) 2021-04-20 14:49:25 -04:00
Brian Kassouf c400dfdd3d
core/token: fix panic looking up invalid batch tokens (#11415) 2021-04-19 18:26:51 -07:00
Nick Cabatoff 474c4e8134
Make cubbyhole revocation/tidying compatible with cubbys in namespaces. (#11408) 2021-04-19 17:28:04 -04:00
Nick Cabatoff a8023e0fdb
Add support for unauthenticated pprof access on a per-listener basis,… (#11324)
* Add support for unauthenticated pprof access on a per-listener basis, as we do for metrics.

* Add missing pprof sub-targets like 'allocs' and 'block'.  Capture the goroutine subtarget a second time in text form.  This is mostly a convenience, but also I think the pprof format might be a bit lossy?
2021-04-19 14:30:59 -04:00
Nick Cabatoff 541ae8636c
On lease deletion, also delete non-orphan batch token parent index (#11377) 2021-04-16 17:03:22 -04:00
Nick Cabatoff b07a10331f
Add metrics for requests forwarded by standbys. (#11366) 2021-04-16 14:02:20 -04:00
Nick Cabatoff 242d258e94
Fix goroutine leak caused by updating rate quotas (#11371)
Make sure that when we modify a rate quota, we stop the existing goroutine before starting the new one.
2021-04-16 14:00:01 -04:00
Nick Cabatoff 33dd025278
When a standby does a ForwardRequest, it's not using the request context, and thus not getting timed out properly when it takes too long. (#11322)
The rpcClientConnContext is still used to terminate gRPC internal/dialer-related goroutines, but the actual RPC is now timed out when the request times out, e.g. due to the default max request duration.  This mirrors what we do with the parallel forwarding code in ENT.
2021-04-15 10:23:26 -04:00
Hridoy Roy 996f114ad7
TLS Diagnose Formatting Fixes (#11342)
* diagnose formatting fixes

* diagnose formatting fixes
2021-04-12 10:55:33 -07:00
Hridoy Roy fde9f2f71d
Add More TLS Tests and Verification of TLS Root Certificate (#11300)
* tls tests and root verification

* make the certificate verification check correct for non root CA case

* add expiry test

* addressed comments but struggling with the bug in parsing Cas and inters from single file:

* final checks on tls and listener

* cleanup
2021-04-12 08:39:40 -07:00
Vishal Nayak 4666f40925
Support autopilot when raft is for HA only (#11260) 2021-04-12 09:33:21 -04:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Brian Kassouf de0253056c
Fix a few static analysis findings (#11307) 2021-04-07 16:48:40 -07:00
Nick Cabatoff c2673ee86a
Move SanitizedConfig back to a shared-ent file. (#11291) 2021-04-07 10:25:05 -04:00
Hridoy Roy 049f2513e6
Initial Diagnose Command for TLS and Listener [VAULT-1896, VAULT-1899] (#11249)
* sanity checks for tls config in diagnose

* backup

* backup

* backup

* added necessary tests

* remove comment

* remove parallels causing test flakiness

* comments

* small fix

* separate out config hcl test case into new hcl file

* newline

* addressed comments

* addressed comments

* addressed comments

* addressed comments

* addressed comments

* reload funcs should be allowed to be nil
2021-04-06 16:40:43 -07:00
Nick Cabatoff 44c00cd54f
Fix: leader_tls_servername raft option only worked when used with mTLS and/or an explicit CA cert. (#11252) 2021-04-06 09:16:54 -04:00
Nick Cabatoff df7404e67e
Fix deadlock in inmemlayer (#11225)
Also tweak autopilot test timings to adapt to things running faster.
2021-03-31 13:40:26 -04:00
swayne275 532093d065
fix end date calculation (#11232) 2021-03-30 09:58:45 -06:00
Scott Miller c44f18e3e4
Use pointers to 64-bit atomics to enforce alignment in AES-GCM Barrier (#11211)
* Align atomics in barrier

* Use the pointer solution
2021-03-26 14:51:56 -05:00
Nick Cabatoff 41d9030fbb
Disable autopilot in raft-ha mode. (#11181)
* Disable autopilot in raft-ha mode.

* Also don't run autopilot on DR secondaries.
2021-03-23 14:13:44 -07:00
Brian Kassouf d01a068929
Remove retry from new raft test (#11158) 2021-03-19 12:41:57 -07:00
Nick Cabatoff b3af58d758
Expose snapshot_interval tunable instead of setting it in prod code for the sake of a test. (#11160) 2021-03-19 15:41:42 -04:00
Brian Kassouf 28aba513f2
storage/raft: Ensure peers are informed of their correct suffrage when added with AutoPilot (#11155)
* storage/raft: Ensure peers are informed of their correct suffrage when added with AutoPilot

* Add test ensuring peer sets are equivalent
2021-03-19 11:53:50 -07:00
Scott Miller 535bcf289e
Fix handling of minimum operations, and forward rotate/config requests to Primary (#11116)
* Boost max_operations to the greater of that specified or absoluteMinOperations

* Forward rotation config requests to the primary

* Reject rotation configs outside the min/max range

* Minor wording fix
2021-03-18 15:08:47 -05:00
Nick Cabatoff 411495514c
Add a test for server stabilization (#11128) 2021-03-17 17:23:13 -04:00
Vishal Nayak 9839e76192
Remove unneeded fields from state output (#11073) 2021-03-10 12:08:12 -05:00
Brian Kassouf aa00b53ba1
Make sure we sanitize the rotation config on each clone (#11050)
* Make sure we sanitize the rotation config on each clone

* Add regression test for missing rotation config

* use Equals

* simplify

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
2021-03-08 10:59:21 -06:00