Commit Graph

2410 Commits

Author SHA1 Message Date
John-Michael Faircloth 32152e10fd
Identity: check NextSigningKey existence during key rotation (#13298)
* oidc: fix key rotation panic

* refactor and update unit tests

* add changelog
2021-11-29 15:10:58 -06:00
hghaf099 9640d35136
Unify HTTPResponseWriter and StatusHeaderResponseWriter (#13200)
* Unify NewHTTPResponseWriter ant NewStatusHeaderResponseWriter to fix ResponseWriter issues

* adding changelog

* removing unnecessary function from the WrappingResponseWriter interface

* changing logical requests responseWriter type

* reverting change to HTTPResponseWriter
2021-11-23 14:30:25 -05:00
akshya96 f77223bfe5
Authenticate to "login" endpoint for non-existent mount path bug (#13162)
* changing response from missing client token to permission denied

* removing todo comment

* fix tests

* adding changelog

* fixing changelog
2021-11-22 17:06:59 -08:00
Austin Gebauer d5f4fbecc1
identity/oidc: optional nonce parameter for authorize request (#13231) 2021-11-22 09:42:22 -08:00
divyapola5 5236fe93aa
Add a new parameter "allowed_managed_keys" to mount config (#13202)
* Add a new parameter "allowed_managed_keys" to mount config

* Adjust formatting in mount.go

* Add changelog entry
2021-11-21 19:08:38 -06:00
swayne275 3458c22df0
Vault-2257: don't log token error on DR Secondary (#13137)
* don't log token error on DR Secondary

* stop gauge collector expiration errors on dr secondary

* don't check dr secondary for token create

* see if CI hits panic

* Revert "don't check dr secondary for token create"

This reverts commit c036a1a544d3a20d29d046f1ee239ab1563ce4d9.

* don't check dr secondary for token create

* Revert "see if CI hits panic"

This reverts commit 1e15aa535cac6e4d1684aaf47c8746c094068eb8.

* remove condition on log
2021-11-17 09:21:54 -07:00
Nick Cabatoff 1ec904976a
Note that versionTimestamps are only loaded on the active node. 2021-11-16 15:05:59 -05:00
Nick Cabatoff c2d9215d1d
Fix startup failures when aliases from a pre-1.9 vault version exist (#13169)
* Add AllowMissing to local_bucket_key schema, preventing startup failures in post-unseal when aliases from an older version exist.
2021-11-16 14:56:34 -05:00
Scott Miller a5e55f6b05
Fix a data race in the new autoseal health check (#13136)
* Move the ctx capture outside the goroutine to avoid a race

* refactor the toggleable wrapper to avoid races

* Move the capture back outside the goroutine

* defer
2021-11-12 15:58:46 -06:00
Scott Miller 10270b6985
Add a periodic test of the autoseal to detect loss of connectivity. (#13078)
* Add a periodic test of the autoseal to detect loss of connectivity

* Keep the logic adjacent to autoseal

* imports

* typo, plus unnecessary constant time compare

* changelog

* pr feedback

* More feedback

* Add locking and a unit test

* unnecessary

* Add timeouts to encrypt/decrypt operations, capture activeContext before starting loop

* Add a block scope for the timeout

* copy/paste ftl

* Refactor to use two timeouts, and cleanup the repetitive failure code

* Readd 0ing gauge

* use millis

* Invert the unit test logic
2021-11-10 14:46:07 -06:00
John-Michael Faircloth e6ffaaf835
OIDC: return full issuer uri on read provider (#13058)
* return full issuer uri on read provider

* remove err check

* simplify full issuer logic
2021-11-10 12:35:31 -06:00
swayne275 e137045050
Vault 936: use core.activeContext in ActivityLog (#13083)
* update activity log to use core's activeContext for cleaner worker termination

* update tests to use core activeContext instead of generic context

* pass context around instead

* revert context change

* undo test context changes

* change worker context

* accidentally undid context for fcn signature changes
2021-11-09 11:47:39 -07:00
Steven Clark 8c50afc07c
Address a data race issue within identity_store_util::processLocalAlias (#13093)
- When loading an existing alias within processLocalAlias we aren't
   cloning the object from the memory store. There seems to be a data
   race within the function when calling entity.UpsertAlias and
   a concurrent invalidation routine.

 ==================
WARNING: DATA RACE
Read at 0x00c00bd03d08 by goroutine 94:
  google.golang.org/protobuf/internal/impl.pointer.Elem()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/pointer_unsafe.go:118 +0x2b3
  google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:76 +0x265
  google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:56 +0x12a
  google.golang.org/protobuf/internal/impl.(*MessageInfo).size()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:40 +0x95
  google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:33 +0x6c
  google.golang.org/protobuf/proto.MarshalOptions.marshal()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:153 +0x1f3
  google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:122 +0xa5
  github.com/golang/protobuf/proto.marshalAppend()
      /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:40 +0xe4
  github.com/golang/protobuf/proto.Marshal()
      /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:23 +0x64
  github.com/hashicorp/vault/helper/identity.(*Entity).Clone()
      /go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150
  github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn()
      /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306
  github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate()
      /go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c
  github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm()
      /go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d
  github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey()
      /go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a
  github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey()
      <autogenerated>:1 +0x7d
  github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey()
      /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390
  github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler()
      /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b
  github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453()
      /go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71

Previous write at 0x00c00bd03d08 by goroutine 52:
  github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias()
      /go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271
  github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias()
      /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672
  github.com/hashicorp/vault/vault.possiblyForwardEntityCreation()
      /go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286
  github.com/hashicorp/vault/vault.(*Core).handleLoginRequest()
      /go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a
  github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest()
      /go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11
  github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest()
      /go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5
  github.com/hashicorp/vault/vault.(*Core).HandleRequest()
      /go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2
  github.com/hashicorp/vault/http.request()
      /go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1
  github.com/hashicorp/vault/http.handleLogicalInternal.func1()
      /go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.handleRequestForwarding.func1()
      /go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  net/http.(*ServeMux).ServeHTTP()
      /usr/local/go/src/net/http/server.go:2424 +0xc5
  github.com/hashicorp/vault/http.wrapHelpHandler.func1()
      /go/src/github.com/hashicorp/vault/http/help.go:23 +0x281
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.wrapCORSHandler.func1()
      /go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1()
      /go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1()
      /go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.wrapGenericHandler.func1()
      /go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1()
      /go/pkg/mod/github.com/hashicorp/go-cleanhttp@v0.5.2/handlers.go:42 +0xc1
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  net/http.serverHandler.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2878 +0x89a
  net/http.initALPNRequest.ServeHTTP()
      /usr/local/go/src/net/http/server.go:3479 +0x34d
  net/http.(*initALPNRequest).ServeHTTP()
      <autogenerated>:1 +0x8f
  net/http.Handler.ServeHTTP-fm()
      /usr/local/go/src/net/http/server.go:87 +0x75
  net/http.(*http2serverConn).runHandler()
      /usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd
  net/http.(*http2serverConn).processHeaders·dwrap·31()
      /usr/local/go/src/net/http/h2_bundle.go:5562 +0x64
2021-11-09 10:00:26 -05:00
Hridoy Roy e7e881c559
Port: Allow Routing to Partial Monthly Client Count From Namespaces (#13086)
* add function for routing activity log client counts to ent namespaces

* changelog
2021-11-08 15:38:35 -08:00
Rémi Lapeyre ae0b5e41e0
Add read support to sys/mounts/:path (#12792)
* Add read support to sys/mounts/:path

Closes https://github.com/hashicorp/vault/issues/12349

* Add changelog entry

* Empty commit to trigger CI

* Empty commit to trigger CI
2021-11-08 10:32:01 -08:00
Nick Cabatoff 40640ef43f
Fix errors logged on standbys when we try to write versions to storage (#13042) 2021-11-08 10:04:17 -05:00
Meggie 627c43e496
1.10.0-dev version bump (#12987)
* 1.10.0-dev version bump

* Remove hard-coded versions from tests. (#13026)

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-11-03 12:50:12 -04:00
Brian Kassouf 0e1b2c1b7b
Fix some more error shadowing issues (#12990) 2021-11-01 11:43:00 -07:00
John-Michael Faircloth 792400a24c
ensure errors are checked (#12989) 2021-11-01 13:34:51 -05:00
Austin Gebauer 83dcbc0cd5
Deny access to UserInfo endpoint if client no longer allowed by provider (#12949) 2021-10-28 09:46:07 -07:00
Austin Gebauer e5b45b5416
Adds check to ensure authorization code can only be exchanged by the issuing provider (#12948) 2021-10-28 09:45:41 -07:00
Brian Kassouf 350ecd6bc5
Fix leak of restore worker go routines after an error (#12933) 2021-10-27 09:08:10 -07:00
Austin Gebauer b3fab954fe
Use mutex in OIDC configuration handlers (#12932) 2021-10-27 08:23:05 -07:00
swayne275 92e6972b86
oss LastDRWAL (#12931) 2021-10-26 15:17:20 -06:00
Hridoy Roy 1d29b93eb5
fix adding clientID to request in audit log (#12919)
* fix adding clientID to request in audit log

* fix boolean statement

* use standard encoding for client ID instead of urlEncoding

* change encoding in tests

* add in client counts to request handling

* remove redundant client ID generation in request handling

* directly add clientID to req after handling token usage
2021-10-26 09:02:13 -07:00
Austin Gebauer a25cfe822e
Adds missing unlock of RWMutex in OIDC delete key (#12916) 2021-10-25 09:59:26 -07:00
swayne275 95e5cdd000
VAULT 2844: remove legacy lease revocation strategy (#12888)
* remove legacy lease revocation strategy

* add deprecation change log note

* remove VAULT_16_REVOKE_PERMITPOOL

* update changelog
2021-10-22 17:37:01 -06:00
hghaf099 442b5920e2
Entities may have duplicate policies (#12812)
* Entities may have duplicate plicies

* Adding changelog

* removing duplicates on reading entity policies

* fix changelog
2021-10-22 19:28:31 -04:00
Nick Cabatoff ff74f49047
Move to go 1.17 (#12868)
Also ensure that the go 1.17 breaking changes to net.ParseCIDR don't make us choke on stored CIDRs that were acceptable to older Go versions.
2021-10-21 09:32:03 -04:00
Vishal Nayak 6eead9f09b
Fix entity alias deletion (#12834)
* Fix entity alias deletion

* Fix tests

* Add CL
2021-10-19 15:05:06 -04:00
Brian Kassouf c705adc79c
Fix some linting errors (#12860) 2021-10-18 17:29:47 -07:00
Austin Gebauer 4e5b865c4f
Rename scopes to scopes_supported for OIDC providers (#12851) 2021-10-15 19:33:32 -07:00
Hridoy Roy 19822781cc
use IsRoot helper before clientID generation in activity log (#12846)
* use IsRoot helper in activity log

* use IsRoot helper in activity log
2021-10-15 14:04:55 -07:00
Vishal Nayak 476fb08e0d
Local aliases OSS patch (#12848)
* Local aliases OSS patch

* build fix
2021-10-15 15:20:00 -04:00
saltperfect 66369469d9
Removed unpublished:true for sys/internal/* endpoints (#12713)
* removed unpublished:true for sys/internal/* endpoints

* added changelog file

* updated change log and added placeholder summary as these endpoints are not mentioned in docs.

* added documentation for internal/ui/namspaces and resultant-acl

* updated log configs
2021-10-15 14:50:14 -04:00
Pratyoy Mukhopadhyay 148109b8ed
[VAULT-3252] Disallow alias creation if entity/accessor combination exists (#12747)
* Disallow alias creation if entity/accessor combination exists

* Add changelog

* Address review comments

* Add handling to aliasUpdate, some field renaming

* Update tests to work under new entity-alias constraint

* Add check to entity merge, other review fixes

* Log duplicated accessors only once

* Fix flaky test

* Add note about new constraint to docs

* Update entity merge warn log
2021-10-14 09:52:07 -07:00
Hridoy Roy 1c427d3286
Port: add client ID to TWEs in activity log [vault-3136] (#12820)
* port for tracking twes as clients

* comment clean up

* changelog

* change changelog entry phrasing
2021-10-14 09:10:59 -07:00
Austin Gebauer 0551f91068
Adds OIDC Token and UserInfo endpoints (#12711) 2021-10-13 18:59:36 -07:00
swayne275 76556c867b
port relevant ent changes (#12823) 2021-10-13 18:15:54 -06:00
Chris Capurso bbb4ab4a41
Add HTTP PATCH support to KV (#12687)
* handle HTTP PATCH requests as logical.PatchOperation

* update go.mod, go.sum

* a nil response for logical.PatchOperation should result in 404

* respond with 415 for incorrect MIME type in PATCH Content-Type header

* add abstraction to handle PatchOperation requests

* add ACLs for patch

* Adding JSON Merge support to the API client

* add HTTP PATCH tests to check high level response logic

* add permission-based 'kv patch' tests in prep to add HTTP PATCH

* adding more 'kv patch' CLI command tests

* fix TestHandler_Patch_NotFound

* Fix TestKvPatchCommand_StdinValue

* add audit log test for HTTP PATCH

* patch CLI changes

* add patch CLI tests

* change JSONMergePatch func to accept a ctx

* fix TestKVPatchCommand_RWMethodNotExists and TestKVPatchCommand_RWMethodSucceeds to specify -method flag

* go fmt

* add a test to verify patching works by default with the root token

* add changelog entry

* get vault-plugin-secrets-kv@add-patch-support

* PR feedback

* reorder some imports; go fmt

* add doc comment for HandlePatchOperation

* add json-patch@v5.5.0 to go.mod

* remove unnecessary cancelFunc for WriteBytes

* remove default for -method

* use stable version of json-patch; go mod tidy

* more PR feedback

* temp go get vault-plugin-secrets-kv@master until official release

Co-authored-by: Josh Black <raskchanky@users.noreply.github.com>
2021-10-13 15:24:31 -04:00
John-Michael Faircloth 2abf916ddb
Add support to parameterize unauthenticated paths (#12668)
* store unauthenticated path wildcards in map

* working unauthenticated paths with basic unit tests

* refactor wildcard logic

* add parseUnauthenticatedPaths unit tests

* use parseUnauthenticatedPaths when reloading backend

* add more wildcard test cases

* update special paths doc; add changelog

* remove buggy prefix check; add test cases

* prevent false positives for prefix matches

If we ever encounter a mismatched segment, break and set a flag to
prevent false positives for prefix matches.

If it is a match we need to do a prefix check. But we should not return
unless HasPrefix also evaluates to true. Otherwise we should let the for
loop continue to check other possibilities and only return false once
all wildcard paths have been evaluated.

* refactor switch and add more test cases

* remove comment leftover from debug session

* add more wildcard path validation and test cases

* update changelong; feature -> improvement

* simplify wildcard segment matching logic

* refactor wildcard matching into func

* fix glob matching, add more wildcard validation, refactor

* refactor common wildcard errors to func

* move doc comment to logical.Paths

* optimize wildcard paths storage with pre-split slices

* fix comment typo

* fix test case after changing wildcard paths storage type

* move prefix check to parseUnauthenticatedPaths

* tweak regex, remove unneeded array copy, refactor

* add test case around wildcard and glob matching
2021-10-13 11:51:20 -05:00
hghaf099 ad2ef412cc
Customizing HTTP headers in the config file (#12485)
* Customizing HTTP headers in the config file

* Add changelog, fix bad imports

* fixing some bugs

* fixing interaction of custom headers and /ui

* Defining a member in core to set custom response headers

* missing additional file

* Some refactoring

* Adding automated tests for the feature

* Changing some error messages based on some recommendations

* Incorporating custom response headers struct into the request context

* removing some unused references

* fixing a test

* changing some error messages, removing a default header value from /ui

* fixing a test

* wrapping ResponseWriter to set the custom headers

* adding a new test

* some cleanup

* removing some extra lines

* Addressing comments

* fixing some agent tests

* skipping custom headers from agent listener config,
removing two of the default headers as they cause issues with Vault in UI mode
Adding X-Content-Type-Options to the ui default headers
Let Content-Type be set as before

* Removing default custom headers, and renaming some function varibles

* some refacotring

* Refactoring and addressing comments

* removing a function and fixing comments
2021-10-13 11:06:33 -04:00
Victor Rodriguez ae40d1c2c9
Wait for expiration manager to be out of restore mode while testing. (#12779) 2021-10-13 08:58:02 -04:00
Matt Greenfield 0b3eea4441
Dedup from_entity_ids when merging two entities (#10101)
Fixes #10100
2021-10-12 15:35:19 -04:00
Pratyoy Mukhopadhyay c52ff27e9d
Update naptime for retrytoken test (#12799) 2021-10-12 11:12:06 -07:00
John-Michael Faircloth f30c3ac621
Filter identity token keys (#12780)
* filter identity token keys

* Update test cases to associate keys with roles

* use getOIDCRole helper

* add func comment and test assertion

* add changelog

* remove unnecessary code

* build list of keys to return by starting with a list of roles

* move comment

* update changelog
2021-10-12 11:14:03 -05:00
swayne275 2edac287ae
update function signature and call (#12806) 2021-10-11 18:21:38 -06:00
John-Michael Faircloth e72c15426b
OIDC: Unique prefix to client_secret (#12745)
* Unique prefix to client_id and client_secret

* only prefix client secret and use constants

* split client secret checks
2021-10-08 16:02:20 -05:00
Scott Miller 1097f356af
Add a TRACE log with TLS connection details on replication connections (#12754)
* remove cruft
use helper
Add a helper for getting public key sizes
wip

* error names

* Fix ecdsa

* only if trace is on

* Log listener side as well

* rename

* Add remote address

* Make the log level configurable via the env var, and a member of the Listener and thus modifiable by tests

* Fix certutil_test
2021-10-07 14:17:31 -05:00
Austin Gebauer e09657e1f3
Adds ability to define an inline policy and internal metadata on tokens (#12682)
* Adds ability to define an inline policy and internal metadata to tokens

* Update comment on fetchEntityAndDerivedPolicies

* Simplify handling of inline policy

* Update comment on InternalMeta

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Improve argument name

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Use explicit SkipIdentityInheritance token field instead of implicit InlinePolicy behavior

* Add SkipIdentityInheritance to pb struct in token store create method

* Rename SkipIdentityInheritance to NoIdentityPolicies

* Merge latest from main and make proto

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-10-07 10:36:22 -07:00
swayne275 b9fde1dd6f
oss port (#12755) 2021-10-07 11:25:16 -06:00
vinay-gopalan 458927c2ed
[VAULT-3157] Move `mergeStates` utils from Agent to api module (#12731)
* move merge and compare states to vault core

* move MergeState, CompareStates and ParseRequiredStates to api package

* fix merge state reference in API Proxy

* move mergeStates test to api package

* add changelog

* ghost commit to trigger CI

* rename CompareStates to CompareReplicationStates

* rename MergeStates and make compareStates and parseStates private methods

* improved error messaging in parseReplicationState

* export ParseReplicationState for enterprise files
2021-10-06 10:57:06 -07:00
hghaf099 a3796997d9
Fix a Deadlock on HA leadership transfer (#12691)
* Fix a Deadlock on HA leadership transfer when standby
was actively forwarding a request
fixes GH #12601

* adding the changelog
2021-10-04 13:55:15 -04:00
vinay-gopalan 680e8515fa
[VAULT-3472] Cap Client `id_token_ttl` field to associated Key's `verification_ttl` (#12677) 2021-10-01 10:47:40 -07:00
Matt Greenfield 8577602395
Fix entity group associations (#10085)
- When two entities are merged, remove the from entity ID in any
  associated groups.
- When two entities are merged, also merge their associated group
  memberships.

Fixes #10084
2021-10-01 10:22:52 -04:00
Brian Kassouf 39a9727c8b
Update protobuf & grpc libraries and protoc plugins (#12679) 2021-09-29 18:25:15 -07:00
Tero Saarni 944332d12d
Update Go client libraries for etcd (#11980)
* Update Go client libraries for etcd

* Added etcd server container to run etcd3 tests automatically.

* Removed etcd2 test case: it fails the backend tests but the failure is
  unrelated to the uplift.  The etcd2 backend implementation does not
  remove empty nested nodes when removing leaf (see comments in #11980).
2021-09-29 14:28:13 -04:00
swayne275 af3136d393
move from static sleep to timed loop waiting for lease revocation (#12627) 2021-09-27 14:23:17 -06:00
Pratyoy Mukhopadhyay 73ae1fcfff
Update flaky expiration test (#12628)
* Update flaky expiration test

* Remove extraneous comment

Co-authored-by: swayne275 <swayne275@gmail.com>

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-09-27 11:51:27 -07:00
Austin Gebauer b58913ad9f
Adds OIDC Authorization Endpoint to OIDC providers (#12538) 2021-09-27 10:55:29 -07:00
Austin Gebauer da394f34b1
Adds additional OIDC discovery metadata (#12623) 2021-09-27 10:05:55 -07:00
Austin Gebauer 93f8d248d3
Use durations for ID and access token TTLs (#12632) 2021-09-27 10:05:28 -07:00
Calvin Leung Huang 7ad62f5be4
core: set namespace within GeneratePasswordFromPolicy (#12635)
* core: set namespace from the sysview's mount entry on GeneratePasswordFromPolicy

* test: update TestDynamicSystemView to be ns-aware, update tests

* add changelog entry
2021-09-27 09:08:07 -07:00
Pratyoy Mukhopadhyay 8e6698fb4a
[VAULT-3519] Return no_default_policy on token role read (#12565)
* [VAULT-3519] Return no_default_policy on token role read if set

* [VAULT-3519] Add changelog

* [VAULT-3519] Always return token_no_default_policy on role read

* Fix broken test

* Update role read response in docs
2021-09-21 09:53:08 -07:00
Tiernan a538936367
Allow globbing dis/allowed_policies_glob in token roles (#7277)
* Add allowed_policies_glob and disallowed_policies_glob that are the same as allowed_policies and disallowed_policies but allow glob matching.

* Update changelog, docs, tests, and comments for (dis)allowed_token_glob token role feature.

* Improve docs and unit tests for auth/token role policy globbing.
2021-09-21 08:25:06 -07:00
Nick Cabatoff 2bd95232cf
Fail alias rename if the resulting (name,accessor) exists already (#12473) 2021-09-21 08:19:44 -04:00
akshya96 c643dc1d53
Add Custom metadata field to alias (#12502)
* adding changes

* removing q.Q

* removing empty lines

* testing

* checking tests

* fixing tests

* adding changes

* added requested changes

* added requested changes

* added policy templating changes and fixed tests

* adding proto changes

* making changes

* adding unit tests

* using suggested function
2021-09-17 11:03:47 -07:00
John-Michael Faircloth bee9f25277
OIDC provider: show success message in CLI (#12574) 2021-09-17 11:41:08 -05:00
Michael Ward c15ac1053f
Expose secret_id_accessor as WrappedAccessor when wrapping secret-id creation. (#12425)
* Expose secret_id_accessor as WrappedAccessor when wrapping secret-id creation.

* Add changelog.

* Minor updates as suggested.

* Adding external test for wrapped accessor.

* Add check that mounttype is approle.

* Update changelog text to use improvement
2021-09-16 10:47:49 -07:00
John-Michael Faircloth 4ed0eb4493
identity: fix bug and increase logging for jwks cache control max age test (#12561)
* identity: increase logging for jwks cache control max age test

* clarify comment

* add more logging for jwks cache control max age in test
2021-09-15 14:38:29 -05:00
John-Michael Faircloth c42c9993a0
feature: OIDC keys endpoint (#12525)
* add keys path and initial handler

* read provider public keys

* add test cases

* remove some debug logs

* update tests after merging main

* refactor list all clients

* refactor logic to collect Key IDs
2021-09-14 15:37:53 -05:00
hghaf099 2576be23d0
Fixing a flaky test TestQuotas_RateLimitQuota_ExemptPaths (#12532)
* Fixing a flaky test TestQuotas_RateLimitQuota_ExemptPaths

* fixing a string formatting by removing an extra parameter
2021-09-10 17:41:14 -04:00
John-Michael Faircloth c42bbb369c
Identity: prepublish jwt signing keys (#12414)
* pre-publish new signing keys for `rotation_period` of time before using

* Work In Progress: Prepublish JWKS and even cache control

* remove comments

* use math/rand instead of math/big

* update tests

* remove debug comment

* refactor cache control logic into func

* don't set expiry when create/update key

* update cachecontrol name in oidccache for test

* fix bug in periodicfunc test case

* add changelog

* remove confusing comment

* add logging and comments

* update change log from bug to improvement

Co-authored-by: Ian Ferguson <ian.ferguson@datadoghq.com>
2021-09-09 13:47:42 -05:00
John-Michael Faircloth 0d04a9892a
identity: enforce key param and key existence on role creation (#12208)
* identity: handle creation of role without a key parameter

* update docs to not require key parameter for creation of a role

* add changelog

* require key param when creating a role

* lock create/update role; remove now redundant key check

* update changelog and UTs

* update change log to refelct actual implementation

* remove deprecated test case
2021-09-08 10:46:58 -05:00
Daniel Kimsey b4b61efc75
Auto-join support for IPv6 discovery (#12366)
* Auto-join support for IPv6 discovery

The go-discover library returns IP addresses and not URLs. It just so
happens net.URL parses "127.0.0.1", which isn't a valid URL.

Instead, we construct the URL ourselves. Being careful to check if it's
an ipv6 address and making sure it's in explicit form if so.

Fixes #12323

* feedback: addrs & ipv6 test

Rename addrs to clusterIPs to improve clarity and intent

Tighten up our IPv6 address detection to be more correct and to ensure
it's actually in implicit form
2021-09-07 11:55:07 -07:00
John-Michael Faircloth 01011973a3
feature: OIDC discovery endpoint (#12481)
* OIDC Provider: implement discovery endpoint

* handle case when provider does not exist

* refactor providerDiscover struct and add scopes_supported

* fix authz endpoint
2021-09-07 13:35:23 -05:00
akshya96 f4bd14ed3f
Vault 2823 cc namespace (#12393)
* vault-2823 adding changes

* VAULT-2823 adding alias

* Vault-2823 addressing comments

* Vault-2823 removing comments

* Vault-2823 removing comments

* vault-2823 removing q debug

* adding changelog

* Vault-2823 updating external test

* adding approved changes

* fixing returns

* fixing returns
2021-09-07 09:16:12 -07:00
Nick Cabatoff 45a83d8e0f
Add code to api.RaftSnapshot to detect incomplete snapshots (#12388) 2021-09-07 11:16:37 -04:00
Scott Miller 8b20cae44f
Make necessary OSS changes for contexts in quotas (#12480)
* Make necessary OSS changes for contexts in quotas

* missed one
2021-09-02 09:54:21 -05:00
Scott Miller 0f6543fb41
Upgrade go-limiter to fix building on 1.17 (#12358)
* Upgrade go-limiter

* Modify quota system to pass contexts to upgraded go-limiter

* One more spot

* Add context vars to unit tests

* missed one
2021-09-01 16:28:47 -05:00
vinay-gopalan c99cf35b6a
[VAULT-3347] Ensure Deduplication in Provider and Client APIs in OIDC Provider (#12460)
* add deduplication for Provider

* add deduplication to provider client API

* add changelog

* delete changelog

* update comments

* update test names
2021-08-30 13:57:28 -07:00
Nick Cabatoff 0762f9003d
Refactor usages of Core in IdentityStore so they can be decoupled. (#12461) 2021-08-30 15:31:11 -04:00
vinay-gopalan 6338f3dc62
feature: OIDC Provider API (#12279) 2021-08-26 10:13:51 -07:00
John-Michael Faircloth 924cab3b06
OIDC Client API: add more test coverage (#12392)
* initial commit

* add read and delete operations

* fix bug in delete and add list unit test

* func doc typo fix

* add existence check for assignment

* remove locking on the assignment resource

It is not needed at this time.

* convert Callbacks to Operations

- convert Callbacks to Operations
- add test case for update operations

* add CRUD operations and test cases

* add client api and tests

* remove use of oidcCache

* remove use of oidcCache

* add template validation and update tests

* remove usage of oidcCache

* refactor struct and var names

* harmonize test name conventions

* refactor struct and var names

* add changelog and refactor

- add changelog
- be more explicit in the case where we do not recieve a path field

* refactor

be more explicit in the case where a field is not provided

* remove extra period from changelog

* update scope path to be OIDC provider specific

* refactor naming conventions

* update assignment path

* update scope path

* enforce key existence on client creation

* removed unused name field

* removed unused name field

* removed unused name field

* prevent assignment deletion when ref'ed by a client

* enfoce assignment existence on client create/update

* update scope template description

* error when attempting to created scope with openid reserved name

* fix UT failures after requiring assignment existence

* disallow key deletion when ref'ed by existing client

* generate client_id and client_secret on CreateOp

* do not allow key modification on client update

* return client_id and client_secret on read ops

* small refactor

* fix bug in delete assignment op

* remove client secret get call

* OIDC Client API: add more test coverage

* change name convention in tests
2021-08-23 14:08:03 -05:00
John-Michael Faircloth fec8e8b21a
feature: OIDC provider client API (#12272)
* initial commit

* add read and delete operations

* fix bug in delete and add list unit test

* func doc typo fix

* add existence check for assignment

* remove locking on the assignment resource

It is not needed at this time.

* convert Callbacks to Operations

- convert Callbacks to Operations
- add test case for update operations

* add CRUD operations and test cases

* add client api and tests

* remove use of oidcCache

* remove use of oidcCache

* add template validation and update tests

* remove usage of oidcCache

* refactor struct and var names

* harmonize test name conventions

* refactor struct and var names

* add changelog and refactor

- add changelog
- be more explicit in the case where we do not recieve a path field

* refactor

be more explicit in the case where a field is not provided

* remove extra period from changelog

* update scope path to be OIDC provider specific

* refactor naming conventions

* update assignment path

* update scope path

* enforce key existence on client creation

* removed unused name field

* removed unused name field

* removed unused name field

* prevent assignment deletion when ref'ed by a client

* enfoce assignment existence on client create/update

* update scope template description

* error when attempting to created scope with openid reserved name

* fix UT failures after requiring assignment existence

* disallow key deletion when ref'ed by existing client

* generate client_id and client_secret on CreateOp

* do not allow key modification on client update

* return client_id and client_secret on read ops

* small refactor

* fix bug in delete assignment op

* remove client secret get call
2021-08-23 08:42:31 -05:00
John-Michael Faircloth 2b3da872b3
feature: OIDC provider scope API (#12266)
* initial commit

* add read and delete operations

* fix bug in delete and add list unit test

* func doc typo fix

* add existence check for assignment

* remove locking on the assignment resource

It is not needed at this time.

* convert Callbacks to Operations

- convert Callbacks to Operations
- add test case for update operations

* add CRUD operations and test cases

* remove use of oidcCache

* remove use of oidcCache

* add template validation and update tests

* refactor struct and var names

* harmonize test name conventions

* refactor struct and var names

* add changelog and refactor

- add changelog
- be more explicit in the case where we do not recieve a path field

* refactor

be more explicit in the case where a field is not provided

* remove extra period from changelog

* update scope path to be OIDC provider specific

* update assignment path

* update scope path

* removed unused name field

* removed unused name field

* update scope template description

* error when attempting to created scope with openid reserved name
2021-08-18 13:20:27 -05:00
Hridoy Roy 2554563268
(OSS Port) Restrict Quota Deletion to Primary Cluster [vault-2399] (#12339)
* oss part of vault 2399

* Update vault/quotas/quotas.go

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* use OSS PR number as changelog entry as indicated by the changelog guide

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2021-08-17 15:34:43 -07:00
John-Michael Faircloth 40fd60342a
feature: OIDC provider assignment API (#12198)
* initial commit

* add read and delete operations

* fix bug in delete and add list unit test

* func doc typo fix

* add existence check for assignment

* remove locking on the assignment resource

It is not needed at this time.

* convert Callbacks to Operations

- convert Callbacks to Operations
- add test case for update operations

* remove use of oidcCache

* refactor struct and var names

* harmonize test name conventions

* add changelog and refactor

- add changelog
- be more explicit in the case where we do not recieve a path field

* remove extra period from changelog

* update assignment path

* removed unused name field
2021-08-17 15:55:06 -05:00
Lars Lehtonen 3bb1c68996
vault: fix dropped test errors (#12242)
* vault: fix dropped test error

* vault/activity: fix dropped test error
2021-08-16 09:52:08 -04:00
Pratyoy Mukhopadhyay 691a43030d
[VAULT-2822] Sort client count response (#12308) 2021-08-12 09:35:15 -07:00
hghaf099 f885d97774
VAULT-2285 adding capability to accept comma separated entries for au… (#12126)
* VAULT-2285 adding capability to accept comma separated entries for auth enable/tune

* Adding changelog

* Adding logic to detect invalid input parameter for auth enable config

* Updating tune.mdx

* Updating secret enable/tune for comma separated parameters

* Adding further parameter checks for auth/secret tests
Fixing changelog
using builtin type for a switch statement
Fixing a possible panic scenario

* Changing a function name, using deep.Equal instead of what reflect package provides

* Fixing auth/secret enable/tune mdx files

* One more mdx file fix

* Only when users provide a single comma separated string in a curl command, split the entries by commas

* Fixing API docs for auth/mount enable/tune for comma separated entries

* updating docs, removing an unnecessary switch case
2021-08-09 15:37:03 -04:00
John-Michael Faircloth 0d94a6530f
identity: allow creating a role with a non-existent key (#12251)
* identity: allow creating a role with a non-existent key

* remove whitespace

* add changelog
2021-08-04 11:01:13 -07:00
Hridoy Roy eaa88155d1
change advice to be used with spot warning (#12237) 2021-08-02 14:41:12 -07:00
Hridoy Roy a3fefdca35
oss part of license diagnose test fix (#12234)
* oss part of license diagnose test fix

* cl
2021-08-02 10:50:49 -07:00
Hridoy Roy 28f33d2384
Fix Diagnose Formatting In Disk Usage Checks (#12229)
* save

* fix diagnose formatting errors

* fix diagnose formatting errors

* change powers

* change powers

* use humanize instead of doing the conversion to mb manually

* cl
2021-08-02 10:06:04 -07:00
Pratyoy Mukhopadhyay 113b6885c3
[VAULT-2852] deprecate req counters in oss (#12197) 2021-07-29 10:21:40 -07:00
John-Michael Faircloth 39c744ca4e
identity: do not allow a role's token_ttl to be longer than verification_ttl (#12151)
* do not allow token_ttl to be longer than verification_ttl

* add verification when updating an existing key

When updating a key, ensure any roles referencing the key do not already
have a token_ttl greater than the key's verification_ttl

* add changelog

* remove unneeded UT check and comment

* refactor based on PR comments

- remove make slice in favor of var delcaration
- remove unneeded if check
- validate expiry value during token generation
- update changelog as bug

* refactor get roles referencing target key names logic

* add note about thread safety to helper func

* update func comment

* sort array and refactor func names

* add warning to return response

* remove unnecessary code from unit test

* Update vault/identity_store_oidc.go

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>

Co-authored-by: Austin Gebauer <34121980+austingebauer@users.noreply.github.com>
2021-07-28 20:34:52 -05:00
Pratyoy Mukhopadhyay fa29e780f0
[NO-TICKET] Upgrade protoc-gen-go to 1.26, upgrade protoc to 3.17.3 (#12171)
* [NO-TICKET] Set protoc-gen-go to 1.23, upgrade protoc to 3.17.3

* [NO-TICKET] Upgrade version of protoc-gen-go to 1.26
2021-07-28 14:51:36 -07:00
Pratyoy Mukhopadhyay 11a3c3d72d
[VAULT-2807] Count entity tokens on use (#12153) 2021-07-22 14:01:49 -07:00
Romain Aviolat c76a4e8bd1
fix: typo in function documentation (#11852) 2021-07-21 13:21:45 -07:00