7e64e105a0
Clone identity objects to prevent races. ( #15123 )
2022-04-22 13:04:34 -04:00
2551a3e8ce
Ensure that fewer goroutines survive after a test completes ( #14197 )
...
* Various changes to try to ensure that fewer goroutines survive after a test completes:
* add Core.ShutdownWait that doesn't return until shutdown is done
* create the usedCodes cache on seal and nil it out on pre-seal so that the finalizer kills the janitor goroutine
* stop seal health checks on seal rather than wait for them to discover the active context is done
* make sure all lease-loading goroutines are done before returning from restore
* make uniquePoliciesGc discover closed quitCh immediately instead of only when the ticker fires
* make sure all loading goroutines are done before returning from loadEntities, loadCachedEntitiesOfLocalAliases
2022-02-23 10:33:52 -05:00
462a924722
identity/oidc: Adds default provider, key, and allow_all assignment ( #14119 )
2022-02-22 08:33:19 -08:00
d249fad2df
reformat using 'make fmt' ( #13794 )
2022-01-27 10:06:34 -08:00
b9e5aeb459
Attempt to address a data race issue within identity store - take 2 ( #13476 )
...
* Attempt to address a data race issue within identity store
* Testcase TestIdentityStore_LocalAliasInvalidations identified a data race issue.
* This reverts the previous attempt to address the issue from #13093
2021-12-22 09:51:13 -05:00
8c50afc07c
Address a data race issue within identity_store_util::processLocalAlias ( #13093 )
...
- When loading an existing alias within processLocalAlias we aren't
cloning the object from the memory store. There seems to be a data
race within the function when calling entity.UpsertAlias and
a concurrent invalidation routine.
==================
WARNING: DATA RACE
Read at 0x00c00bd03d08 by goroutine 94:
google.golang.org/protobuf/internal/impl.pointer.Elem()
/go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/pointer_unsafe.go:118 +0x2b3
google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow()
/go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:76 +0x265
google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer()
/go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:56 +0x12a
google.golang.org/protobuf/internal/impl.(*MessageInfo).size()
/go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:40 +0x95
google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm()
/go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:33 +0x6c
google.golang.org/protobuf/proto.MarshalOptions.marshal()
/go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:153 +0x1f3
google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend()
/go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:122 +0xa5
github.com/golang/protobuf/proto.marshalAppend()
/go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:40 +0xe4
github.com/golang/protobuf/proto.Marshal()
/go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:23 +0x64
github.com/hashicorp/vault/helper/identity.(*Entity).Clone()
/go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150
github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn()
/go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306
github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate()
/go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c
github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm()
/go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d
github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey()
/go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a
github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey()
<autogenerated>:1 +0x7d
github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey()
/go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390
github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler()
/go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b
github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453()
/go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71
Previous write at 0x00c00bd03d08 by goroutine 52:
github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias()
/go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271
github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias()
/go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672
github.com/hashicorp/vault/vault.possiblyForwardEntityCreation()
/go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286
github.com/hashicorp/vault/vault.(*Core).handleLoginRequest()
/go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a
github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest()
/go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11
github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest()
/go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5
github.com/hashicorp/vault/vault.(*Core).HandleRequest()
/go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2
github.com/hashicorp/vault/http.request()
/go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1
github.com/hashicorp/vault/http.handleLogicalInternal.func1()
/go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca
net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2046 +0x4d
github.com/hashicorp/vault/http.handleRequestForwarding.func1()
/go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb
net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2046 +0x4d
net/http.(*ServeMux).ServeHTTP()
/usr/local/go/src/net/http/server.go:2424 +0xc5
github.com/hashicorp/vault/http.wrapHelpHandler.func1()
/go/src/github.com/hashicorp/vault/http/help.go:23 +0x281
net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2046 +0x4d
github.com/hashicorp/vault/http.wrapCORSHandler.func1()
/go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e
net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2046 +0x4d
github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1()
/go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28
net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2046 +0x4d
github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1()
/go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3
net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2046 +0x4d
github.com/hashicorp/vault/http.wrapGenericHandler.func1()
/go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843
net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2046 +0x4d
github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1()
/go/pkg/mod/github.com/hashicorp/go-cleanhttp@v0.5.2/handlers.go:42 +0xc1
net/http.HandlerFunc.ServeHTTP()
/usr/local/go/src/net/http/server.go:2046 +0x4d
net/http.serverHandler.ServeHTTP()
/usr/local/go/src/net/http/server.go:2878 +0x89a
net/http.initALPNRequest.ServeHTTP()
/usr/local/go/src/net/http/server.go:3479 +0x34d
net/http.(*initALPNRequest).ServeHTTP()
<autogenerated>:1 +0x8f
net/http.Handler.ServeHTTP-fm()
/usr/local/go/src/net/http/server.go:87 +0x75
net/http.(*http2serverConn).runHandler()
/usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd
net/http.(*http2serverConn).processHeaders·dwrap·31()
/usr/local/go/src/net/http/h2_bundle.go:5562 +0x64
2021-11-09 10:00:26 -05:00
6eead9f09b
Fix entity alias deletion ( #12834 )
...
* Fix entity alias deletion
* Fix tests
* Add CL
2021-10-19 15:05:06 -04:00
476fb08e0d
Local aliases OSS patch ( #12848 )
...
* Local aliases OSS patch
* build fix
2021-10-15 15:20:00 -04:00
148109b8ed
[VAULT-3252] Disallow alias creation if entity/accessor combination exists ( #12747 )
...
* Disallow alias creation if entity/accessor combination exists
* Add changelog
* Address review comments
* Add handling to aliasUpdate, some field renaming
* Update tests to work under new entity-alias constraint
* Add check to entity merge, other review fixes
* Log duplicated accessors only once
* Fix flaky test
* Add note about new constraint to docs
* Update entity merge warn log
2021-10-14 09:52:07 -07:00
b58913ad9f
Adds OIDC Authorization Endpoint to OIDC providers ( #12538 )
2021-09-27 10:55:29 -07:00
c643dc1d53
Add Custom metadata field to alias ( #12502 )
...
* adding changes
* removing q.Q
* removing empty lines
* testing
* checking tests
* fixing tests
* adding changes
* added requested changes
* added requested changes
* added policy templating changes and fixed tests
* adding proto changes
* making changes
* adding unit tests
* using suggested function
2021-09-17 11:03:47 -07:00
0762f9003d
Refactor usages of Core in IdentityStore so they can be decoupled. ( #12461 )
2021-08-30 15:31:11 -04:00
f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib ( #12090 )
...
* Swap sdk/helper libs to go-secure-stdlib
* Migrate to go-secure-stdlib reloadutil
* Migrate to go-secure-stdlib kv-builder
* Migrate to go-secure-stdlib gatedwriter
2021-07-15 20:17:31 -04:00
53dd619d2f
vault: deprecate errwrap.Wrapf() ( #11577 )
2021-05-11 13:12:54 -04:00
1e61f799ca
Use correct mount accessor when refreshing external group memberships ( #11506 )
...
* Use correct mount accessor when refreshing external group memberships
* Add CL
* Handle the renew case properly
2021-05-03 08:23:59 -04:00
303c2aee7c
Run a more strict formatter over the code ( #11312 )
...
* Update tooling
* Run gofumpt
* go mod vendor
2021-04-08 09:43:39 -07:00
c1ddfbb538
OSS parts of the new client controlled consistency feature ( #10974 )
2021-02-24 06:58:10 -05:00
a54a3b6a66
Entity and alias counts ( #9262 )
...
* Added gauge collectors for entity counts.
* Entity and alias gauges.
* Locking around accessor to core.identityStore and core.mount.
2020-06-23 19:45:59 -05:00
33b3e6857e
identity: group refresh shouldn't lock unless an update is needed ( #8795 )
2020-04-23 11:31:22 -07:00
6b8b02f77a
identity: Fix potential deadlock on error to load groups ( #8613 )
2020-03-25 09:42:23 -07:00
5986ce922d
add counters for active service tokens, and identity entities ( #7541 )
2019-10-08 13:58:19 -04:00
8603573611
Update group alias handling to better protect against namespace differences
...
Also, allow canonical ID to be switched
2019-06-18 16:43:30 -04:00
1ea0c0314a
Prevent entity alias creation when entity is in different NS than mount ( #943 ) ( #6886 )
2019-06-14 12:53:00 -04:00
550f2a52bd
Storage packer V1 updates ( #6531 )
...
* spv1 updates
* fix tests
2019-05-07 15:29:51 -04:00
e8a9d47aca
Port over some SP v2 bits ( #6516 )
...
* Port over some SP v2 bits
Specifically:
* Add too-large handling to Physical (Consul only for now)
* Contextify some identity funcs
* Update SP protos
* Add size limiting to inmem storage
2019-05-01 13:47:41 -04:00
9ebc57581d
Switch to go modules ( #6585 )
...
* Switch to go modules
* Make fmt
2019-04-13 03:44:06 -04:00
8bcb533a1b
Create sdk/ and api/ submodules ( #6583 )
2019-04-12 17:54:35 -04:00
7ab753b569
Use NS context when loading entities and groups ( #6563 )
...
* Use NS context when loading entities and groups
* update context while group loading as well
* Address review feedback
2019-04-10 15:05:57 -04:00
f4876b744c
Fix handling member group IDs ( #6527 )
...
* Process member_group_ids only if supplied
2019-04-05 09:12:39 -04:00
3f92836771
Port identity store cleanup code
2019-02-26 16:11:16 -05:00
c9918b93e8
Avoid panic at login when external group has a nil alias ( #6230 )
2019-02-14 12:57:20 -08:00
b4ba344782
Merge entities during unseal only on the primary ( #6075 )
...
* Merge entities during unseal only on the primary
* Add another guard check
* Add perf standby to the check
* Make primary to not differ from case-insensitivity status w.r.t secondaries
* Ensure mutual exclusivity between loading and invalidations
* Both primary and secondaries won't persist during startup and invalidations
* Allow primary to persist when loading case sensitively
* Using core.perfStandby
* Add a tweak in core for testing
* Address review feedback
* update memdb but not storage in secondaries
* Wire all the things directly do mergeEntity
* Fix persist behavior
* Address review feedback
2019-02-08 16:32:06 -05:00
43e3ff808a
Update group memberships when entity is deleted ( #5786 )
...
* Use common abstraction for entity deletion
* Update group memberships before deleting entity
* Added test
* Fix return statements
* Update comment
* Cleanup member entity IDs while loading groups
* Added test to ensure that upgrade happens properly
* Ensure that the group gets persisted if upgrade code modifies it
2018-11-15 20:07:45 -05:00
c01983cffd
Fix two problems with entity alias updating ( #5733 )
...
Fix two problems with entity alias updating
2018-11-08 13:04:24 -05:00
16f52969f4
Fix memory issue caused by append of group slice to itself. ( #5611 )
...
The slice returned by `collectGroupsReverseDFS` is an updated copy of
the slice given to it when called. Appending `pGroups` to `groups`
therefore led to expontential memory usage as the slice was repeatedly
appended to itself.
Fixes #5605
2018-10-29 10:38:34 -04:00
c677cd0790
Case insensitive identity names ( #5404 )
...
* case insensitive identity names
* TestIdentityStore_GroupHierarchyCases
* address review feedback
* Use errwrap.Contains instead of errwrap.ContainsType
* Warn about duplicate names all the time to help fix them
* Address review feedback
2018-10-19 12:47:26 -07:00
6ab030511c
Remove lookup check during alias removal ( #5524 )
...
* Possible fix for 5348
* Fix compilation
2018-10-18 07:53:12 -07:00
fbec18fef0
Added test for verifying member group id deletion ( #5469 )
2018-10-04 10:38:41 -07:00
782f8dedd2
Fix remove Group Member in Identity Group not working ( #5466 )
2018-10-04 09:27:29 -07:00
8e66e474ca
Ensure old group alias is removed when a new one is written ( #5350 )
2018-10-01 10:06:10 -07:00
68a496dde4
Support operating on entities and groups by their names ( #5355 )
...
* Support operating on entities and groups by their names
* address review feedback
2018-09-25 12:28:28 -07:00
919b968c27
The big one ( #5346 )
2018-09-17 23:03:00 -04:00
d51f3a45f7
Fix group alias loading when identity memdb is initialized ( #5289 )
2018-09-06 09:17:44 -07:00
50197d5bfd
Only write valid group alias memberships into leases ( #5164 )
2018-08-22 21:53:04 -04:00
2ed2e696a7
Merge Identity Entities if two claim the same alias ( #5075 )
...
* Merge Identity Entities if two claim the same alias
Past bugs/race conditions meant two entities could be created each
claiming the same alias. There are planned longer term fixes for this
(outside of the race condition being fixed in 0.10.4) that involve
changing the data model, but this is an immediate workaround that has
the same net effect: if two entities claim the same alias, assume they
were created due to this race condition and merge them.
In this situation, also automatically merge policies so we don't lose
e.g. RGPs.
2018-08-09 15:37:36 -05:00
7d90e6eae9
Fix typo
2018-08-08 14:38:11 -04:00
1578c5b982
Add locking when adding aliases to existing entities ( #4965 )
2018-07-24 22:01:58 -04:00
ec42bc1514
avoid panic ( #4841 )
2018-06-27 10:16:12 -04:00
235b29f1d3
Add back upsertEntityNonLocked
2018-06-24 07:54:00 -04:00
57c7ecfcd4
Identity: Remove unused MemDB indexes and unused functions ( #4817 )
...
* refactor delete utility
* refactor delete alias utility
* remove MemDBUpsertAlias
* Remove MemDBAliasByCanonicalID
* remove MemDBAliasesByMetadata
* remove MemDBDeleteAliasByID
* Remove MemDBUpsertEntity and MemDBEntityByNameInTxn
* Remove is.MemDBEntitiesByBucketEntryKeyHash
* Remove MemDBEntitiesByBucketEntryKeyHash and MemDBEntityByMergedEntityID
* Remove MemDBEntities
* Remove validateMemberGroupID
* Remove validateEntityID, validateGroupID, deleteAliasFromEntity
* Remove updateAliasInEntity
* Remove satisfiesMetadataFilters and UpsertGroup
* Remove MemDBUpsertGroup
* Remove deleteGroupByID
* Remove deleleGroupByName
* Remove MemDBDeleteGroupByNameInTxn
* Remove MemDBGroupsByPolicy and MemDBGroupsByPolicyInTxn
* Remove MemDBGroupIterator
* Remove MemDBGroupsByBucketEntryKeyHash
* Remove deleteGroupAlias
* Remove metadata index from entities table
* Remove unneeded indexes from entity alias and group alias schema
* Remove unneeded index from groups table schema
* Fix test
* s/entity/lockEntity
* Don't expose the memdb instance outside identity store
* More txn.Abort() corrections
* switch back to deferring abort calls
2018-06-24 07:45:53 -04:00
Nick Cabatoff