luxolus/open-vault
2
0
Fork 0
Code Issues 1 Pull requests Releases Activity
14597 commits 1 branch 0 tags 214 MiB
Commit graph

64 commits

Author SHA1 Message Date
Josh Black d249fad2df
reformat using 'make fmt' (#13794) 2022-01-27 10:06:34 -08:00
Steven Clark b9e5aeb459
Attempt to address a data race issue within identity store - take 2 (#13476) 
* Attempt to address a data race issue within identity store
* Testcase TestIdentityStore_LocalAliasInvalidations identified a data race issue.
* This reverts the previous attempt to address the issue from #13093
 2021-12-22 09:51:13 -05:00
Steven Clark 8c50afc07c
Address a data race issue within identity_store_util::processLocalAlias (#13093) 
- When loading an existing alias within processLocalAlias we aren't
   cloning the object from the memory store. There seems to be a data
   race within the function when calling entity.UpsertAlias and
   a concurrent invalidation routine.

 ==================
WARNING: DATA RACE
Read at 0x00c00bd03d08 by goroutine 94:
  google.golang.org/protobuf/internal/impl.pointer.Elem()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/pointer_unsafe.go:118 +0x2b3
  google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:76 +0x265
  google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:56 +0x12a
  google.golang.org/protobuf/internal/impl.(*MessageInfo).size()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:40 +0x95
  google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/internal/impl/encode.go:33 +0x6c
  google.golang.org/protobuf/proto.MarshalOptions.marshal()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:153 +0x1f3
  google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend()
      /go/pkg/mod/google.golang.org/protobuf@v1.27.1/proto/encode.go:122 +0xa5
  github.com/golang/protobuf/proto.marshalAppend()
      /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:40 +0xe4
  github.com/golang/protobuf/proto.Marshal()
      /go/pkg/mod/github.com/golang/protobuf@v1.5.2/proto/wire.go:23 +0x64
  github.com/hashicorp/vault/helper/identity.(*Entity).Clone()
      /go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150
  github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn()
      /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306
  github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate()
      /go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c
  github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm()
      /go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d
  github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey()
      /go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a
  github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey()
      <autogenerated>:1 +0x7d
  github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey()
      /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390
  github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler()
      /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b
  github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453()
      /go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71

Previous write at 0x00c00bd03d08 by goroutine 52:
  github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias()
      /go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271
  github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias()
      /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672
  github.com/hashicorp/vault/vault.possiblyForwardEntityCreation()
      /go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286
  github.com/hashicorp/vault/vault.(*Core).handleLoginRequest()
      /go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a
  github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest()
      /go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11
  github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest()
      /go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5
  github.com/hashicorp/vault/vault.(*Core).HandleRequest()
      /go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2
  github.com/hashicorp/vault/http.request()
      /go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1
  github.com/hashicorp/vault/http.handleLogicalInternal.func1()
      /go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.handleRequestForwarding.func1()
      /go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  net/http.(*ServeMux).ServeHTTP()
      /usr/local/go/src/net/http/server.go:2424 +0xc5
  github.com/hashicorp/vault/http.wrapHelpHandler.func1()
      /go/src/github.com/hashicorp/vault/http/help.go:23 +0x281
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.wrapCORSHandler.func1()
      /go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1()
      /go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1()
      /go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/vault/http.wrapGenericHandler.func1()
      /go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1()
      /go/pkg/mod/github.com/hashicorp/go-cleanhttp@v0.5.2/handlers.go:42 +0xc1
  net/http.HandlerFunc.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2046 +0x4d
  net/http.serverHandler.ServeHTTP()
      /usr/local/go/src/net/http/server.go:2878 +0x89a
  net/http.initALPNRequest.ServeHTTP()
      /usr/local/go/src/net/http/server.go:3479 +0x34d
  net/http.(*initALPNRequest).ServeHTTP()
      <autogenerated>:1 +0x8f
  net/http.Handler.ServeHTTP-fm()
      /usr/local/go/src/net/http/server.go:87 +0x75
  net/http.(*http2serverConn).runHandler()
      /usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd
  net/http.(*http2serverConn).processHeaders·dwrap·31()
      /usr/local/go/src/net/http/h2_bundle.go:5562 +0x64
 2021-11-09 10:00:26 -05:00
Vishal Nayak 6eead9f09b
Fix entity alias deletion (#12834) 
* Fix entity alias deletion

* Fix tests

* Add CL
 2021-10-19 15:05:06 -04:00
Vishal Nayak 476fb08e0d
Local aliases OSS patch (#12848) 
* Local aliases OSS patch

* build fix
 2021-10-15 15:20:00 -04:00
Pratyoy Mukhopadhyay 148109b8ed
[VAULT-3252] Disallow alias creation if entity/accessor combination exists (#12747) 
* Disallow alias creation if entity/accessor combination exists

* Add changelog

* Address review comments

* Add handling to aliasUpdate, some field renaming

* Update tests to work under new entity-alias constraint

* Add check to entity merge, other review fixes

* Log duplicated accessors only once

* Fix flaky test

* Add note about new constraint to docs

* Update entity merge warn log
 2021-10-14 09:52:07 -07:00
Austin Gebauer b58913ad9f
Adds OIDC Authorization Endpoint to OIDC providers (#12538) 2021-09-27 10:55:29 -07:00
akshya96 c643dc1d53
Add Custom metadata field to alias (#12502) 
* adding changes

* removing q.Q

* removing empty lines

* testing

* checking tests

* fixing tests

* adding changes

* added requested changes

* added requested changes

* added policy templating changes and fixed tests

* adding proto changes

* making changes

* adding unit tests

* using suggested function
 2021-09-17 11:03:47 -07:00
Nick Cabatoff 0762f9003d
Refactor usages of Core in IdentityStore so they can be decoupled. (#12461) 2021-08-30 15:31:11 -04:00
Jeff Mitchell f7147025dd
Migrate to sdk/internalshared libs in go-secure-stdlib (#12090) 
* Swap sdk/helper libs to go-secure-stdlib

* Migrate to go-secure-stdlib reloadutil

* Migrate to go-secure-stdlib kv-builder

* Migrate to go-secure-stdlib gatedwriter
 2021-07-15 20:17:31 -04:00
Lars Lehtonen 53dd619d2f
vault: deprecate errwrap.Wrapf() (#11577) 2021-05-11 13:12:54 -04:00
Vishal Nayak 1e61f799ca
Use correct mount accessor when refreshing external group memberships (#11506) 
* Use correct mount accessor when refreshing external group memberships

* Add CL

* Handle the renew case properly
 2021-05-03 08:23:59 -04:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312) 
* Update tooling

* Run gofumpt

* go mod vendor
 2021-04-08 09:43:39 -07:00
Nick Cabatoff c1ddfbb538
OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
Mark Gritter a54a3b6a66
Entity and alias counts (#9262) 
* Added gauge collectors for entity counts.
* Entity and alias gauges.
* Locking around accessor to core.identityStore and core.mount.
 2020-06-23 19:45:59 -05:00
Brian Kassouf 33b3e6857e
identity: group refresh shouldn't lock unless an update is needed (#8795) 2020-04-23 11:31:22 -07:00
Brian Kassouf 6b8b02f77a
identity: Fix potential deadlock on error to load groups (#8613) 2020-03-25 09:42:23 -07:00
Mike Jarmy 5986ce922d
add counters for active service tokens, and identity entities (#7541) 2019-10-08 13:58:19 -04:00
Jeff Mitchell 8603573611 Update group alias handling to better protect against namespace differences 
Also, allow canonical ID to be switched
 2019-06-18 16:43:30 -04:00
Jeff Mitchell 1ea0c0314a
Prevent entity alias creation when entity is in different NS than mount (#943) (#6886) 2019-06-14 12:53:00 -04:00
Vishal Nayak 550f2a52bd
Storage packer V1 updates (#6531) 
* spv1 updates

* fix tests
 2019-05-07 15:29:51 -04:00
Jeff Mitchell e8a9d47aca
Port over some SP v2 bits (#6516) 
* Port over some SP v2 bits

Specifically:

* Add too-large handling to Physical (Consul only for now)
* Contextify some identity funcs
* Update SP protos

* Add size limiting to inmem storage
 2019-05-01 13:47:41 -04:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585) 
* Switch to go modules

* Make fmt
 2019-04-13 03:44:06 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Vishal Nayak 7ab753b569
Use NS context when loading entities and groups (#6563) 
* Use NS context when loading entities and groups

* update context while group loading as well

* Address review feedback
 2019-04-10 15:05:57 -04:00
Vishal Nayak f4876b744c Fix handling member group IDs (#6527) 
* Process member_group_ids only if supplied
 2019-04-05 09:12:39 -04:00
vishalnayak 3f92836771 Port identity store cleanup code 2019-02-26 16:11:16 -05:00
Martin c9918b93e8 Avoid panic at login when external group has a nil alias (#6230) 2019-02-14 12:57:20 -08:00
Vishal Nayak b4ba344782
Merge entities during unseal only on the primary (#6075) 
* Merge entities during unseal only on the primary

* Add another guard check

* Add perf standby to the check

* Make primary to not differ from case-insensitivity status w.r.t secondaries

* Ensure mutual exclusivity between loading and invalidations

* Both primary and secondaries won't persist during startup and invalidations

* Allow primary to persist when loading case sensitively

* Using core.perfStandby

* Add a tweak in core for testing

* Address review feedback

* update memdb but not storage in secondaries

* Wire all the things directly do mergeEntity

* Fix persist behavior

* Address review feedback
 2019-02-08 16:32:06 -05:00
Vishal Nayak 43e3ff808a
Update group memberships when entity is deleted (#5786) 
* Use common abstraction for entity deletion

* Update group memberships before deleting entity

* Added test

* Fix return statements

* Update comment

* Cleanup member entity IDs while loading groups

* Added test to ensure that upgrade happens properly

* Ensure that the group gets persisted if upgrade code modifies it
 2018-11-15 20:07:45 -05:00
Jeff Mitchell c01983cffd Fix two problems with entity alias updating (#5733) 
Fix two problems with entity alias updating
 2018-11-08 13:04:24 -05:00
Chris Thunes 16f52969f4 Fix memory issue caused by append of group slice to itself. (#5611) 
The slice returned by `collectGroupsReverseDFS` is an updated copy of
the slice given to it when called. Appending `pGroups` to `groups`
therefore led to expontential memory usage as the slice was repeatedly
appended to itself.

Fixes #5605
 2018-10-29 10:38:34 -04:00
Vishal Nayak c677cd0790
Case insensitive identity names (#5404) 
* case insensitive identity names

* TestIdentityStore_GroupHierarchyCases

* address review feedback

* Use errwrap.Contains instead of errwrap.ContainsType

* Warn about duplicate names all the time to help fix them

* Address review feedback
 2018-10-19 12:47:26 -07:00
Vishal Nayak 6ab030511c
Remove lookup check during alias removal (#5524) 
* Possible fix for 5348

* Fix compilation
 2018-10-18 07:53:12 -07:00
Vishal Nayak fbec18fef0
Added test for verifying member group id deletion (#5469) 2018-10-04 10:38:41 -07:00
Sebastian Plattner 782f8dedd2 Fix remove Group Member in Identity Group not working (#5466) 2018-10-04 09:27:29 -07:00
Vishal Nayak 8e66e474ca Ensure old group alias is removed when a new one is written (#5350) 2018-10-01 10:06:10 -07:00
Vishal Nayak 68a496dde4
Support operating on entities and groups by their names (#5355) 
* Support operating on entities and groups by their names

* address review feedback
 2018-09-25 12:28:28 -07:00
Jeff Mitchell 919b968c27
The big one (#5346) 2018-09-17 23:03:00 -04:00
Martin d51f3a45f7 Fix group alias loading when identity memdb is initialized (#5289) 2018-09-06 09:17:44 -07:00
Jeff Mitchell 50197d5bfd
Only write valid group alias memberships into leases (#5164) 2018-08-22 21:53:04 -04:00
Jeff Mitchell 2ed2e696a7
Merge Identity Entities if two claim the same alias (#5075) 
* Merge Identity Entities if two claim the same alias

Past bugs/race conditions meant two entities could be created each
claiming the same alias. There are planned longer term fixes for this
(outside of the race condition being fixed in 0.10.4) that involve
changing the data model, but this is an immediate workaround that has
the same net effect: if two entities claim the same alias, assume they
were created due to this race condition and merge them.

In this situation, also automatically merge policies so we don't lose
e.g. RGPs.
 2018-08-09 15:37:36 -05:00
Jeff Mitchell 7d90e6eae9 Fix typo 2018-08-08 14:38:11 -04:00
Chris Hoffman 1578c5b982 Add locking when adding aliases to existing entities (#4965) 2018-07-24 22:01:58 -04:00
Vishal Nayak ec42bc1514
avoid panic (#4841) 2018-06-27 10:16:12 -04:00
vishalnayak 235b29f1d3 Add back upsertEntityNonLocked 2018-06-24 07:54:00 -04:00
Vishal Nayak 57c7ecfcd4
Identity: Remove unused MemDB indexes and unused functions (#4817) 
* refactor delete utility

* refactor delete alias utility

* remove MemDBUpsertAlias

* Remove MemDBAliasByCanonicalID

* remove MemDBAliasesByMetadata

* remove MemDBDeleteAliasByID

* Remove MemDBUpsertEntity and MemDBEntityByNameInTxn

* Remove is.MemDBEntitiesByBucketEntryKeyHash

* Remove MemDBEntitiesByBucketEntryKeyHash and MemDBEntityByMergedEntityID

* Remove MemDBEntities

* Remove validateMemberGroupID

* Remove validateEntityID, validateGroupID, deleteAliasFromEntity

* Remove updateAliasInEntity

* Remove satisfiesMetadataFilters and UpsertGroup

* Remove MemDBUpsertGroup

* Remove deleteGroupByID

* Remove deleleGroupByName

* Remove MemDBDeleteGroupByNameInTxn

* Remove MemDBGroupsByPolicy and MemDBGroupsByPolicyInTxn

* Remove MemDBGroupIterator

* Remove MemDBGroupsByBucketEntryKeyHash

* Remove deleteGroupAlias

* Remove metadata index from entities table

* Remove unneeded indexes from entity alias and group alias schema

* Remove unneeded index from groups table schema

* Fix test

* s/entity/lockEntity

* Don't expose the memdb instance outside identity store

* More txn.Abort() corrections

* switch back to deferring abort calls
 2018-06-24 07:45:53 -04:00
Jeff Mitchell 61f2144adf Add identity store nil checks 2018-04-24 23:10:22 -04:00
Vishal Nayak 0612103c2f
external identity groups across mounts (#4365) 2018-04-17 12:01:43 -04:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252) 
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
 2018-04-05 11:49:21 -04:00