Commit Graph

395 Commits

Author SHA1 Message Date
Clint b55303eddb
Combined Database Backend: Static Accounts (#6834)
* Add priority queue to sdk

* fix issue of storing pointers and now copy

* update to use copy structure

* Remove file, put Item struct def. into other file

* add link

* clean up docs

* refactor internal data structure to hide heap method implementations. Other cleanup after feedback

* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods

* updates after feedback

* refactoring/renaming

* guard against pushing a nil item

* minor updates after feedback

* Add SetCredentials, GenerateCredentials gRPC methods to combined database backend gPRC

* Initial Combined database backend implementation of static accounts and automatic rotation

* vendor updates

* initial implementation of static accounts with Combined database backend, starting with PostgreSQL implementation

* add lock and setup of rotation queue

* vendor the queue

* rebase on new method signature of queue

* remove mongo tests for now

* update default role sql

* gofmt after rebase

* cleanup after rebasing to remove checks for ErrNotFound error

* rebase cdcr-priority-queue

* vendor dependencies with 'go mod vendor'

* website database docs for Static Role support

* document the rotate-role API endpoint

* postgres specific static role docs

* use constants for paths

* updates from review

* remove dead code

* combine and clarify error message for older plugins

* Update builtin/logical/database/backend.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* cleanups from feedback

* code and comment cleanups

* move db.RLock higher to protect db.GenerateCredentials call

* Return output with WALID if we failed to delete the WAL

* Update builtin/logical/database/path_creds_create.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* updates after running 'make fmt'

* update after running 'make proto'

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* update comment and remove and rearrange some dead code

* Update website/source/api/secret/databases/index.html.md

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* cleanups after review

* Update sdk/database/dbplugin/grpc_transport.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* code cleanup after feedback

* remove PasswordLastSet; it's not used

* document GenerateCredentials and SetCredentials

* Update builtin/logical/database/path_rotate_credentials.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* wrap pop and popbykey in backend methods to protect against nil cred rotation queue

* use strings.HasPrefix instead of direct equality check for path

* Forgot to commit this

* updates after feedback

* re-purpose an outdated test to now check that static and dynamic roles cannot share a name

* check for unique name across dynamic and static roles

* refactor loadStaticWALs to return a map of name/setCredentialsWAL struct to consolidate where we're calling set credentials

* remove commented out code

* refactor to have loadstaticwals filter out wals for roles that no longer exist

* return error if nil input given

* add nil check for input into setStaticAccount

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* add constant for queue tick time in seconds, used for comparrison in updates

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Jim Kalafut <jim@kalafut.net>

* code cleanup after review

* remove misplaced code comment

* remove commented out code

* create a queue in the Factory method, even if it's never used

* update path_roles to use a common set of fields, with specific overrides for dynamic/static roles by type

* document new method

* move rotation things into a specific file

* rename test file and consolidate some static account tests

* Update builtin/logical/database/path_roles.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update builtin/logical/database/rotation.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* update code comments, method names, and move more methods into rotation.go

* update comments to be capitalized

* remove the item from the queue before we try to destroy it

* findStaticWAL returns an error

* use lowercase keys when encoding WAL entries

* small cleanups

* remove vestigial static account check

* remove redundant DeleteWAL call in populate queue

* if we error on loading role, push back to queue with 10 second backoff

* poll in initqueue to make sure the backend is setup and can write/delete data

* add revoke_user_on_delete flag to allow users to opt-in to revoking the static database user on delete of the Vault role. Default false

* add code comments on read-only loop

* code comment updates

* re-push if error returned from find static wal

* add locksutil and acquire locks when pop'ing from the queue

* grab exclusive locks for updating static roles

* Add SetCredentials and GenerateCredentials stubs to mockPlugin

* add a switch in initQueue to listen for cancelation

* remove guard on zero time, it should have no affect

* create a new context in Factory to pass on and use for closing the backend queue

* restore master copy of vendor dir
2019-06-19 14:45:39 -05:00
Jeff Mitchell 2ff5380179 Remove gogo proto from where it snuck in 2019-06-18 14:45:42 -04:00
Jeff Mitchell fde6a90edc Return integers, not floats, when reading token params 2019-06-18 12:22:12 -04:00
Jeff Mitchell c0db3df73d Update go-plugin dep 2019-06-18 10:53:38 -04:00
Jeff Mitchell 402ba1b0f0
Tokenhelper v2 (#6662)
This provides an sdk util for common token fields and parsing and plumbs it into token store roles.
2019-06-14 10:17:04 -04:00
Calvin Leung Huang 5259ec8a30
core: add ForwardGenericRequest to StaticSystemView to satisfy ExtendedSystemView (#6867) 2019-06-11 14:07:04 -07:00
Calvin Leung Huang 08e17cc111
core: add generic request forwarding bits to oss (#6866) 2019-06-11 13:13:03 -07:00
Jeff Mitchell 841616defa Remove data races around error/latency injector rand objects 2019-06-05 01:37:40 -04:00
Lexman 9aa4662cec transit cache is an Interface implemented by wrapped versions of sync… (#6225)
* transit cache is an Interface implemented by wrapped versions of syncmap and golang-lru

* transit cache is an Interface implemented by wrapped versions of syncmap and golang-lru

* changed some import paths to point to sdk

* Apply suggestions from code review

Co-Authored-By: Lexman42 <Lexman42@users.noreply.github.com>

* updates docs with information on transit/cache-config endpoint

* updates vendored files

* fixes policy tests to actually use a cache where expected and renames the struct and storage path used for cache configurations to be more generic

* updates document links

* fixed a typo in a documentation link

* changes cache_size to just size for the cache-config endpoint
2019-06-04 15:40:56 -07:00
tonyd 0570966cb9 Allow logical backends access to the disabled state of an entity (#6791)
* Allow logical backends access to the disabled state of an entity via SystemView.EntityInfo().

* Add generated file in vendor directory.
2019-05-28 16:31:50 -05:00
ncabatoff ad28263b69
Allow plugins to submit audit requests/responses via extended SystemView (#6777)
Move audit.LogInput to sdk/logical.  Allow the Data values in audited
logical.Request and Response to implement OptMarshaler, in which case
we delegate hashing/serializing responsibility to them.  Add new
ClientCertificateSerialNumber audit request field.

SystemView can now be cast to ExtendedSystemView to expose the Auditor
interface, which allows submitting requests and responses to the audit
broker.
2019-05-22 18:52:53 -04:00
Clint e80fa396d2
Add Priority Queue library to sdk (#6664)
* Add priority queue to sdk

* fix issue of storing pointers and now copy

* update to use copy structure

* Remove file, put Item struct def. into other file

* add link

* clean up docs

* refactor internal data structure to hide heap method implementations. Other cleanup after feedback

* rename PushItem and PopItem to just Push/Pop, after encapsulating the heap methods

* updates after feedback

* refactoring/renaming

* guard against pushing a nil item

* minor updates after feedback

* Add read lock to the Len() method and move the interface check into the test file

* fix a deadlock

* make the mutex a RWMutex, and make it private again

* nil check itemRaw before trying to type cast it
2019-05-16 11:15:57 -05:00
Patrick Hayes 359dbfc092 Maximum typo in Vault UI (#6743) 2019-05-16 08:44:34 +02:00
ncabatoff eb58dd958d
Copy LogInput from audit package, add OptMarshaler interface (#6735)
Adds Type field and makes Request and Response interface{}.  

Add OptMarshaler interface for doing JSON marshaling with options.
2019-05-15 09:05:30 -04:00
Jeff Mitchell ec3ea45858
Update grpc and protos (#6725)
gRPC updated to 1.20.1 which fixes a couple of important bugs.

Updates protos as well.
2019-05-13 12:09:30 -04:00
Jeff Mitchell 2f7019f5bd
Fix some regressions (#6723)
Multierror is not nil unless you return ErrorOrNil, so this was causing
non-nil errors to be returned when not expected.

Also we need to ensure we only call handleWALRollback if a function
exists.
2019-05-13 11:04:06 -04:00
Clint 970840d88c
Update handleRollback to run both PeriodicFunc and handleWALRollback (#6717) 2019-05-10 14:11:42 -05:00
ncabatoff c48936c4fd
Refactor cert util (#6676)
Break dataBundle into two pieces: inputBundle, which contains data that
is specific to the pki backend, and creationBundle, which is a more
generic bundle of validated inputs given to certificate creation/signing routines.

Move functions that only take creationBundle to certutil and make them public.
2019-05-09 11:43:11 -04:00
Mark Gritter a9e8f3e7b1
Merge branch 'master' into fix-ou-ordering 2019-05-06 14:41:44 -05:00
Jim Kalafut 944adb53d6
Don't show TypeHeader fields as being sent as headers in OpenAPI (#6679)
Fixes #6671
2019-05-03 15:12:24 -07:00
Mark Gritter 56c46b852c
Merge branch 'master' into fix-ou-ordering 2019-05-02 18:59:17 -05:00
mgritter 2d3d6a856b gofmt fixes. 2019-05-02 16:29:41 -07:00
Jim Kalafut 2835131117
Apply suggestions from code review
Co-Authored-By: mgritter <mgritter@gmail.com>
2019-05-02 18:02:15 -05:00
mgritter 4e22fb6704 Ensure OU entries are not reordered. 2019-05-02 14:31:29 -07:00
Jim Kalafut a4755ec076
Update SDK vendor (#6669) 2019-05-01 18:48:12 -07:00
Jim Kalafut 8bc9fa4583
Fix Okta auth to allow group names containing slashes (#6665)
This PR also adds CollectKeysPrefix which allows a more memory efficient
key scan for those cases where the result is immediately filtered by
prefix.
2019-05-01 14:56:18 -07:00
Jeff Mitchell e8a9d47aca
Port over some SP v2 bits (#6516)
* Port over some SP v2 bits

Specifically:

* Add too-large handling to Physical (Consul only for now)
* Contextify some identity funcs
* Update SP protos

* Add size limiting to inmem storage
2019-05-01 13:47:41 -04:00
Jeff Mitchell ebc3fda060 Update deep to pull in default full-level-diff behavior 2019-04-19 19:52:03 -04:00
Jeff Mitchell 9a7eb54a68 Merge branch '1.1.2' into master-oss 2019-04-18 18:49:49 -04:00
Brian Kassouf 43783a5dca
Move cluster logic out of vault package (#6601)
* Move cluster logic out of vault package

* Dedup heartbeat and fix tests

* Fix test
2019-04-17 13:50:31 -07:00
Jeff Mitchell 24b92d6e29 Move physical/file to sdk 2019-04-15 14:51:33 -04:00
Jeff Mitchell ffd6a87959 More rearranging of API and SDK 2019-04-15 13:38:08 -04:00
Jeff Mitchell f491851ed1 Move some things around in api/sdk 2019-04-15 12:14:20 -04:00
Jeff Mitchell c4cd113b14 Migrate database plugin methods to sdk 2019-04-15 11:36:10 -04:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell 1b5155080b Update protobufs, sdk changes 2019-04-12 23:13:14 -04:00
Jeff Mitchell 19bd861581 Update physical_util.go with entry.go 2019-04-12 22:56:04 -04:00
Jeff Mitchell 1640ebeb82 Remove mock-plugin binary causing circular dep 2019-04-12 22:22:07 -04:00
Jeff Mitchell 7a89a2f7e3 Fix more tests 2019-04-12 22:14:50 -04:00
Jeff Mitchell 80c303ac83 Move ldaputil and tlsutil over to sdk 2019-04-12 18:26:54 -04:00
Jeff Mitchell 371db36ede Move useragent to sdk 2019-04-12 18:17:49 -04:00
Jeff Mitchell a1796b3ece Move password to sdk 2019-04-12 18:12:13 -04:00
Jeff Mitchell 8d6ce1ffb5 Move policyutil to sdk 2019-04-12 18:08:46 -04:00
Jeff Mitchell 7ca424e8d2 Move cidrutil to sdk 2019-04-12 18:03:59 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00