Commit graph

1336 commits

Author SHA1 Message Date
Calvin Leung Huang 79cb82e133
Add logic for using Auth.Period when handling auth login/renew requests (#3677)
* Add logic for using Auth.Period when handling auth login/renew requests

* Set auth.TTL if not set in handleLoginRequest

* Always set auth.TTL = te.TTL on handleLoginRequest, check TTL and period against sys values on RenewToken

* Get sysView from le.Path, revert tests

* Add back auth.Policies

* Fix TokenStore tests, add resp warning when capping values

* Use switch for ttl/period check on RenewToken

* Move comments around
2017-12-15 13:30:05 -05:00
Brian Kassouf afe53eb862
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Brian Kassouf de9b7d779d
Fix leaking connections on cluster port (#3680) 2017-12-12 17:18:04 -08:00
Chris Hoffman 822ce95dc4
adding ability to override temp dir in dev cluster (#3673) 2017-12-11 18:02:35 -05:00
lemondrank 255212af23 Non-recursive DFS token tree revoke (#2478) 2017-12-11 16:51:37 -05:00
Vishal Nayak 0928a65c38
remove unused function (#3657) 2017-12-06 18:55:43 -05:00
Chris Hoffman f966d20225
Adding ability to cache core values, cache transaction improvements (#3640) 2017-12-06 12:25:17 -05:00
Jeff Mitchell 8f159b12b1
allowed/disallowed_policies as TypeCommaStringSlice (#3641)
Our docs apparently claim that this is a list, but the code is
string-only. This fixes that discrepancy.
2017-12-04 12:47:05 -05:00
Jeff Mitchell 20320cd71e
Move location of quit channel closing in exp manager (#3638)
* Move location of quit channel closing in exp manager

If it happens after stopping timers any timers firing before all timers
are stopped will still run the revocation function. With plugin
auto-crash-recovery this could end up instantiating a plugin that could
then try to unwrap a token from a nil token store.

This also plumbs in core so that we can grab a read lock during the
operation and check standby/sealed status before running it (after
grabbing the lock).

* Use context instead of checking core values directly

* Use official Go context in a few key places
2017-12-01 17:08:38 -05:00
Jeff Mitchell 548629e8ef Port over some changes 2017-11-30 09:43:07 -05:00
Jeff Mitchell eef6afec3e
Fix mount path for credential values in aliases (#3580) 2017-11-14 01:31:10 -05:00
Calvin Leung Huang 2f9ff5c062 Acquire state lock at the start of UnsealWithRecoveryKeys (#3579)
* Acquire state lock from the start of UnsealWithRecoveryKeys, add metrics

* Change metrics variable
2017-11-13 22:05:22 -05:00
Vishal Nayak 5d976794d4
API refactoring and doc updates (#3577)
* Doc updates and API refactoring

* fix tests

* change metadata fieldtype to TypeKVPairs

* Give example for TypeKVPairs in CLI for metadata

* Update API docs examples to reflect the native expected value for TypeKVPairs

* Don't mention comma separation in the docs for TypeCommaStringSlice

* s/groups/group; s/entities/entity; s/entity-aliases/entity-alias; s/group-aliases/group-alias

* Address review feedback

* Fix formatting

* fix sidebar links
2017-11-13 20:59:42 -05:00
Chris Hoffman 9e79e9b397
generate token functions to share common names (#3576) 2017-11-13 15:44:26 -05:00
Jeff Mitchell 2b78bc2a9b
Port over bits (#3575) 2017-11-13 15:31:32 -05:00
Jeff Mitchell d55f94f4a3
Plumb more seal wrap stuff through and move to outside layer of mount options (#3572) 2017-11-13 11:22:22 -05:00
Jeff Mitchell 0798ccdd7c Populate config error in three node test function 2017-11-10 17:44:25 -05:00
Jeff Mitchell e66c4b11d5 Add core numbers to output in dev three node 2017-11-10 16:21:46 -05:00
Brian Kassouf ab3b625a3b
Add API methods for creating a DR Operation Token and make generate root accept strategy types (#3565)
* Add API and Command code for generating a DR Operation Token

* Update generate root to accept different token strategies
2017-11-10 10:19:42 -08:00
Jeff Mitchell c3b382d010 If kv entry is sealwrapped and is response-wrapped ensure it carries through 2017-11-10 12:56:31 -05:00
Brian Shumate a7af6c4cb7 Debug level logging of socket audit backend config at startup (#3560) 2017-11-10 12:08:36 -05:00
Jeff Mitchell 18f76d0909 Change test cluster to request client cert rather than verify if given 2017-11-09 15:55:23 -05:00
Jeff Mitchell fdaed84e82
Add ability to send seal wrap response info into cubbyhole request. (#3562)
Ref 84f80db4bf499ce7345615cc2def77e7d48bc690
2017-11-09 12:47:42 -05:00
Jeff Mitchell 922ac56222
Seal wrap requesting in logical.Request wrapinfo (#3559) 2017-11-09 10:32:49 -05:00
Vishal Nayak 660c9ab382
Merge identity alias lookups into either entity or group lookup endpoints (#3538)
* merge identity alias lookups into either entity or group lookups

* Address review feedback

* address review feedback
2017-11-09 01:29:19 -05:00
Chris Hoffman a7f510c9b4 converting identity metadata to use TypeKVPairs (#3549) 2017-11-08 14:51:40 -05:00
Jeff Mitchell dd551eb12f Put back original test cluster client redirect behavior 2017-11-08 10:15:56 -05:00
Jeff Mitchell 0c9e692414
Minor mount logic updates (#3553) 2017-11-07 20:30:02 -05:00
Jeff Mitchell febda30382
Fix potential panic reading local mount tables (#3552) 2017-11-07 18:04:37 -05:00
Jeff Mitchell 98dc8e9bea Fix regression involving cluster listener 2017-11-07 17:27:13 -05:00
Jeff Mitchell a2db5671c5 Add logbridge for testing 2017-11-07 15:50:09 -05:00
Calvin Leung Huang d9eaacf5de
Barrier unseal using recovery keys (#3541)
* Barrier unseal using recovery keys

* Remove tests
2017-11-07 15:15:39 -05:00
Jonathan Freedman 4109473134 More Mount Conflict Detection (#2919) 2017-11-06 15:29:09 -05:00
Lars Lehtonen 8ccdaa1860 Fix swallowed errors in TestRollbackManager_Join() (#3327) 2017-11-06 14:34:27 -05:00
Jeff Mitchell 119607dcb7
Seal wrap all root tokens and their leases (#3540) 2017-11-06 13:10:36 -05:00
Vishal Nayak 2af5b9274f
Return group memberships of entity during read (#3526)
* return group memberships of entity during read

* Add implied group memberships to read response of entity

* distinguish between all, direct and inherited group IDs of an entity

* address review feedback

* address review feedback

* s/implied/inherited in tests
2017-11-06 13:01:48 -05:00
Chris Hoffman 1e03e1bad3
Fix group/policy iterators with multiple groups (#3527)
* fixing some group iterators

* fix slice rewrite
2017-11-03 11:26:22 -04:00
Vishal Nayak e4e4a7ba67
Capabilities responds considering policies on entities and groups (#3522)
* Capabilities endpoint will now return considering policies on entities and groups

* refactor the policy derivation into a separate function

* Docs: Update docs to reflect the change in capabilities endpoint
2017-11-03 11:20:10 -04:00
Chris Hoffman 16059b4e94
Fix policy lookup when entity is part of multiple groups (#3524) 2017-11-03 07:19:29 -04:00
Vishal Nayak 707e270f54
Added lookup endpoint for entity (#3519)
* Added lookup endpoint for entity

* Address review comments
2017-11-02 16:38:15 -04:00
Vishal Nayak 7bae606662
External identity groups (#3447)
* external identity groups

* add local LDAP groups as well to group aliases

* add group aliases for okta credential backend

* Fix panic in tests

* fix build failure

* remove duplicated struct tag

* add test steps to test out removal of group member during renewals

* Add comment for having a prefix check in router

* fix tests

* s/parent_id/canonical_id

* s/parent/canonical in comments and errors
2017-11-02 16:05:48 -04:00
Jeff Mitchell d229d7d5b0
Redo API locking (#3508)
* Redo the API client quite a bit to make the behavior of NewClient more
predictable and add locking to make it safer to use with Clone() and if
multiple goroutines for some reason decide to change things.

Along the way I discovered that currently, the x/net/http2 package is
broke with the built-in h2 support in released Go. For those using
DefaultConfig (the vast majority of cases) this will be a non-event.
Others can manually call http2.ConfigureTransport as needed. We should
keep an eye on commits on that repo and consider more updates before
release. Alternately we could go back revisions but miss out on bug
fixes; my theory is that this is not a purposeful break and I'll be
following up on this in the Go issue tracker.

In a few tests that don't use NewTestCluster, either for legacy or other
reasons, ensure that http2.ConfigureTransport is called.

* Use tls config cloning

* Don't http2.ConfigureServer anymore as current Go seems to work properly without requiring the http2 package

* Address feedback
2017-11-02 09:30:04 -05:00
Jeff Mitchell 1ba4e828b4
Ensure revocation happens before seal/step-down since token store isn't (#3500)
available after when using single-use tokens.

Fixes #3497
2017-11-02 08:47:02 -05:00
Jeff Mitchell 3be99466d6
Change some instances of adding headers to setting headers, since really (#3501)
we want to replace anything that might be there (e.g. for request
forwarding and content-type).

Hopefully fixes #3485
2017-11-02 07:31:50 -05:00
Chris Hoffman 3d8d887676
Add ability to require parameters in ACLs (#3510) 2017-11-02 07:18:49 -04:00
Jeff Mitchell e670447947 Use an atomic store in expiration loading test to fix race detector 2017-11-01 15:52:59 -04:00
Vishal Nayak 859cdd35fd
fix deadlock while loading groups (#3515) 2017-11-01 14:14:21 -04:00
Jeff Mitchell d548181d50
Fix memory leak when a connection would hit the cluster port and go away (#3513) 2017-10-31 20:58:45 -05:00
Jeff Mitchell bb924a404f Update seal type names 2017-10-27 17:28:50 -04:00
Jeff Mitchell 03cd40345f Properly format autogenerated clusteraddr 2017-10-25 14:43:05 -04:00
Jeff Mitchell d9bef0abca Update storedBarrierKeysPath name 2017-10-25 11:59:02 -04:00
Jeff Mitchell cf1cf8eee4 If no clusterAddr is given but we have a single synthesized cluster addresses, automatically use it. (#3486) 2017-10-24 16:58:53 -04:00
Seth Vargo c5665920f6
Standardize on "auth method"
This removes all references I could find to:

- credential provider
- authentication backend
- authentication provider
- auth provider
- auth backend

in favor of the unified:

- auth method
2017-10-24 09:32:15 -04:00
Jeff Mitchell a80791ce7e Update proto 2017-10-23 17:45:05 -04:00
Jeff Mitchell d38a699c32 Make compile 2017-10-23 17:15:56 -04:00
Jeff Mitchell 8e9317792d Fix some merge/update bugs 2017-10-23 16:49:46 -04:00
Jeff Mitchell c144f95be0 Sync over 2017-10-23 16:43:07 -04:00
Jeff Mitchell aaa1094240 Remove extra curly brace 2017-10-23 16:06:27 -04:00
Jeff Mitchell b7ffe7ec99 Sync over 2017-10-23 16:03:36 -04:00
Jeff Mitchell 3c6fe40a91 Sync 2017-10-23 15:35:28 -04:00
Jeff Mitchell d7c5a3acfc Remove some diffing cruft 2017-10-23 15:13:04 -04:00
Jeff Mitchell 40643aa14c Remove some old policy upgrade code 2017-10-23 15:11:29 -04:00
Jeff Mitchell 47dae8ffc7 Sync 2017-10-23 14:59:37 -04:00
Jeff Mitchell d6a9a770c1 Sync some seal testing stuff 2017-10-23 13:42:04 -04:00
Jeff Mitchell 14b832fde0 Sync some changes 2017-10-23 12:50:34 -04:00
Chris Hoffman e4065e33d2 copying general purpose tools from transit backend to /sys/tools (#3391) 2017-10-20 10:59:17 -04:00
Seth Vargo 6fd9d11c79 Allow more complex errors from plugins (no interface change) (#3444)
* Allow more complex errors from plugins

This enables more complex types to be registered and returned from plugins.

*  Register common error types

This is a slightly less drastic change, which keeps the HTTPCodedError
as an interface.

* Remove replication error from list
2017-10-19 13:29:59 -07:00
Vishal Nayak ea1f552d48 set entity id on the token (#3468) 2017-10-18 13:23:05 -04:00
Vishal Nayak 6f531986c4 remove premium version checks for identity store (#3462) 2017-10-17 14:08:51 -04:00
Vishal Nayak f7ed6732a5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Chris Hoffman 375587fbf2 Removing reset of quit channel (#3376) 2017-09-26 20:29:42 -04:00
Chris Hoffman 1029ad3b33 Rename "generic" secret backend to "kv" (#3292) 2017-09-15 09:02:29 -04:00
Chris Hoffman 9d73c81f38 Disable the sys/raw endpoint by default (#3329)
* disable raw endpoint by default

* adding docs

* config option raw -> raw_storage_endpoint

* docs updates

* adding listing on raw endpoint

* reworking tests for enabled raw endpoints

* root protecting base raw endpoint
2017-09-15 00:21:35 -04:00
Chris Hoffman 91338d7aa2 Adding latency injector option to -dev mode for storage operations (#3289) 2017-09-11 14:49:08 -04:00
Chris Hoffman 2002543245 Switching atomic vars to int32 to address panic on some architectures (#3314) 2017-09-11 11:17:22 -04:00
Jeff Mitchell 675cbe1bcd Handle expiration manager being nil 2017-09-05 12:01:02 -04:00
Chris Hoffman 71952b7738 ExpirationManager restoration to load in the background (#3260) 2017-09-05 11:09:00 -04:00
Jeff Mitchell dae06d9a0e Simplify a lot of the mount tuning code (#3285) 2017-09-05 10:57:25 -04:00
Brian Kassouf a8d9426d9f Update locking components from DR replication changes (#3283)
* Update locking components from DR replication changes

* Fix plugin backend test

* Add a comment about needing the statelock:
2017-09-04 19:38:37 -04:00
Jeff Mitchell d7e933a072 Fix inverted skipinit logic 2017-09-04 18:39:57 -04:00
Jeff Mitchell 4785e56283 Fix nil pointer on test 2017-09-04 18:08:56 -04:00
Jeff Mitchell 0da8fa412d DoNotInit -> SkipInit 2017-09-04 16:29:43 -04:00
Jeff Mitchell cbb0313f78 Add option to NewTestCluster to not init 2017-09-04 16:28:21 -04:00
Calvin Leung Huang 48263622cb Add comment to the reason behind mounting a nil backend for plugin mounts during setup 2017-09-01 12:26:31 -04:00
Calvin Leung Huang a581e96b78 Lazy-load plugin mounts (#3255)
* Lazy load plugins to avoid setup-unwrap cycle

* Remove commented blocks

* Refactor NewTestCluster, use single core cluster on basic plugin tests

* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly

* Add special path to mock plugin

* Move ensureCoresSealed to vault/testing.go

* Use same method for EnsureCoresSealed and Cleanup

* Bump ensureCoresSealed timeout to 60s

* Correctly handle nil opts on NewTestCluster

* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap

* Check metadata flag directly on the plugin process

* Plumb isMetadataMode down to PluginRunner

* Add NOOP shims when running in metadata mode

* Remove unused flag from the APIMetadata object

* Remove setupSecretPlugins and setupCredentialPlugins functions

* Move when we setup rollback manager to after the plugins are initialized

* Fix tests

* Fix merge issue

* start rollback manager after the credential setup

* Add guards against running certain client and server functions while in metadata mode

* Call initialize once a plugin is loaded on the fly

* Add more tests, update basic secret/auth plugin tests to trigger lazy loading

* Skip mount if plugin removed from catalog

* Fixup

* Remove commented line on LookupPlugin

* Fail on mount operation if plugin is re-added to catalog and mount is on existing path

* Check type and special paths on startBackend

* Fix merge conflicts

* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 01:02:03 -04:00
Jeff Mitchell 4ec737b013 Don't append a trailing slash to the request path if it doesn't actually help find something (#3271) 2017-08-31 16:50:03 -04:00
Calvin Leung Huang 6f417d39da Normalize plugin_name option for mount and enable-auth (#3202) 2017-08-31 12:16:59 -04:00
Jeff Mitchell 3edb337a00 Add option to set cluster TLS cipher suites. (#3228)
* Add option to set cluster TLS cipher suites.

Fixes #3227
2017-08-30 16:28:23 -04:00
Jeff Mitchell 654e7d92ac Properly lowercase policy names. (#3210)
Previously we lowercased names on ingress but not on lookup or delete
which could cause unexpected results. Now, just unilaterally lowercase
policy names on write and delete. On get, to avoid the performance hit
of always lowercasing when not necessary since it's in the critical
path, we have a minor optimization -- we check the LRU first before
normalizing. For tokens, because they're already normalized when adding
policies during creation, this should always work; it might just be
slower for API calls.

Fixes #3187
2017-08-18 19:47:23 -04:00
Jeff Mitchell 411419cbf8 plugins/backend/reload -> plugins/reload/backend (#3186) 2017-08-16 12:40:38 -04:00
Jeff Mitchell c34a5b2e93 * Add ability to specify a plugin dir in dev mode (#3184)
* Change (with backwards compatibility) sha_256 to sha256 for plugin
registration
2017-08-16 11:17:50 -04:00
Calvin Leung Huang 86ea7e945d Add plugin auto-reload capability (#3171)
* Add automatic plugin reload

* Refactor builtin/backend

* Remove plugin reload at the core level

* Refactor plugin tests

* Add auto-reload test case

* Change backend to use sync.RWMutex, fix dangling test plugin processes

* Add a canary to plugin backends to avoid reloading many times (#3174)

* Call setupPluginCatalog before mount-related operations in postUnseal

* Don't create multiple system backends since core only holds a reference (#3176)

to one.
2017-08-15 22:10:32 -04:00
Jeff Mitchell 48bf1d6edc Add core sealing to test cluster shutdown 2017-08-15 17:06:38 -04:00
Jeff Mitchell c864c0bad5 Return 500 if existence check fails, not 400 (#3173)
Fixes #3162
2017-08-15 16:44:16 -04:00
Chris Hoffman 6092538511 splitting cache into transactional and non-transactional cache structs (#3132) 2017-08-08 20:47:14 -04:00
Calvin Leung Huang 6853d80212 Add plugin backend reload capability (#3112)
* Add plugin reload capability on all mounts for a specific plugin type

* Comments cleanup

* Add per-mount plugin backend reload, add tests

* Fix typos

* Remove old comment

* Reuse existing storage view in reloadPluginCommon

* Correctly handle reloading auth plugin backends

* Update path to plugin/backend/reload

* Use multierrors on reloadMatchingPluginMounts, attempt to reload all mounts provided

* Use internal value as check to ensure plugin backend reload

* Remove connection state from request for plugins at the moment

* Minor cleanup

* Refactor tests
2017-08-08 00:18:59 -04:00
Jeff Mitchell 691d00149a Fix exporting stdAllowedHeaders 2017-08-07 15:02:08 -04:00
Aaron Salvo ad1d74cae0 Set allowed headers via API instead of defaulting to wildcard. (#3023) 2017-08-07 10:03:30 -04:00
Tony Cai 12a1681122 Fix minor typo (#3120) 2017-08-07 09:52:49 -04:00
Jeff Mitchell c2c908cdc0 Unlock the statelock on unsuccessful sealInitCommon 2017-08-04 16:49:31 -04:00