Commit graph

291 commits

Author SHA1 Message Date
Brian Kassouf 34d958968d Revert stopoptions (#4088)
* Use an atomic value instead to communicate whether to give up HA lock

* Remove now-unneeded StopOptions struct

* Add a channel based mutex acquisition function to avoid a deadlock

* Make periodic leader refresh only spawn a single waiting goroutine and
extend logic to the key upgrade check
2018-03-06 21:35:58 -05:00
Jeff Mitchell 6b777dc63e
Rearrange shutdown logic to remove some lock switching (#4083)
* Rearrange shutdown logic to remove some lock switching and to allow
giving up the HA lock to be skipped.
2018-03-06 18:06:09 -05:00
Calvin Leung Huang e2fb199ce5
Non-HMAC audit values (#4033)
* Add non-hmac request keys

* Update comment

* Initial audit request keys implementation

* Add audit_non_hmac_response_keys

* Move where req.NonHMACKeys gets set

* Minor refactor

* Add params to auth tune endpoints

* Sync cache on loadCredentials

* Explicitly unset req.NonHMACKeys

* Do not error if entry is nil

* Add tests

* docs: Add params to api sections

* Refactor audit.Backend and Formatter interfaces, update audit broker methods

* Add audit_broker.go

* Fix method call params in audit backends

* Remove fields from logical.Request and logical.Response, pass keys via LogInput

* Use data.GetOk to allow unsetting existing values

* Remove debug lines

* Add test for unsetting values

* Address review feedback

* Initialize values in FormatRequest and FormatResponse using input values

* Update docs

* Use strutil.StrListContains

* Use strutil.StrListContains
2018-03-02 12:18:39 -05:00
Jeff Mitchell f1bd0cbe74
Use atomic values in seal to avoid some data races (#4040) 2018-02-23 17:18:48 -05:00
Jeff Mitchell d4a431b298
Move local cluster parameters to atomic values to fix some potential data races (#4036) 2018-02-23 14:47:07 -05:00
Jeff Mitchell 96ea0620fd
Add sealunwrapper to ease OSS downgrades (#3936) 2018-02-09 16:37:40 -05:00
Jeff Mitchell 60e2209532
Remove core restriction in cache and turn it into an active/standby restriction instead (#3849) 2018-01-25 22:21:51 -05:00
Jeff Mitchell ad9da2e0b8
Fix intermittent panic by storing a reference to the grpc server (#3842)
* Fix intermittent panic by storing a reference to the grpc server and
using that to ensure it will never be nil.

* Just get rid of c.rpcServer
2018-01-24 20:23:08 -05:00
Jeff Mitchell 8e8675053b Sync some bits over 2018-01-22 21:44:49 -05:00
Jeff Mitchell 7d6fed2e86
Use a separate var for active node replication state (#3819) 2018-01-19 19:24:04 -05:00
Jeff Mitchell 69aead14f0 A bit more context plumbing 2018-01-19 04:11:59 -05:00
Jeff Mitchell 33b68ebf3d Remove context from a few extraneous places 2018-01-19 03:44:06 -05:00
Jeff Mitchell 0f7e3bb79b Add context to performPolicyChecks 2018-01-19 02:43:39 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Calvin Leung Huang bffaf9b0c3 Remove vlogger, add log.Logger to PolicyStore struct (#3813) 2018-01-18 01:01:51 -05:00
Jeff Mitchell 5371b0489f Rename var from last commit 2018-01-17 23:08:35 -05:00
Jeff Mitchell 307a177540 Make heartbeatInterval a package var to allow tests to modify it 2018-01-17 23:05:11 -05:00
Jeff Mitchell 88313f0398 Update replication state logic.
Fixes #3727
2018-01-16 13:51:55 -05:00
Brian Kassouf 1c190d4bda
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Jeff Mitchell 6681329041 Update redirect address error to be more clear 2018-01-04 10:45:40 -05:00
Jeff Mitchell 1fe494e8e1
Port IsDRSecondary over and enable returning it via sys_health (#3749) 2018-01-03 15:07:13 -05:00
Jeff Mitchell 92aaebd20e Bust recovery cache after unsealed is set true to force migration 2017-12-19 16:22:09 -05:00
Chris Hoffman f966d20225
Adding ability to cache core values, cache transaction improvements (#3640) 2017-12-06 12:25:17 -05:00
Jeff Mitchell 20320cd71e
Move location of quit channel closing in exp manager (#3638)
* Move location of quit channel closing in exp manager

If it happens after stopping timers any timers firing before all timers
are stopped will still run the revocation function. With plugin
auto-crash-recovery this could end up instantiating a plugin that could
then try to unwrap a token from a nil token store.

This also plumbs in core so that we can grab a read lock during the
operation and check standby/sealed status before running it (after
grabbing the lock).

* Use context instead of checking core values directly

* Use official Go context in a few key places
2017-12-01 17:08:38 -05:00
Calvin Leung Huang 2f9ff5c062 Acquire state lock at the start of UnsealWithRecoveryKeys (#3579)
* Acquire state lock from the start of UnsealWithRecoveryKeys, add metrics

* Change metrics variable
2017-11-13 22:05:22 -05:00
Calvin Leung Huang d9eaacf5de
Barrier unseal using recovery keys (#3541)
* Barrier unseal using recovery keys

* Remove tests
2017-11-07 15:15:39 -05:00
Vishal Nayak e4e4a7ba67
Capabilities responds considering policies on entities and groups (#3522)
* Capabilities endpoint will now return considering policies on entities and groups

* refactor the policy derivation into a separate function

* Docs: Update docs to reflect the change in capabilities endpoint
2017-11-03 11:20:10 -04:00
Vishal Nayak 7bae606662
External identity groups (#3447)
* external identity groups

* add local LDAP groups as well to group aliases

* add group aliases for okta credential backend

* Fix panic in tests

* fix build failure

* remove duplicated struct tag

* add test steps to test out removal of group member during renewals

* Add comment for having a prefix check in router

* fix tests

* s/parent_id/canonical_id

* s/parent/canonical in comments and errors
2017-11-02 16:05:48 -04:00
Jeff Mitchell 1ba4e828b4
Ensure revocation happens before seal/step-down since token store isn't (#3500)
available after when using single-use tokens.

Fixes #3497
2017-11-02 08:47:02 -05:00
Jeff Mitchell aaa1094240 Remove extra curly brace 2017-10-23 16:06:27 -04:00
Jeff Mitchell b7ffe7ec99 Sync over 2017-10-23 16:03:36 -04:00
Vishal Nayak ea1f552d48 set entity id on the token (#3468) 2017-10-18 13:23:05 -04:00
Vishal Nayak f7ed6732a5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Chris Hoffman 1029ad3b33 Rename "generic" secret backend to "kv" (#3292) 2017-09-15 09:02:29 -04:00
Chris Hoffman 9d73c81f38 Disable the sys/raw endpoint by default (#3329)
* disable raw endpoint by default

* adding docs

* config option raw -> raw_storage_endpoint

* docs updates

* adding listing on raw endpoint

* reworking tests for enabled raw endpoints

* root protecting base raw endpoint
2017-09-15 00:21:35 -04:00
Brian Kassouf a8d9426d9f Update locking components from DR replication changes (#3283)
* Update locking components from DR replication changes

* Fix plugin backend test

* Add a comment about needing the statelock:
2017-09-04 19:38:37 -04:00
Calvin Leung Huang a581e96b78 Lazy-load plugin mounts (#3255)
* Lazy load plugins to avoid setup-unwrap cycle

* Remove commented blocks

* Refactor NewTestCluster, use single core cluster on basic plugin tests

* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly

* Add special path to mock plugin

* Move ensureCoresSealed to vault/testing.go

* Use same method for EnsureCoresSealed and Cleanup

* Bump ensureCoresSealed timeout to 60s

* Correctly handle nil opts on NewTestCluster

* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap

* Check metadata flag directly on the plugin process

* Plumb isMetadataMode down to PluginRunner

* Add NOOP shims when running in metadata mode

* Remove unused flag from the APIMetadata object

* Remove setupSecretPlugins and setupCredentialPlugins functions

* Move when we setup rollback manager to after the plugins are initialized

* Fix tests

* Fix merge issue

* start rollback manager after the credential setup

* Add guards against running certain client and server functions while in metadata mode

* Call initialize once a plugin is loaded on the fly

* Add more tests, update basic secret/auth plugin tests to trigger lazy loading

* Skip mount if plugin removed from catalog

* Fixup

* Remove commented line on LookupPlugin

* Fail on mount operation if plugin is re-added to catalog and mount is on existing path

* Check type and special paths on startBackend

* Fix merge conflicts

* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 01:02:03 -04:00
Jeff Mitchell 3edb337a00 Add option to set cluster TLS cipher suites. (#3228)
* Add option to set cluster TLS cipher suites.

Fixes #3227
2017-08-30 16:28:23 -04:00
Calvin Leung Huang 86ea7e945d Add plugin auto-reload capability (#3171)
* Add automatic plugin reload

* Refactor builtin/backend

* Remove plugin reload at the core level

* Refactor plugin tests

* Add auto-reload test case

* Change backend to use sync.RWMutex, fix dangling test plugin processes

* Add a canary to plugin backends to avoid reloading many times (#3174)

* Call setupPluginCatalog before mount-related operations in postUnseal

* Don't create multiple system backends since core only holds a reference (#3176)

to one.
2017-08-15 22:10:32 -04:00
Chris Hoffman 6092538511 splitting cache into transactional and non-transactional cache structs (#3132) 2017-08-08 20:47:14 -04:00
Aaron Salvo ad1d74cae0 Set allowed headers via API instead of defaulting to wildcard. (#3023) 2017-08-07 10:03:30 -04:00
Jeff Mitchell c2c908cdc0 Unlock the statelock on unsuccessful sealInitCommon 2017-08-04 16:49:31 -04:00
Jeff Mitchell 3ef2113be9 Remove a couple unneeded cancels 2017-08-04 16:47:04 -04:00
Jeff Mitchell ae75312185 Make seal/stepdown functions async internally so they can poke the request context 2017-08-04 16:42:51 -04:00
Jeff Mitchell d0f329e124 Add leader cluster address to status/leader output. (#3061)
* Add leader cluster address to status/leader output. This helps in
identifying a particular node when all share the same redirect address.

Fixes #3042
2017-07-31 18:25:27 -04:00
Jeff Mitchell 1bfc6d4fe7 Add a -dev-three-node option for devs. (#3081) 2017-07-31 11:28:06 -04:00
Calvin Leung Huang bb54e9c131 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Jeff Mitchell 753b68fa1b Port TestCluster changes from proxy branch 2017-07-03 14:54:01 -04:00
Jeff Mitchell d55d75a79f Convert listener arguments to map[string]interface{} (#2905)
This allows people to use more natural constructs, e.g. for tls_disable
it can be a bool, int, or string.
2017-06-22 20:29:53 +01:00
Jeff Mitchell cf7d56e8f3 Fix up CORS.
Ref #2021
2017-06-17 01:26:25 -04:00