Commit graph

311 commits

Author SHA1 Message Date
Jeff Mitchell b7ffe7ec99 Sync over 2017-10-23 16:03:36 -04:00
Vishal Nayak ea1f552d48 set entity id on the token (#3468) 2017-10-18 13:23:05 -04:00
Vishal Nayak f7ed6732a5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Chris Hoffman 1029ad3b33 Rename "generic" secret backend to "kv" (#3292) 2017-09-15 09:02:29 -04:00
Chris Hoffman 9d73c81f38 Disable the sys/raw endpoint by default (#3329)
* disable raw endpoint by default

* adding docs

* config option raw -> raw_storage_endpoint

* docs updates

* adding listing on raw endpoint

* reworking tests for enabled raw endpoints

* root protecting base raw endpoint
2017-09-15 00:21:35 -04:00
Brian Kassouf a8d9426d9f Update locking components from DR replication changes (#3283)
* Update locking components from DR replication changes

* Fix plugin backend test

* Add a comment about needing the statelock:
2017-09-04 19:38:37 -04:00
Calvin Leung Huang a581e96b78 Lazy-load plugin mounts (#3255)
* Lazy load plugins to avoid setup-unwrap cycle

* Remove commented blocks

* Refactor NewTestCluster, use single core cluster on basic plugin tests

* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly

* Add special path to mock plugin

* Move ensureCoresSealed to vault/testing.go

* Use same method for EnsureCoresSealed and Cleanup

* Bump ensureCoresSealed timeout to 60s

* Correctly handle nil opts on NewTestCluster

* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap

* Check metadata flag directly on the plugin process

* Plumb isMetadataMode down to PluginRunner

* Add NOOP shims when running in metadata mode

* Remove unused flag from the APIMetadata object

* Remove setupSecretPlugins and setupCredentialPlugins functions

* Move when we setup rollback manager to after the plugins are initialized

* Fix tests

* Fix merge issue

* start rollback manager after the credential setup

* Add guards against running certain client and server functions while in metadata mode

* Call initialize once a plugin is loaded on the fly

* Add more tests, update basic secret/auth plugin tests to trigger lazy loading

* Skip mount if plugin removed from catalog

* Fixup

* Remove commented line on LookupPlugin

* Fail on mount operation if plugin is re-added to catalog and mount is on existing path

* Check type and special paths on startBackend

* Fix merge conflicts

* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 01:02:03 -04:00
Jeff Mitchell 3edb337a00 Add option to set cluster TLS cipher suites. (#3228)
* Add option to set cluster TLS cipher suites.

Fixes #3227
2017-08-30 16:28:23 -04:00
Calvin Leung Huang 86ea7e945d Add plugin auto-reload capability (#3171)
* Add automatic plugin reload

* Refactor builtin/backend

* Remove plugin reload at the core level

* Refactor plugin tests

* Add auto-reload test case

* Change backend to use sync.RWMutex, fix dangling test plugin processes

* Add a canary to plugin backends to avoid reloading many times (#3174)

* Call setupPluginCatalog before mount-related operations in postUnseal

* Don't create multiple system backends since core only holds a reference (#3176)

to one.
2017-08-15 22:10:32 -04:00
Chris Hoffman 6092538511 splitting cache into transactional and non-transactional cache structs (#3132) 2017-08-08 20:47:14 -04:00
Aaron Salvo ad1d74cae0 Set allowed headers via API instead of defaulting to wildcard. (#3023) 2017-08-07 10:03:30 -04:00
Jeff Mitchell c2c908cdc0 Unlock the statelock on unsuccessful sealInitCommon 2017-08-04 16:49:31 -04:00
Jeff Mitchell 3ef2113be9 Remove a couple unneeded cancels 2017-08-04 16:47:04 -04:00
Jeff Mitchell ae75312185 Make seal/stepdown functions async internally so they can poke the request context 2017-08-04 16:42:51 -04:00
Jeff Mitchell d0f329e124 Add leader cluster address to status/leader output. (#3061)
* Add leader cluster address to status/leader output. This helps in
identifying a particular node when all share the same redirect address.

Fixes #3042
2017-07-31 18:25:27 -04:00
Jeff Mitchell 1bfc6d4fe7 Add a -dev-three-node option for devs. (#3081) 2017-07-31 11:28:06 -04:00
Calvin Leung Huang bb54e9c131 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Jeff Mitchell 753b68fa1b Port TestCluster changes from proxy branch 2017-07-03 14:54:01 -04:00
Jeff Mitchell d55d75a79f Convert listener arguments to map[string]interface{} (#2905)
This allows people to use more natural constructs, e.g. for tls_disable
it can be a bool, int, or string.
2017-06-22 20:29:53 +01:00
Jeff Mitchell cf7d56e8f3 Fix up CORS.
Ref #2021
2017-06-17 01:26:25 -04:00
Aaron Salvo 0303f51b68 Cors headers (#2021) 2017-06-17 00:04:55 -04:00
Brian Rodgers e9b52ed928 Log auth info on permission denied due to ACL (#2754) 2017-06-05 18:04:31 -04:00
Jeff Mitchell 605d74d889 Don't try to clean up upgrades if we're sealed 2017-06-05 16:00:56 -04:00
Jeff Mitchell 72a5b5e23b Fix tests 2017-05-25 09:00:49 -04:00
Jeff Mitchell 3d7db4248f Rename peer cluster cache 2017-05-24 21:10:32 -04:00
Jeff Mitchell 5c230c796b Add peer cluster address cache 2017-05-24 20:51:53 -04:00
Jeff Mitchell bbe27aaedf Add heartbeating and cluster address sharing to request forwarding (#2762) 2017-05-24 15:06:56 -04:00
Jeff Mitchell 9d4801b1e8 Revert grpc back a version (they introduced a panic) and clean up a bunch of old request forwarding stuff 2017-05-24 10:38:48 -04:00
Jeff Mitchell 0d4e7fba69 Remove non-gRPC request forwarding 2017-05-24 09:34:59 -04:00
Brian Kassouf cb1f1d418c Only run Abs on the plugin directory if it's set 2017-04-24 16:20:20 -07:00
Brian Kassouf ce9688ce8c Change MlockDisabled to MlockEnabled 2017-04-24 12:21:49 -07:00
Brian Kassouf 6f9d178370 Calls to builtin plugins now go directly to the implementation instead of go-plugin 2017-04-20 18:46:41 -07:00
Brian Kassouf 8ccf10641b Merge branch 'master' into database-refactor 2017-04-12 14:29:10 -07:00
Brian Kassouf 8071aed758 Mlock the plugin process 2017-04-10 17:12:52 -07:00
Brian Kassouf 0034074691 Execute builtin plugins 2017-04-04 14:43:39 -07:00
Brian Kassouf e8781b6a2b Plugin catalog 2017-04-03 17:52:29 -07:00
Jeff Mitchell e7f418c903 Fix poison pill location 2017-03-04 10:21:27 -05:00
Jeff Mitchell 25428971c8 Add poison pill 2017-03-03 15:05:25 -05:00
Jeff Mitchell 31cddc43e1 Use own mutex for updating cluster parameters and fix leader UUID bug 2017-03-02 10:50:54 -05:00
Jeff Mitchell beb3067787 Add some trace level information about new cluster status 2017-03-02 10:21:35 -05:00
Jeff Mitchell 36c84df326 Large update to request forwarding handling. (#2426) 2017-03-02 10:03:49 -05:00
Jeff Mitchell 90389323a2 Some more forwarding client cleanup 2017-03-01 20:59:20 -05:00
Jeff Mitchell b1c2a930fe Clean up request forwarding logic 2017-03-01 18:17:06 -05:00
Jeff Mitchell 00cfaf7f64 Rejig signature of last remote wal 2017-03-01 12:42:10 -05:00
Jeff Mitchell 6ebb2cc958 Add last remote WAL bits 2017-03-01 12:40:36 -05:00
Jeff Mitchell f2282247ef Add seal cache purging back into postUnseal 2017-02-28 18:36:28 -05:00
Jeff Mitchell 09543dceeb Rejig core standby logic to check validity of barrier during active transition 2017-02-28 18:17:30 -05:00
Jeff Mitchell b29861f7bb Do some porting to make diffing easier 2017-02-24 10:45:29 -05:00
Brian Kassouf f992103615 Merge branch 'master' into acl-parameters-permission 2017-02-21 14:46:06 -08:00
Jeff Mitchell 494b4c844b More porting from rep (#2389)
* More porting from rep

* Address feedback
2017-02-16 20:13:19 -05:00
Jeff Mitchell 0c39b613c8 Port some replication bits to OSS (#2386) 2017-02-16 15:15:02 -05:00
Jeff Mitchell 0a9a6d3343 Move ReplicationState to consts 2017-02-16 13:37:21 -05:00
Jeff Mitchell 2fd59ad308 Merge branch 'master-oss' into acl-parameters-permission 2017-02-08 01:59:52 -05:00
Jeff Mitchell 67f96bc64e Rejig check for HA/Sealed in Leader to check for sealed first. (#2342)
Fixes #2334
2017-02-06 18:29:56 -05:00
Brian Kassouf 6701ba8a10 Configure the request headers that are output to the audit log (#2321)
* Add /sys/config/audited-headers endpoint for configuring the headers that will be audited

* Remove some debug lines

* Add a persistant layer and refactor a bit

* update the api endpoints to be more restful

* Add comments and clean up a few functions

* Remove unneeded hash structure functionaility

* Fix existing tests

* Add tests

* Add test for Applying the header config

* Add Benchmark for the ApplyConfig method

* ResetTimer on the benchmark:

* Update the headers comment

* Add test for audit broker

* Use hyphens instead of camel case

* Add size paramater to the allocation of the result map

* Fix the tests for the audit broker

* PR feedback

* update the path and permissions on config/* paths

* Add docs file

* Fix TestSystemBackend_RootPaths test
2017-02-02 11:49:20 -08:00
Jeff Mitchell 69eb5066dd Multi value test seal (#2281) 2017-01-17 15:43:10 -05:00
Jeff Mitchell dd0e44ca10 Add nonce to unseal to allow seeing if the operation has reset (#2276) 2017-01-17 11:47:06 -05:00
Brian Kassouf ae116ada25 Merge branch 'master' into acl-parameters-permission 2017-01-13 16:44:10 -08:00
Jeff Mitchell 252e1f1e84 Port over some work to make the system views a bit nicer 2017-01-13 14:51:27 -05:00
Jeff Mitchell 9923c753d0 Set c.standby true in non-HA context. (#2259)
This value is the key for some checks in core logic. In a non-HA
environment, if the core was sealed it would never be set back to true.
2017-01-11 11:13:09 -05:00
Jeff Mitchell 681e36c4af Split Unseal into Unseal and unsealInternal 2017-01-06 16:30:43 -05:00
Jeff Mitchell 9e5d1eaac9 Port some updates 2017-01-06 15:42:18 -05:00
Jeff Mitchell 0391475c70 Add read locks to LookupToken/ValidateWrappingToken (#2232) 2017-01-04 16:52:03 -05:00
Jeff Mitchell 3129187dc2 JWT wrapping tokens (#2172) 2017-01-04 16:44:03 -05:00
mwoolsey 3e72e50fa5 Merge remote-tracking branch 'upstream/master' 2016-11-20 18:31:55 -08:00
Jeff Mitchell e84a015487 Add extra logic around listener handling. (#2089) 2016-11-11 16:43:33 -05:00
Vishal Nayak b3c805e662 Audit the client token accessors (#2037) 2016-10-29 17:01:49 -04:00
ChaseLEngel c2b512cf46 Changed AllowOperation to take logical.Request 2016-10-16 16:29:52 -07:00
Jeff Mitchell 85315ff188 Rejig where the reload functions live 2016-09-30 00:07:22 -04:00
Jeff Mitchell b45a481365 Wrapping enhancements (#1927) 2016-09-28 21:01:28 -07:00
Jeff Mitchell 2ce4397deb Plumb through the ability to set the storage read cache size. (#1784)
Plumb through the ability to set the storage read cache size.

Fixes #1772
2016-08-26 10:27:06 -04:00
Jeff Mitchell b89073f7e6 Error when an invalid (as opposed to incorrect) unseal key is given. (#1782)
Fixes #1777
2016-08-24 14:15:25 -04:00
Jeff Mitchell 58b32e5432 Convert to logxi 2016-08-21 18:13:37 -04:00
Jeff Mitchell 2bb8adcbde Cleanup and avoid unnecessary advertisement parsing in leader check 2016-08-19 14:49:11 -04:00
Jeff Mitchell b7acf5b5ab Rename proto service stuff and change log levels for some messages 2016-08-19 11:49:25 -04:00
Jeff Mitchell bdcfe05517 Clustering enhancements (#1747) 2016-08-19 11:03:53 -04:00
vishalnayak 87c42a796b s/advertisement/redirect 2016-08-19 10:52:14 -04:00
Jeff Mitchell 62c69f8e19 Provide base64 keys in addition to hex encoded. (#1734)
* Provide base64 keys in addition to hex encoded.

Accept these at unseal/rekey time.

Also fix a bug where backup would not be honored when doing a rekey with
no operation currently ongoing.
2016-08-15 16:01:15 -04:00
Jeff Mitchell 37320f8798 Request forwarding (#1721)
Add request forwarding.
2016-08-15 09:42:42 -04:00
vishalnayak cff7aada7a Fix invalid input getting marked as internal error 2016-07-28 16:23:11 -04:00
vishalnayak a3e6400697 Remove global name/id. Make only cluster name configurable. 2016-07-26 10:01:35 -04:00
vishalnayak c7dabe4def Storing local and global cluster name/id to storage and returning them in health status 2016-07-26 02:32:42 -04:00
Jeff Mitchell df621911d7 Merge pull request #1624 from hashicorp/dynamodb-ha-off-default
Turn off DynamoDB HA by default.
2016-07-18 13:54:26 -04:00
Jeff Mitchell 028d024345 Add metrics around leadership
This can be helpful for detecting flapping.

Fixes #1544
2016-07-18 13:38:44 -04:00
Jeff Mitchell a3ce0dcb0c Turn off DynamoDB HA by default.
The semantics are wonky and have caused issues from people not reading
docs. It can be enabled but by default is off.
2016-07-18 13:19:58 -04:00
Jeff Mitchell 05b0e0a866 Enable audit-logging of seal and step-down commands.
This pulls the logical request building code into its own function so
that it's accessible from other HTTP handlers, then uses that with some
added logic to the Seal() and StepDown() commands to have meaningful
audit log entries.
2016-05-20 17:03:54 +00:00
Jeff Mitchell c4431a7e30 Address most review feedback. Change responses to multierror to better return more useful values when there are multiple errors 2016-05-16 16:11:33 -04:00
Jeff Mitchell 560e9c30a3 Merge branch 'master-oss' into cubbyhole-the-world 2016-05-12 14:59:12 -04:00
Sean Chittenden 99a5213f0b Merge pull request #1355 from hashicorp/f-vault-service
Vault/Consul Service refinement
2016-05-12 11:48:29 -07:00
Jeff Mitchell c52d352332 Merge branch 'master-oss' into cubbyhole-the-world 2016-05-07 16:40:04 -04:00
Jeff Mitchell fe1f56de40 Make a non-caching but still locking variant of transit for when caches are disabled 2016-05-02 22:36:44 -04:00
Jeff Mitchell 8572190b64 Plumb disabling caches through the policy store 2016-05-02 22:36:44 -04:00
Jeff Mitchell 2ebe49d3a1 Change UseToken mechanics.
Add locking around UseToken and Lookup. Have UseToken flag an entry that
needs to be revoked so that it can be done at the appropriate time, but
so that Lookup in the interm doesn't return a value.

The locking is a map of 4096 locks keyed off of the first three
characters of the token ID which should provide good distribution.
2016-05-02 03:44:24 -04:00
Jeff Mitchell aba689a877 Add wrapping through core and change to use TTL instead of Duration. 2016-05-02 00:47:35 -04:00
Sean Chittenden 5068d68a13 Name the output parameters for Leader 2016-04-28 11:05:18 -07:00
Sean Chittenden 0b72906fc3 Change the interface of ServiceDiscovery
Instead of passing state, signal that the state has changed and provide a callback handler that can query Core.
2016-04-28 11:05:18 -07:00
Sean Chittenden 7fe0b2c6a1 Persistently retry to update service registration
If the local Consul agent is not available while attempting to step down from active or up to active, retry once a second.  Allow for concurrent changes to the state with a single registration updater.  Fix standby initialization.
2016-04-25 18:01:13 -07:00
Sean Chittenden 230b59f34c Stub out service discovery functionality
Hook asynchronous notifications into Core to change the status of vault based on its active/standby, and sealed/unsealed status.
2016-04-25 18:00:54 -07:00
Jeff Mitchell 53773f12e3 Register the token entry's path instead of the request path, to handle role suffixes correctly 2016-04-14 08:08:28 -04:00
Jeff Mitchell a4ff72841e Check for seal status when initing and change logic order to avoid defer 2016-04-14 01:13:59 +00:00