Commit graph

303 commits

Author SHA1 Message Date
Brian Kassouf 64da50c27c
Update plugin deps to include context changes (#3765)
* Update plugin deps to include context changes

* Fix tests
2018-01-08 12:26:13 -08:00
Brian Kassouf 1c190d4bda
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Calvin Leung Huang 79cb82e133
Add logic for using Auth.Period when handling auth login/renew requests (#3677)
* Add logic for using Auth.Period when handling auth login/renew requests

* Set auth.TTL if not set in handleLoginRequest

* Always set auth.TTL = te.TTL on handleLoginRequest, check TTL and period against sys values on RenewToken

* Get sysView from le.Path, revert tests

* Add back auth.Policies

* Fix TokenStore tests, add resp warning when capping values

* Use switch for ttl/period check on RenewToken

* Move comments around
2017-12-15 13:30:05 -05:00
Brian Kassouf afe53eb862
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Jeff Mitchell 2b78bc2a9b
Port over bits (#3575) 2017-11-13 15:31:32 -05:00
Jeff Mitchell 922ac56222
Seal wrap requesting in logical.Request wrapinfo (#3559) 2017-11-09 10:32:49 -05:00
Chris Hoffman fe52ce1115
Add TypeKVPairs field type (#3535) 2017-11-07 11:11:49 -05:00
Vishal Nayak 2b8d8f77d2
Handle 'not supplied' case for field type TypeNameString (#3546)
* Fix panic if value is not supplied for variables of TypeNameString

* Add tests for 'not supplied' case of all field types
2017-11-07 10:59:57 -05:00
Calvin Leung Huang 512b254820
Return role info for each role on pathRoleList (#3532)
* Return role info for each role on pathRoleList

* Change roles -> key_info, only return key_type

* Do not initialize result map in parseRole, refactor ListResponseWithInfo

* Add role list test
2017-11-03 17:12:03 -04:00
Vishal Nayak 7bae606662
External identity groups (#3447)
* external identity groups

* add local LDAP groups as well to group aliases

* add group aliases for okta credential backend

* Fix panic in tests

* fix build failure

* remove duplicated struct tag

* add test steps to test out removal of group member during renewals

* Add comment for having a prefix check in router

* fix tests

* s/parent_id/canonical_id

* s/parent/canonical in comments and errors
2017-11-02 16:05:48 -04:00
Jeff Mitchell d38a699c32 Make compile 2017-10-23 17:15:56 -04:00
Jeff Mitchell 8e9317792d Fix some merge/update bugs 2017-10-23 16:49:46 -04:00
Jeff Mitchell c144f95be0 Sync over 2017-10-23 16:43:07 -04:00
Seth Vargo 6fd9d11c79 Allow more complex errors from plugins (no interface change) (#3444)
* Allow more complex errors from plugins

This enables more complex types to be registered and returned from plugins.

*  Register common error types

This is a slightly less drastic change, which keeps the HTTPCodedError
as an interface.

* Remove replication error from list
2017-10-19 13:29:59 -07:00
Vishal Nayak f7ed6732a5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Seth Vargo 7c01dcd6b1 Fix tags on structure (#3442)
The comma breaks mapstructure for this actually.
2017-10-10 10:53:46 -04:00
Brian Kassouf f43d18fba5 Allow for response errors to be passed back from a plugin (#3412)
* Allow for response errors to be passed back from a plugin

* Fix issue with checking plugin.BasicError == nil
2017-10-06 14:20:57 -07:00
Vishal Nayak 3f7454939e plugin version bump (#3423) 2017-10-04 20:02:13 -04:00
Calvin Leung Huang 7df9ce2656 Fix plugin args output (#3424) 2017-10-04 20:00:53 -04:00
Vishal Nayak eb418f59a0 Rename persona to alias (#3420)
Merging this will break the build. I'll fix it post merge by updating the vendor libs.
2017-10-04 13:35:05 -04:00
Jeff Mitchell 7312c80e27 Add PersonaLookahead operation 2017-09-14 20:54:09 -04:00
Brian Kassouf a8d9426d9f Update locking components from DR replication changes (#3283)
* Update locking components from DR replication changes

* Fix plugin backend test

* Add a comment about needing the statelock:
2017-09-04 19:38:37 -04:00
Brian Kassouf ed6e818fea Plugin Version Update (#3275)
* Bump plugin version requirement

* Register time.Duration with gob
2017-09-01 00:01:14 -07:00
Calvin Leung Huang a581e96b78 Lazy-load plugin mounts (#3255)
* Lazy load plugins to avoid setup-unwrap cycle

* Remove commented blocks

* Refactor NewTestCluster, use single core cluster on basic plugin tests

* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly

* Add special path to mock plugin

* Move ensureCoresSealed to vault/testing.go

* Use same method for EnsureCoresSealed and Cleanup

* Bump ensureCoresSealed timeout to 60s

* Correctly handle nil opts on NewTestCluster

* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap

* Check metadata flag directly on the plugin process

* Plumb isMetadataMode down to PluginRunner

* Add NOOP shims when running in metadata mode

* Remove unused flag from the APIMetadata object

* Remove setupSecretPlugins and setupCredentialPlugins functions

* Move when we setup rollback manager to after the plugins are initialized

* Fix tests

* Fix merge issue

* start rollback manager after the credential setup

* Add guards against running certain client and server functions while in metadata mode

* Call initialize once a plugin is loaded on the fly

* Add more tests, update basic secret/auth plugin tests to trigger lazy loading

* Skip mount if plugin removed from catalog

* Fixup

* Remove commented line on LookupPlugin

* Fail on mount operation if plugin is re-added to catalog and mount is on existing path

* Check type and special paths on startBackend

* Fix merge conflicts

* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 01:02:03 -04:00
Calvin Leung Huang 86ea7e945d Add plugin auto-reload capability (#3171)
* Add automatic plugin reload

* Refactor builtin/backend

* Remove plugin reload at the core level

* Refactor plugin tests

* Add auto-reload test case

* Change backend to use sync.RWMutex, fix dangling test plugin processes

* Add a canary to plugin backends to avoid reloading many times (#3174)

* Call setupPluginCatalog before mount-related operations in postUnseal

* Don't create multiple system backends since core only holds a reference (#3176)

to one.
2017-08-15 22:10:32 -04:00
vishalnayak 8902240dfa Added persona to logical auth 2017-08-15 13:55:58 -04:00
Calvin Leung Huang b023d46cb8 Direct plugin logs through vault's logger (#3142)
* Direct plugin logs through vault's logger

* Pass in a logger in testConfig
2017-08-15 10:16:48 -04:00
Calvin Leung Huang 6853d80212 Add plugin backend reload capability (#3112)
* Add plugin reload capability on all mounts for a specific plugin type

* Comments cleanup

* Add per-mount plugin backend reload, add tests

* Fix typos

* Remove old comment

* Reuse existing storage view in reloadPluginCommon

* Correctly handle reloading auth plugin backends

* Update path to plugin/backend/reload

* Use multierrors on reloadMatchingPluginMounts, attempt to reload all mounts provided

* Use internal value as check to ensure plugin backend reload

* Remove connection state from request for plugins at the moment

* Minor cleanup

* Refactor tests
2017-08-08 00:18:59 -04:00
Calvin Leung Huang 0871e7eadb Update mock-plugin (#3107) 2017-08-04 11:43:36 -04:00
Jeff Mitchell fdaaaadee2 Migrate physical backends into separate packages (#3106) 2017-08-03 13:24:27 -04:00
Calvin Leung Huang db9d9e6415 Store original request path in WrapInfo (#3100)
* Store original request path in WrapInfo as CreationPath

* Add wrapping_token_creation_path to CLI output

* Add CreationPath to AuditResponseWrapInfo

* Fix tests

* Add and fix tests, update API docs with new sample responses
2017-08-02 18:28:58 -04:00
Jeff Mitchell 1d0c98257d Add more TypeDurationSecond conversions 2017-08-02 18:12:02 -04:00
Calvin Leung Huang 54dd6967d8 Default mock-plugin type to logical.TypeLogical on Factory func 2017-07-31 16:20:52 -04:00
Vishal Nayak f6b03e8b1b Adding logical/identity.go to OSS (#3054) 2017-07-27 11:56:32 -04:00
Jeff Mitchell 8519b3e8ed Make logical.InmemStorage standalone (#3066) 2017-07-26 17:59:14 -04:00
Calvin Leung Huang 40c1c93937 Fix gob register issue when using tls certs on plugins (#3060) 2017-07-26 13:44:07 -04:00
Calvin Leung Huang c00741d587 Do not send storage on HandleRequest and HandleExistenceCheck on plugins 2017-07-25 16:57:26 -04:00
Calvin Leung Huang bb54e9c131 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Chris Hoffman 52a5d1a8e7 fixing Validate() for field data on TypeNameString (#3030) 2017-07-17 13:44:47 -07:00
Chris Hoffman b1b17cc387 Add field type TypeNameString (#3028) 2017-07-17 11:39:58 -07:00
Jeff Mitchell 338ebffb9d Add string slice with single value test function 2017-06-14 14:20:37 -04:00
Jeff Mitchell 39286c0500 Automatically upgrade values to salted on access if salt exists in PathMap (#2831) 2017-06-07 15:22:08 -04:00
Jeff Mitchell f7df60b131 Allow accessing Warnings directly in Response. (#2806)
A change in copystructure has caused some panics due to the custom copy
function. I'm more nervous about production panics than I am about
keeping some bad code wiping out some existing warnings, so remove the
custom copy function and just allow direct setting of Warnings.
2017-06-05 10:52:43 -04:00
Jeff Mitchell d25aa9fc21 Don't write salts in initialization, look up on demand (#2702) 2017-05-09 17:51:09 -04:00
Jeff Mitchell 490b01d6d8 Add salt mutex to app-id (#2690) 2017-05-08 16:15:24 -04:00
Brian Kassouf 5ee0d696d4 Merge remote-tracking branch 'oss/master' into database-refactor 2017-05-04 10:45:18 -07:00
Brian Kassouf 29bfc0a0d4 PR comments 2017-05-04 10:41:59 -07:00
Brian Kassouf ce9688ce8c Change MlockDisabled to MlockEnabled 2017-04-24 12:21:49 -07:00
Joel Thompson e06a78a474 Create unified aws auth backend (#2441)
* Rename builtin/credential/aws-ec2 to aws

The aws-ec2 authentication backend is being expanded and will become the
generic aws backend. This is a small rename commit to keep the commit
history clean.

* Expand aws-ec2 backend to more generic aws

This adds the ability to authenticate arbitrary AWS IAM principals using
AWS's sts:GetCallerIdentity method. The AWS-EC2 auth backend is being to
just AWS with the expansion.

* Add missing aws auth handler to CLI

This was omitted from the previous commit

* aws auth backend general variable name cleanup

Also fixed a bug where allowed auth types weren't being checked upon
login, and added tests for it.

* Update docs for the aws auth backend

* Refactor aws bind validation

* Fix env var override in aws backend test

Intent is to override the AWS environment variables with the TEST_*
versions if they are set, but the reverse was happening.

* Update docs on use of IAM authentication profile

AWS now allows you to change the instance profile of a running instance,
so the use case of "a long-lived instance that's not in an instance
profile" no longer means you have to use the the EC2 auth method. You
can now just change the instance profile on the fly.

* Fix typo in aws auth cli help

* Respond to PR feedback

* More PR feedback

* Respond to additional PR feedback

* Address more feedback on aws auth PR

* Make aws auth_type immutable per role

* Address more aws auth PR feedback

* Address more iam auth PR feedback

* Rename aws-ec2.html.md to aws.html.md

Per PR feedback, to go along with new backend name.

* Add MountType to logical.Request

* Make default aws auth_type dependent upon MountType

When MountType is aws-ec2, default to ec2 auth_type for backwards
compatibility with legacy roles. Otherwise, default to iam.

* Pass MountPoint and MountType back up to the core

Previously the request router reset the MountPoint and MountType back to
the empty string before returning to the core. This ensures they get set
back to the correct values.
2017-04-24 15:15:50 -04:00
Brian Kassouf 657d433330 Update the ResponseWrapData function to return a wrapping.ResponseWrapInfo object 2017-04-24 12:15:01 -07:00