Commit Graph

337 Commits

Author SHA1 Message Date
Becca Petrin abb621752f Clean up error string formatting (#4304) 2018-04-09 14:35:21 -04:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Vishal Nayak ef60ded908
TypeDurationSecond for ttl and max_ttl (#4268)
* use typedurationsecond for ttl and max_ttl

* address review feedback
2018-04-04 17:47:18 -04:00
Jeff Mitchell 599f691141
Allow returning warnings and other data in 404s in the Go API (#4256)
* Allow returning list information and other data in 404s.

On read it'll output data and/or warnings on a 404 if they exist. On
list, the same behavior; the actual 'vault list' command doesn't change
behavior though in terms of output unless there are no actual keys (so
it doesn't just magically show other data).

This corrects some assumptions in response_util and wrapping.go; it also
corrects a few places in the latter where it could leak a (useless)
token in some error cases.

* Use same 404 logic in delete/put too

* Add the same secret parsing logic to the KV request functions
2018-04-03 22:35:45 -04:00
Brian Kassouf 39970ac23d
Move plugin env checks to their own file (#4253) 2018-04-03 10:36:14 -07:00
Chris Hoffman a7ada08b3b
Core handling of TTLs (#4230)
* govet cleanup in token store

* adding general ttl handling to login requests

* consolidating TTL calculation to system view

* deprecate LeaseExtend

* deprecate LeaseExtend

* set the increment to the correct value

* move calculateTTL out of SystemView

* remove unused value

* add back clearing of lease id

* implement core ttl in some backends

* removing increment and issue time from lease options

* adding ttl tests, fixing some compile issue

* adding ttl tests

* fixing some explicit max TTL logic

* fixing up some tests

* removing unneeded test

* off by one errors...

* adding back some logic for bc

* adding period to return on renewal

* tweaking max ttl capping slightly

* use the appropriate precision for ttl calculation

* deprecate proto fields instead of delete

* addressing feedback

* moving TTL handling for backends to core

* mongo is a secret backend not auth

* adding estimated ttl for backends that also manage the expiration time

* set the estimate values before calling the renew request

* moving calculate TTL to framework, revert removal of increment and issue time from logical

* minor edits

* addressing feedback

* address more feedback
2018-04-03 12:20:20 -04:00
Brian Kassouf 2fcf66896b plugin/grpc: Fix a panic when a transport error occurs during a list (#4244) 2018-04-03 08:00:04 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Brian Kassouf cc625e19ee
Add options to mount tune and mount endpoints in preparation for versioning (#4155)
* Add some requirements for versioned k/v

* Add a warning message when an upgrade is triggered

* Add path help values

* Make the kv header a const

* Add the uid to mount entry instead of options map

* Pass the backend aware uuid to the mounts and plugins

* Fix comment

* Add options to secret/auth enable and tune CLI commands (#4170)

* Switch mount/tune options to use TypeKVPairs (#4171)

* switching options to TypeKVPairs, adding bool parse for versioned flag

* flipping bool check

* Fix leases coming back from non-leased pluin kv store

* add a test for updating mount options

* Fix tests
2018-03-21 12:04:27 -07:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Brian Kassouf 9dba3590ac
Add context to the NewSalt function (#4102) 2018-03-08 11:21:11 -08:00
Jeff Mitchell b83f61f20c
Make grpc plugin client use an atomic server value to fix a data race. (#4089)
Also add some coordination to ensure we don't try to clean up the grpc
server before it's created/started
2018-03-07 09:09:37 -05:00
Brian Kassouf 1638ce6204
logical/framework: add a field data type for parsing integer arrays (#4064)
* logical/framework: add a field data type for parsing integer arrays

* Fix comment

* Add zero value
2018-03-02 15:01:13 -08:00
Brian Kassouf 9456e15904
http/logical: Add http GET parameters to the data map (#4012)
* Add get parameters to the data object

* Add test for get params
2018-02-21 14:36:53 -08:00
Brian Kassouf c5cfb8eea1
http: Add a method for returning a 404 with data (#3994)
* Add a method for returning a 404 with data

* Pass the full resp object through to respond raw

* Add comment

* Refactor so it works across plugin gRPC

* Handle some review comments

* Pass request object instead of request ID
2018-02-21 14:22:21 -08:00
Brian Kassouf db9d58e4c1
plugin/grpc: Add RemoteAddr to the request object (#3997) 2018-02-16 17:41:01 -08:00
Jeff Mitchell f29bde0052
Support other names in SANs (#3889) 2018-02-16 17:19:34 -05:00
Brian Kassouf 92f347c659
plugin/gRPC: Add more documentation to the proto file (#3988)
* plugin/gRPC: Add more documentation to the proto file

* Fix typo

* Update comments
2018-02-15 14:20:50 -08:00
Jeff Mitchell 609648de4f Convert logical.InmemStorage to a wrapper around physical/inmem.
The original reason for the split was physical's dependencies, but those
haven't been onerous for a long time. Meanwhile it's a totally separate
implementation so we could be getting faulty results from tests. Get rid
of it and use the unified physical/inmem.
2018-02-12 11:16:16 -05:00
Vishal Nayak bf66dc2841 Make mount view read only until after mount persist (#3910) 2018-02-09 14:04:25 -05:00
Brian Kassouf 948db9b32d
plugin/gRPC: fix panic when special paths are not set (#3946)
* plugin/gRPC: fix panic when special paths are not set

* Remove comment
2018-02-08 16:51:26 -08:00
Jeff Mitchell 1a8a843588
Change grpc's max sent/recv size to a very large value. (#3912) 2018-02-06 13:52:35 -05:00
Jeff Mitchell fba5f46e4d Fix compilation of test against new go-plugin 2018-02-06 13:33:29 -05:00
Jeff Mitchell f33563f667 Some vet fixes 2018-02-04 20:37:57 -05:00
Jeff Mitchell 676b302dbf
Add a sysview call to determine if a mount is local. (#3899)
This is useful for deciding when to run upgrade logic, e.g. if on a
performance secondary but local it's fine to run.
2018-02-02 18:17:12 -05:00
Brian Kassouf 1cee2a1415
plugins/gRPC: fix issues with reserved keywords in response data (#3881)
* plugins/gRPC: fix issues with reserved keywords in response data

* Add the path raw file for mock plugin

* Fix panic when special paths is nil

* Add tests for Listing and raw requests from plugins

* Add json.Number case when decoding the status

* Bump the version required for gRPC defaults

* Fix test for gRPC version check
2018-02-01 14:30:17 -08:00
Vishal Nayak 150ad8405b
Remove logical.Initialize() method (#3848)
* Remove logical.Initialize() method

* More cleanup

* Fix test
2018-01-25 20:19:27 -05:00
Calvin Leung Huang 385140ee6b
Version protocol switch (#3833)
* Use version to determine plugin protocol to use

* Remove field from ServeOpts

* Fix missing assignment, handle errors

* contraint -> constraint

* Inject the version string from the vault side

* Fix the version check

* Add grpc support check to database plugins

* Default to use grpc unless missing env var or fail on contraint check

* Add GRPCSupport test

* Add greater than test case

* Add go-version dep
2018-01-23 17:29:26 -05:00
Jeff Mitchell 8e8675053b Sync some bits over 2018-01-22 21:44:49 -05:00
Brian Kassouf b597e14f01
Update data values from byte arrays to strings in proto definition (#3829)
* Update data values from byte arrays to strings in proto definition

* Update comments
2018-01-22 17:56:34 -08:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Brian Kassouf 7050c1ca41
gRPC Backend Plugins (#3808)
* Add grpc plugins

* Add grpc plugins

* Translate wrap info to/from proto

* Add nil checks

* Fix nil marshaling errors

* Provide logging through the go-plugin logger

* handle errors in the messages

* Update the TLS config so bidirectional connections work

* Add connectivity checks

* Restart plugin and add timeouts where context is not availible

* Add the response wrap data into the grpc system implementation

* Add leaseoptions to pb.Auth

* Add an error translator

* Add tests for translating the proto objects

* Fix rename of function

* Add tracing to plugins for easier debugging

* Handle plugin crashes with the go-plugin context

* Add test for grpcStorage

* Add tests for backend and system

* Bump go-plugin for GRPCBroker

* Remove RegisterLicense

* Add casing translations for new proto messages

* Use doneCtx in grpcClient

* Use doneCtx in grpcClient

* s/shutdown/shut down/
2018-01-18 13:49:20 -08:00
Jeff Mitchell 842a3a4a05 Remove RegisterLicense from logical.Backend
It's almost certainly the wrong signature and nothing uses it currently
anyways.
2018-01-18 13:44:29 -05:00
Vishal Nayak b826c56686
SHA2-256 salting for AppID (#3806)
* Use SHA2-256 hash with prefix to upgrade the paths

* test the SHA1 upgrade to SHA256

* Remove hash identifier and the delimiter; use 's' instead

* Added API test to verify the correctness of the fix

* Fix broken test

* remove unneeded test
2018-01-17 19:48:32 -05:00
Brian Kassouf 64da50c27c
Update plugin deps to include context changes (#3765)
* Update plugin deps to include context changes

* Fix tests
2018-01-08 12:26:13 -08:00
Brian Kassouf 1c190d4bda
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Calvin Leung Huang 79cb82e133
Add logic for using Auth.Period when handling auth login/renew requests (#3677)
* Add logic for using Auth.Period when handling auth login/renew requests

* Set auth.TTL if not set in handleLoginRequest

* Always set auth.TTL = te.TTL on handleLoginRequest, check TTL and period against sys values on RenewToken

* Get sysView from le.Path, revert tests

* Add back auth.Policies

* Fix TokenStore tests, add resp warning when capping values

* Use switch for ttl/period check on RenewToken

* Move comments around
2017-12-15 13:30:05 -05:00
Brian Kassouf afe53eb862
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Jeff Mitchell 2b78bc2a9b
Port over bits (#3575) 2017-11-13 15:31:32 -05:00
Jeff Mitchell 922ac56222
Seal wrap requesting in logical.Request wrapinfo (#3559) 2017-11-09 10:32:49 -05:00
Chris Hoffman fe52ce1115
Add TypeKVPairs field type (#3535) 2017-11-07 11:11:49 -05:00
Vishal Nayak 2b8d8f77d2
Handle 'not supplied' case for field type TypeNameString (#3546)
* Fix panic if value is not supplied for variables of TypeNameString

* Add tests for 'not supplied' case of all field types
2017-11-07 10:59:57 -05:00
Calvin Leung Huang 512b254820
Return role info for each role on pathRoleList (#3532)
* Return role info for each role on pathRoleList

* Change roles -> key_info, only return key_type

* Do not initialize result map in parseRole, refactor ListResponseWithInfo

* Add role list test
2017-11-03 17:12:03 -04:00
Vishal Nayak 7bae606662
External identity groups (#3447)
* external identity groups

* add local LDAP groups as well to group aliases

* add group aliases for okta credential backend

* Fix panic in tests

* fix build failure

* remove duplicated struct tag

* add test steps to test out removal of group member during renewals

* Add comment for having a prefix check in router

* fix tests

* s/parent_id/canonical_id

* s/parent/canonical in comments and errors
2017-11-02 16:05:48 -04:00
Jeff Mitchell d38a699c32 Make compile 2017-10-23 17:15:56 -04:00
Jeff Mitchell 8e9317792d Fix some merge/update bugs 2017-10-23 16:49:46 -04:00
Jeff Mitchell c144f95be0 Sync over 2017-10-23 16:43:07 -04:00
Seth Vargo 6fd9d11c79 Allow more complex errors from plugins (no interface change) (#3444)
* Allow more complex errors from plugins

This enables more complex types to be registered and returned from plugins.

*  Register common error types

This is a slightly less drastic change, which keeps the HTTPCodedError
as an interface.

* Remove replication error from list
2017-10-19 13:29:59 -07:00
Vishal Nayak f7ed6732a5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Seth Vargo 7c01dcd6b1 Fix tags on structure (#3442)
The comma breaks mapstructure for this actually.
2017-10-10 10:53:46 -04:00