Commit graph

406 commits

Author SHA1 Message Date
Jeff Mitchell 4b7d2bed01 Transit convergent v3 2018-06-05 18:53:39 -04:00
Chris Hoffman 5344b7c5ae
adding option go_package to protos (#4687)
* adding option go_package to protos

* switching proto output dir to relative paths
2018-06-04 10:19:26 -04:00
Jeff Mitchell bd0ac25eb9
Merge branch 'master' into rekey-verification 2018-05-29 10:19:57 -04:00
Becca Petrin 44678e9ada
Fix possible nil pointer on mapping method (#4609) 2018-05-22 12:10:36 -07:00
Becca Petrin 4c1d8013f3
move fields and field parsing to helper (#4603) 2018-05-21 17:04:26 -07:00
Jeff Mitchell c55a2ec486 Finish api tests for verification 2018-05-20 19:01:24 -04:00
Becca Petrin 4693f4ee6c
export EscapeLDAPValue (#4556) 2018-05-11 15:51:20 -07:00
Becca Petrin 910925457f
Move LDAP client and config code to helper (#4532) 2018-05-10 14:12:42 -07:00
Brian Kassouf 55997b6bf0
physical/cache: Add a list of prefixes to not cache (#4515)
* physical/cache: Add a list of prefixes to not cache

* Rename the pathmanager

* Move cache back to the beggining of postUnseal

* Fix comment
2018-05-10 10:29:26 -07:00
Becca Petrin 76c717b081
Restrict cert auth by CIDR (#4478) 2018-05-09 15:39:55 -07:00
Jeff Mitchell c0ed57feae
Revert "proto changes (#4503)" (#4504)
This reverts commit 14594bd76e04ff09c442738800be5fdebc45512f.
2018-05-03 15:38:53 -04:00
Vishal Nayak 7549ea0d12
proto changes (#4503) 2018-05-03 15:23:14 -04:00
Becca Petrin d51acbde68
New proto version (#4501) 2018-05-03 10:19:39 -07:00
Brian Kassouf c7f9d185b0
Kv preflight (#4430)
* Update kv command to use a preflight check

* Make the existing ui endpoint return the allowed mounts

* Add kv subcommand tests

* Enable `-field` in `vault kv get/put` (#4426)

* Enable `-field` in `vault kv get/put`

Fixes #4424

* Unify nil value handling

* Use preflight helper

* Update vkv plugin

* Add all the mount info when authenticated

* Add fix the error message on put

* add metadata test

* No need to sort the capabilities

* Remove the kv client header

* kv patch command (#4432)

* Fix test

* Fix tests

* Use permission denied instead of entity disabled
2018-04-23 15:00:02 -07:00
Becca Petrin b3b7fba67e
Release database resources on each iteration of a loop (#4305) 2018-04-17 16:31:09 -07:00
Jeff Mitchell 805b5e5160
X-Forwarded-For (#4380) 2018-04-17 18:52:09 -04:00
Jeff Mitchell 530121c655
Add ability to disable an entity (#4353) 2018-04-13 21:49:40 -04:00
Becca Petrin abb621752f Clean up error string formatting (#4304) 2018-04-09 14:35:21 -04:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Brian Kassouf 39970ac23d
Move plugin env checks to their own file (#4253) 2018-04-03 10:36:14 -07:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Jeff Mitchell 5b281503eb Fix test 2018-03-29 17:40:42 -04:00
Jeff Mitchell 7a6f582168
1.10 Updates (#4218) 2018-03-29 15:32:16 -04:00
Brian Kassouf 5c84c36915
command/kv: Add a "kv" subcommand for using the key-value store (#4168)
* Add more cli subcommands

* Add metadata commands

* Add more subcommands

* Update cli

* Move archive commands to delete

* Add helpers for making http calls to the kv backend

* rename cli header

* Format the various maps from kv

* Add list command

* Update help text

* Add a command to enable versioning on a backend

* Rename enable-versions command

* Some review feedback

* Fix listing of top level keys

* Fix issue when metadata is nil

* Add test for lising top level keys

* Fix some typos

* Add a note about deleting all versions
2018-03-21 15:02:41 -07:00
Brian Kassouf cc625e19ee
Add options to mount tune and mount endpoints in preparation for versioning (#4155)
* Add some requirements for versioned k/v

* Add a warning message when an upgrade is triggered

* Add path help values

* Make the kv header a const

* Add the uid to mount entry instead of options map

* Pass the backend aware uuid to the mounts and plugins

* Fix comment

* Add options to secret/auth enable and tune CLI commands (#4170)

* Switch mount/tune options to use TypeKVPairs (#4171)

* switching options to TypeKVPairs, adding bool parse for versioned flag

* flipping bool check

* Fix leases coming back from non-leased pluin kv store

* add a test for updating mount options

* Fix tests
2018-03-21 12:04:27 -07:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Jeff Mitchell 2e50667b12
Codify using strings.Join and strings.TrimSpace around PEM handling to ensure newline sanity (#4148)
Fixes #4136
2018-03-18 16:00:51 -04:00
Brian Kassouf 5e0da6d871
helper/keysutil: Add a Wrapper object to wrap storage objects (#4147) 2018-03-18 12:59:07 -07:00
Jeff Mitchell 7b58d3b2fa Use public forked big package for encrypted_key_storage file until Go 1.10 2018-03-16 10:30:05 -04:00
Brian Nuszkowski 76be90f384 Add PKCS1v15 as a RSA signature and verification option on the Transit secret engine (#4018)
Option to specify the RSA signature type, in specific add support for PKCS1v15
2018-03-15 09:17:02 -07:00
Brian Kassouf e7b04e206b
helper/keysutil: Add a LoadPolicy function (#4116)
* helper/keysutil: Add a LoadPolicy function

* Use the load policy function in the lock manager
2018-03-09 11:01:24 -08:00
Brian Kassouf 148883fc05
helper/keyutil: Add a NewPolicy function so unexported variables can be set (#4113)
* helper/keyutil: Add a NewPolicy function so unexported variables can be set

* Set the convergent version
2018-03-08 21:06:25 -08:00
Brian Kassouf f226fbbbb9
helper/keysutil: Add a storage implementation that uses policies to encrypt paths (#3989)
* helper/keysutil: Add a policy encrypted path storage

* Add vendored deps

* Fix spelling and paths that start with a /

* Add a key version template to change configure the ciphertext prefix

* Use big.Int for base58 instead of external lib

* Update go requirment to 1.10

* Add a version prefix cache

* Move logic to helper function

* Cache the template parts

* Add a storage prefix to policy

* Add an error if the policy passed in is nil

* Pull in the go1.10 version of the math/big package until we can update
2018-03-08 17:58:50 -08:00
Vishal Nayak 527eb418fe
approle: Use TypeCommaStringSlice for BoundCIDRList (#4078)
* Use TypeCommaStringSlice for Approle bound_cidr_list

* update docs

* Add comments in the test
2018-03-08 17:49:08 -05:00
Brian Kassouf 9dba3590ac
Add context to the NewSalt function (#4102) 2018-03-08 11:21:11 -08:00
Jeff Mitchell e42033a566 Minor fixes from vet 2018-02-26 02:23:24 -05:00
Brian Kassouf c82d39c77e helper/gpgkeys: fix for vault 1.10 (#4038) 2018-02-23 14:47:25 -05:00
Jeff Mitchell b1cf13f14c Fix certutil test 2018-02-23 01:35:07 -05:00
Jeff Mitchell d19b10e680 Update comment to replication consts 2018-02-22 21:38:52 -05:00
Jeff Mitchell 7f479c4547 Add a helpful comment to replication consts 2018-02-22 21:37:54 -05:00
Jeff Mitchell cf7c86e0f8 *Partially* revert "Remove now-unneeded PKCS8 code and update certutil tests for Go 1.10"
This partially reverts commit 83f6b21d3ef930df0352a4ae7b1e971790e3eb22.
2018-02-22 20:15:56 -05:00
Jeff Mitchell 0f26cb9b8d Fix PKI tests by generating on-demand 2018-02-20 00:23:37 -05:00
Jeff Mitchell aa9fc736c7 Sanitize pem encoding to Go default of a newline at the end rather than break backwards compat 2018-02-20 00:12:05 -05:00
Jeff Mitchell ce8f652ef9 Remove now-unneeded PKCS8 code and update certutil tests for Go 1.10 2018-02-19 22:46:17 -05:00
Jeff Mitchell f29bde0052
Support other names in SANs (#3889) 2018-02-16 17:19:34 -05:00
Seth Vargo b5e4db975e Add useragent helper (#3991)
* Add useragent package

This helper provides a consistent user-agent header for Vault, taking into account different versions.

* Add user-agent headers to spanner and gcs
2018-02-15 18:30:31 -05:00
Jeff Mitchell 35906aaa6c
Add ChaCha20-Poly1305 support to transit (#3975) 2018-02-14 11:59:46 -05:00
Jeff Mitchell 4969505c7e
Add transaction-like behavior for Transit persists. (#3959) 2018-02-12 17:27:28 -05:00
Jeff Mitchell 2ff01bb3ec Remove package level variables from transit policy test, makes it easier to parallelize later and less magical 2018-02-12 11:04:58 -05:00
Jeff Mitchell 642b88c76a go vet fixes 2018-02-05 14:26:31 -05:00
Jeff Mitchell f33563f667 Some vet fixes 2018-02-04 20:37:57 -05:00
Brian Kassouf 1cee2a1415
plugins/gRPC: fix issues with reserved keywords in response data (#3881)
* plugins/gRPC: fix issues with reserved keywords in response data

* Add the path raw file for mock plugin

* Fix panic when special paths is nil

* Add tests for Listing and raw requests from plugins

* Add json.Number case when decoding the status

* Bump the version required for gRPC defaults

* Fix test for gRPC version check
2018-02-01 14:30:17 -08:00
John Eismeier 6d18e0da3d Propose small spelling change (#3890) 2018-02-01 12:51:38 -05:00
Brian Kassouf dc326a83d4 helper/storagepacker: Fix panic when bucket doesn't exist (#3875) 2018-01-31 18:42:22 -05:00
Calvin Leung Huang 385140ee6b
Version protocol switch (#3833)
* Use version to determine plugin protocol to use

* Remove field from ServeOpts

* Fix missing assignment, handle errors

* contraint -> constraint

* Inject the version string from the vault side

* Fix the version check

* Add grpc support check to database plugins

* Default to use grpc unless missing env var or fail on contraint check

* Add GRPCSupport test

* Add greater than test case

* Add go-version dep
2018-01-23 17:29:26 -05:00
Jeff Mitchell feed3b9b95
Better duo status message handling (#3834) 2018-01-23 14:18:48 -05:00
Jeff Mitchell 85560b6295 Fix build 2018-01-23 11:33:49 -05:00
Jeff Mitchell bf05b5c482
Fix intermittent panic connecting to Duo (#3832)
Fixes #2030
2018-01-23 11:29:22 -05:00
Jeff Mitchell 8e8675053b Sync some bits over 2018-01-22 21:44:49 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Brian Kassouf 7050c1ca41
gRPC Backend Plugins (#3808)
* Add grpc plugins

* Add grpc plugins

* Translate wrap info to/from proto

* Add nil checks

* Fix nil marshaling errors

* Provide logging through the go-plugin logger

* handle errors in the messages

* Update the TLS config so bidirectional connections work

* Add connectivity checks

* Restart plugin and add timeouts where context is not availible

* Add the response wrap data into the grpc system implementation

* Add leaseoptions to pb.Auth

* Add an error translator

* Add tests for translating the proto objects

* Fix rename of function

* Add tracing to plugins for easier debugging

* Handle plugin crashes with the go-plugin context

* Add test for grpcStorage

* Add tests for backend and system

* Bump go-plugin for GRPCBroker

* Remove RegisterLicense

* Add casing translations for new proto messages

* Use doneCtx in grpcClient

* Use doneCtx in grpcClient

* s/shutdown/shut down/
2018-01-18 13:49:20 -08:00
Vishal Nayak b826c56686
SHA2-256 salting for AppID (#3806)
* Use SHA2-256 hash with prefix to upgrade the paths

* test the SHA1 upgrade to SHA256

* Remove hash identifier and the delimiter; use 's' instead

* Added API test to verify the correctness of the fix

* Fix broken test

* remove unneeded test
2018-01-17 19:48:32 -05:00
Chris Hoffman 5b2b168e97
Converting OU and Organization role fields to CommaStringSlice (#3804) 2018-01-17 11:53:49 -05:00
Jeff Mitchell d8009bced1 Merge branch 'master-oss' into sethvargo/cli-magic 2018-01-10 11:15:49 -05:00
Brian Kassouf 64da50c27c
Update plugin deps to include context changes (#3765)
* Update plugin deps to include context changes

* Fix tests
2018-01-08 12:26:13 -08:00
Brian Kassouf 1c190d4bda
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Jeff Mitchell d1803098ae Merge branch 'master-oss' into sethvargo/cli-magic 2018-01-03 14:02:31 -05:00
dmwilcox 39dd122663 Update docs to reflect ability to load cold CA certs to output full chains. (#3740) 2018-01-03 10:59:18 -05:00
Chris Hoffman 164849f056
Add support for encrypted TLS key files (#3685) 2017-12-15 17:33:55 -05:00
Brian Kassouf ccb7cdc3f5 Fix plaintext backup in transit (#3692) 2017-12-15 09:08:28 -05:00
Brian Kassouf afe53eb862
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Vishal Nayak 15b3d8738e Transit: backup/restore (#3637) 2017-12-14 12:51:50 -05:00
Vishal Nayak 48ac5caaa9
Transit: Refactor internal representation of key entry map (#3652)
* convert internal map to index by string

* Add upgrade test for internal key entry map

* address review feedback
2017-12-06 18:24:00 -05:00
Calvin Leung Huang 8f87854b86
Clarify api_addr related errors on VaultPluginTLSProvider (#3620)
* Mention api_addr on VaultPluginTLSProvider logs, update docs

* Clarify message and mention automatic api_address detection

* Change error message to use api_addr

* Change error messages to use api_addr
2017-12-05 12:01:35 -05:00
Jeff Mitchell 548629e8ef Port over some changes 2017-11-30 09:43:07 -05:00
Jeff Mitchell 2b78bc2a9b
Port over bits (#3575) 2017-11-13 15:31:32 -05:00
Jeff Mitchell fdaed84e82
Add ability to send seal wrap response info into cubbyhole request. (#3562)
Ref 84f80db4bf499ce7345615cc2def77e7d48bc690
2017-11-09 12:47:42 -05:00
Jeff Mitchell a2db5671c5 Add logbridge for testing 2017-11-07 15:50:09 -05:00
Jeff Mitchell 17310654a1
Add PKCS8 marshaling to PKI (#3518) 2017-11-06 12:05:07 -05:00
Vishal Nayak 52df62d4ff
Encrypt/Decrypt/Sign/Verify using RSA in Transit backend (#3489)
* encrypt/decrypt/sign/verify RSA

* update path-help and doc

* Fix the bug which was breaking convergent encryption

* support both 2048 and 4096

* update doc to contain both 2048 and 4096

* Add test for encrypt, decrypt and rotate on RSA keys

* Support exporting RSA keys

* Add sign and verify test steps

* Remove 'RSA' from PEM header

* use the default salt length

* Add 'RSA' to PEM header since openssl is expecting that

* export rsa keys as signing-key as well

* Comment the reasoning behind the PEM headers

* remove comment

* update comment

* Parameterize hashing for RSA signing and verification

* Added test steps to check hash algo choice for RSA sign/verify

* fix test by using 'prehashed'
2017-11-03 10:45:53 -04:00
Chris Hoffman 83ea47f706
fixing test after field rename (#3530) 2017-11-03 10:29:42 -04:00
Jeff Mitchell 3a2440a651
Check input size to avoid a panic (#3521) 2017-11-02 16:40:52 -05:00
Vishal Nayak 7bae606662
External identity groups (#3447)
* external identity groups

* add local LDAP groups as well to group aliases

* add group aliases for okta credential backend

* Fix panic in tests

* fix build failure

* remove duplicated struct tag

* add test steps to test out removal of group member during renewals

* Add comment for having a prefix check in router

* fix tests

* s/parent_id/canonical_id

* s/parent/canonical in comments and errors
2017-11-02 16:05:48 -04:00
Brian Kassouf 7fed43c035
Add the ability to glob allowed roles in the Database Backend (#3387)
* Add the ability to glob allowed roles in the Database Backend

* Make the error messages better

* Switch to the go-glob repo
2017-10-30 13:24:25 -07:00
Jeff Mitchell 713d5d5307
Don't swallow errors on token functions. 2017-10-24 09:39:35 -04:00
Seth Vargo c5665920f6
Standardize on "auth method"
This removes all references I could find to:

- credential provider
- authentication backend
- authentication provider
- auth provider
- auth backend

in favor of the unified:

- auth method
2017-10-24 09:32:15 -04:00
Seth Vargo 844fe0a507
Make pgpkeys helper implement our flags interface 2017-10-24 09:28:06 -04:00
Jeff Mitchell a80791ce7e Update proto 2017-10-23 17:45:05 -04:00
Jeff Mitchell c144f95be0 Sync over 2017-10-23 16:43:07 -04:00
Vishal Nayak 2ede750c78 return the actual error for base64 decoding failure (#3397) 2017-10-20 11:21:45 -04:00
Vishal Nayak f7ed6732a5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Jeff Mitchell 9077adb377 Sanitize policy behavior across backends (#3324)
Fixes #3323
Fixes #3318

* Fix tests

* Fix tests
2017-09-13 11:36:52 -04:00
Jeff Mitchell 481b4a8b7f Fix typo in transit error 2017-09-13 11:05:57 -04:00
Brian Kassouf a8d9426d9f Update locking components from DR replication changes (#3283)
* Update locking components from DR replication changes

* Fix plugin backend test

* Add a comment about needing the statelock:
2017-09-04 19:38:37 -04:00
Calvin Leung Huang a581e96b78 Lazy-load plugin mounts (#3255)
* Lazy load plugins to avoid setup-unwrap cycle

* Remove commented blocks

* Refactor NewTestCluster, use single core cluster on basic plugin tests

* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly

* Add special path to mock plugin

* Move ensureCoresSealed to vault/testing.go

* Use same method for EnsureCoresSealed and Cleanup

* Bump ensureCoresSealed timeout to 60s

* Correctly handle nil opts on NewTestCluster

* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap

* Check metadata flag directly on the plugin process

* Plumb isMetadataMode down to PluginRunner

* Add NOOP shims when running in metadata mode

* Remove unused flag from the APIMetadata object

* Remove setupSecretPlugins and setupCredentialPlugins functions

* Move when we setup rollback manager to after the plugins are initialized

* Fix tests

* Fix merge issue

* start rollback manager after the credential setup

* Add guards against running certain client and server functions while in metadata mode

* Call initialize once a plugin is loaded on the fly

* Add more tests, update basic secret/auth plugin tests to trigger lazy loading

* Skip mount if plugin removed from catalog

* Fixup

* Remove commented line on LookupPlugin

* Fail on mount operation if plugin is re-added to catalog and mount is on existing path

* Check type and special paths on startBackend

* Fix merge conflicts

* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 01:02:03 -04:00
Jeff Mitchell d62937aaf3 Use TypeDurationSecond for TTL values in PKI. (#3270) 2017-08-31 15:46:13 -04:00
Jeff Mitchell 3edb337a00 Add option to set cluster TLS cipher suites. (#3228)
* Add option to set cluster TLS cipher suites.

Fixes #3227
2017-08-30 16:28:23 -04:00
Calvin Leung Huang b023d46cb8 Direct plugin logs through vault's logger (#3142)
* Direct plugin logs through vault's logger

* Pass in a logger in testConfig
2017-08-15 10:16:48 -04:00
Jeff Mitchell a133286609 Switch policies in AppRole to TypeCommaStringSlice (#3163) 2017-08-14 20:15:51 -04:00
Jeff Mitchell cca327feae Only use specified HTTP client for AWS creds if one is passed in. (#3161)
Probably fixes #3159
2017-08-14 12:43:11 -04:00