Commit graph

432 commits

Author SHA1 Message Date
Jeff Mitchell ba0d029247
Restricts ACL templating to paths but allows failures (#5167)
When a templating failure happens, we now simply ignore that path,
rather than fail all access to all policies
2018-08-23 12:15:02 -04:00
Jeff Mitchell 66a0029195 Sync some ns stuff to api/command 2018-08-22 14:37:40 -04:00
Jeff Mitchell 051bb9fc13
Two PKI improvements: (#5134)
* Disallow adding CA's serial to revocation list
* Allow disabling revocation list generation. This returns an empty (but
signed) list, but does not affect tracking of revocations so turning it
back on will populate the list properly.
2018-08-21 11:20:57 -04:00
Jim Kalafut 3be8a737c1
Request less random data in truncate mode (#5118) 2018-08-17 10:17:40 -07:00
Clint 96d8bd4bf7 [WIP] Support custom max Nomad token name length [supersedes https://github.com/hashicorp/vault/pull/4361] (#5117)
* Nomad: updating max token length to 256

* Initial support for supporting custom max token name length for Nomad

* simplify/correct tests

* document nomad max_token_name_length

* removed support for max token length env var. Rename field for clarity

* cleanups after removing env var support

* move RandomWithPrefix to testhelpers

* fix spelling

* Remove default 256 value. Use zero as a sentinel value and ignore it

* update docs
2018-08-16 15:48:23 -04:00
Jim Kalafut f8a0daafa0
Add check of truncated length (#5109) 2018-08-15 13:16:17 -07:00
Jeff Mitchell 71d92ef093 ACL Templating (#4994)
* Initial work on templating

* Add check for unbalanced closing in front

* Add missing templated assignment

* Add first cut of end-to-end test on templating.

* Make template errors be 403s and finish up testing

* Review feedback
2018-08-15 11:42:56 -07:00
Jim Kalafut 09e9a7a203 Make base62.Random truncation optional (#5100) 2018-08-14 12:38:25 -04:00
Jim Kalafut 5f86155e6a
Consolidate base62 functions (#5099) 2018-08-14 08:44:30 -07:00
Stephen J. Butler 672668dea5 Add tokenGroups and tokenGroupsGlobalAndUniversal (#4936) 2018-08-13 14:57:10 -07:00
Jeff Mitchell 24cb96a9c6 Port over ns identity proto info 2018-08-10 12:45:34 -04:00
Jeff Mitchell fb3c7eb449 Port some ns stuff over 2018-08-10 12:13:11 -04:00
Jeff Mitchell 2ed2e696a7
Merge Identity Entities if two claim the same alias (#5075)
* Merge Identity Entities if two claim the same alias

Past bugs/race conditions meant two entities could be created each
claiming the same alias. There are planned longer term fixes for this
(outside of the race condition being fixed in 0.10.4) that involve
changing the data model, but this is an immediate workaround that has
the same net effect: if two entities claim the same alias, assume they
were created due to this race condition and merge them.

In this situation, also automatically merge policies so we don't lose
e.g. RGPs.
2018-08-09 15:37:36 -05:00
Jim Kalafut 70d516b34d
Update ParseStringSlice and address lint/vet warnings (#5069) 2018-08-09 11:13:37 -07:00
Jim Kalafut 41df651499
Add plugin user-agent helper (#5039) 2018-08-06 09:02:04 -07:00
Jeff Mitchell e72890e83f
VSI (#4985) 2018-07-24 22:02:27 -04:00
Jeff Mitchell 4261618d10 Add request timeouts in normal request path and to expirations (#4971)
* Add request timeouts in normal request path and to expirations

* Add ability to adjust default max request duration

* Some test fixes

* Ensure tests have defaults set for max request duration

* Add context cancel checking to inmem/file

* Fix tests

* Fix tests

* Set default max request duration to basically infinity for this release for BC

* Address feedback
2018-07-24 14:50:49 -07:00
Jeff Mitchell 8b0561aad8
Two small items: (#4934)
1) Disable MaxRetries in test cluster clients. We generally want to fail
as fast as possible in tests so adding unpredictable timing in doesn't
help things, especially if we're timing sensitive in the test.

2) EquivalentPolicies is supposed to return true if only one set
contains `default` and the other is empty, but if one set was nil
instead of simply a zero length slice it would always return false. This
means that renewing against, say, `userpass` when not actually
specifying any user policies would always fail.
2018-07-17 01:23:26 -04:00
Jeff Mitchell 28890ee198 Make proto
It appears the only thing that actually change is that the tag got
`proto3` values added.
2018-07-10 20:49:48 -04:00
Jeff Mitchell 4a3fe87a39
Allow max request size to be user-specified (#4824)
* Allow max request size to be user-specified

This turned out to be way more impactful than I'd expected because I
felt like the right granularity was per-listener, since an org may want
to treat external clients differently from internal clients. It's pretty
straightforward though.

This also introduces actually using request contexts for values, which
so far we have not done (using our own logical.Request struct instead),
but this allows non-logical methods to still get this benefit.

* Switch to ioutil.ReadAll()
2018-07-06 15:44:56 -04:00
Jeff Mitchell a8826b28e0 Port pubkey parsing from kube-auth to helper/certutil 2018-07-06 10:25:32 -04:00
Brian Kassouf 8b344acaac
secret/kv: Sort keys during list operation (#4845) 2018-06-27 11:48:59 -07:00
Jeff Mitchell e52b554c0b
Add an idle timeout for the server (#4760)
* Add an idle timeout for the server

Because tidy operations can be long-running, this also changes all tidy
operations to behave the same operationally (kick off the process, get a
warning back, log errors to server log) and makes them all run in a
goroutine.

This could mean a sort of hard stop if Vault gets sealed because the
function won't have the read lock. This should generally be okay
(running tidy again should pick back up where it left off), but future
work could use cleanup funcs to trigger the functions to stop.

* Fix up tidy test

* Add deadline to cluster connections and an idle timeout to the cluster server, plus add readheader/read timeout to api server
2018-06-16 18:21:33 -04:00
Calvin Leung Huang c4abeb9ea5
Move checkHCLKeys into hclutil (#4749) 2018-06-12 12:38:08 -04:00
Jeff Mitchell 76b0d11793
Redo transit locking (#4720)
This massively simplifies transit locking behavior by pushing some
locking down to the Policy level, and embedding either a local or global
lock in the Policy depending on whether caching is enabled or not.
2018-06-12 12:24:12 -04:00
Jeff Mitchell e5a43d3431 Remove final usage of gogo in our code 2018-06-07 10:48:36 -04:00
Jeff Mitchell 4b7d2bed01 Transit convergent v3 2018-06-05 18:53:39 -04:00
Chris Hoffman 5344b7c5ae
adding option go_package to protos (#4687)
* adding option go_package to protos

* switching proto output dir to relative paths
2018-06-04 10:19:26 -04:00
Jeff Mitchell bd0ac25eb9
Merge branch 'master' into rekey-verification 2018-05-29 10:19:57 -04:00
Becca Petrin 44678e9ada
Fix possible nil pointer on mapping method (#4609) 2018-05-22 12:10:36 -07:00
Becca Petrin 4c1d8013f3
move fields and field parsing to helper (#4603) 2018-05-21 17:04:26 -07:00
Jeff Mitchell c55a2ec486 Finish api tests for verification 2018-05-20 19:01:24 -04:00
Becca Petrin 4693f4ee6c
export EscapeLDAPValue (#4556) 2018-05-11 15:51:20 -07:00
Becca Petrin 910925457f
Move LDAP client and config code to helper (#4532) 2018-05-10 14:12:42 -07:00
Brian Kassouf 55997b6bf0
physical/cache: Add a list of prefixes to not cache (#4515)
* physical/cache: Add a list of prefixes to not cache

* Rename the pathmanager

* Move cache back to the beggining of postUnseal

* Fix comment
2018-05-10 10:29:26 -07:00
Becca Petrin 76c717b081
Restrict cert auth by CIDR (#4478) 2018-05-09 15:39:55 -07:00
Jeff Mitchell c0ed57feae
Revert "proto changes (#4503)" (#4504)
This reverts commit 14594bd76e04ff09c442738800be5fdebc45512f.
2018-05-03 15:38:53 -04:00
Vishal Nayak 7549ea0d12
proto changes (#4503) 2018-05-03 15:23:14 -04:00
Becca Petrin d51acbde68
New proto version (#4501) 2018-05-03 10:19:39 -07:00
Brian Kassouf c7f9d185b0
Kv preflight (#4430)
* Update kv command to use a preflight check

* Make the existing ui endpoint return the allowed mounts

* Add kv subcommand tests

* Enable `-field` in `vault kv get/put` (#4426)

* Enable `-field` in `vault kv get/put`

Fixes #4424

* Unify nil value handling

* Use preflight helper

* Update vkv plugin

* Add all the mount info when authenticated

* Add fix the error message on put

* add metadata test

* No need to sort the capabilities

* Remove the kv client header

* kv patch command (#4432)

* Fix test

* Fix tests

* Use permission denied instead of entity disabled
2018-04-23 15:00:02 -07:00
Becca Petrin b3b7fba67e
Release database resources on each iteration of a loop (#4305) 2018-04-17 16:31:09 -07:00
Jeff Mitchell 805b5e5160
X-Forwarded-For (#4380) 2018-04-17 18:52:09 -04:00
Jeff Mitchell 530121c655
Add ability to disable an entity (#4353) 2018-04-13 21:49:40 -04:00
Becca Petrin abb621752f Clean up error string formatting (#4304) 2018-04-09 14:35:21 -04:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Brian Kassouf 39970ac23d
Move plugin env checks to their own file (#4253) 2018-04-03 10:36:14 -07:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Jeff Mitchell 5b281503eb Fix test 2018-03-29 17:40:42 -04:00
Jeff Mitchell 7a6f582168
1.10 Updates (#4218) 2018-03-29 15:32:16 -04:00
Brian Kassouf 5c84c36915
command/kv: Add a "kv" subcommand for using the key-value store (#4168)
* Add more cli subcommands

* Add metadata commands

* Add more subcommands

* Update cli

* Move archive commands to delete

* Add helpers for making http calls to the kv backend

* rename cli header

* Format the various maps from kv

* Add list command

* Update help text

* Add a command to enable versioning on a backend

* Rename enable-versions command

* Some review feedback

* Fix listing of top level keys

* Fix issue when metadata is nil

* Add test for lising top level keys

* Fix some typos

* Add a note about deleting all versions
2018-03-21 15:02:41 -07:00
Brian Kassouf cc625e19ee
Add options to mount tune and mount endpoints in preparation for versioning (#4155)
* Add some requirements for versioned k/v

* Add a warning message when an upgrade is triggered

* Add path help values

* Make the kv header a const

* Add the uid to mount entry instead of options map

* Pass the backend aware uuid to the mounts and plugins

* Fix comment

* Add options to secret/auth enable and tune CLI commands (#4170)

* Switch mount/tune options to use TypeKVPairs (#4171)

* switching options to TypeKVPairs, adding bool parse for versioned flag

* flipping bool check

* Fix leases coming back from non-leased pluin kv store

* add a test for updating mount options

* Fix tests
2018-03-21 12:04:27 -07:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Jeff Mitchell 2e50667b12
Codify using strings.Join and strings.TrimSpace around PEM handling to ensure newline sanity (#4148)
Fixes #4136
2018-03-18 16:00:51 -04:00
Brian Kassouf 5e0da6d871
helper/keysutil: Add a Wrapper object to wrap storage objects (#4147) 2018-03-18 12:59:07 -07:00
Jeff Mitchell 7b58d3b2fa Use public forked big package for encrypted_key_storage file until Go 1.10 2018-03-16 10:30:05 -04:00
Brian Nuszkowski 76be90f384 Add PKCS1v15 as a RSA signature and verification option on the Transit secret engine (#4018)
Option to specify the RSA signature type, in specific add support for PKCS1v15
2018-03-15 09:17:02 -07:00
Brian Kassouf e7b04e206b
helper/keysutil: Add a LoadPolicy function (#4116)
* helper/keysutil: Add a LoadPolicy function

* Use the load policy function in the lock manager
2018-03-09 11:01:24 -08:00
Brian Kassouf 148883fc05
helper/keyutil: Add a NewPolicy function so unexported variables can be set (#4113)
* helper/keyutil: Add a NewPolicy function so unexported variables can be set

* Set the convergent version
2018-03-08 21:06:25 -08:00
Brian Kassouf f226fbbbb9
helper/keysutil: Add a storage implementation that uses policies to encrypt paths (#3989)
* helper/keysutil: Add a policy encrypted path storage

* Add vendored deps

* Fix spelling and paths that start with a /

* Add a key version template to change configure the ciphertext prefix

* Use big.Int for base58 instead of external lib

* Update go requirment to 1.10

* Add a version prefix cache

* Move logic to helper function

* Cache the template parts

* Add a storage prefix to policy

* Add an error if the policy passed in is nil

* Pull in the go1.10 version of the math/big package until we can update
2018-03-08 17:58:50 -08:00
Vishal Nayak 527eb418fe
approle: Use TypeCommaStringSlice for BoundCIDRList (#4078)
* Use TypeCommaStringSlice for Approle bound_cidr_list

* update docs

* Add comments in the test
2018-03-08 17:49:08 -05:00
Brian Kassouf 9dba3590ac
Add context to the NewSalt function (#4102) 2018-03-08 11:21:11 -08:00
Jeff Mitchell e42033a566 Minor fixes from vet 2018-02-26 02:23:24 -05:00
Brian Kassouf c82d39c77e helper/gpgkeys: fix for vault 1.10 (#4038) 2018-02-23 14:47:25 -05:00
Jeff Mitchell b1cf13f14c Fix certutil test 2018-02-23 01:35:07 -05:00
Jeff Mitchell d19b10e680 Update comment to replication consts 2018-02-22 21:38:52 -05:00
Jeff Mitchell 7f479c4547 Add a helpful comment to replication consts 2018-02-22 21:37:54 -05:00
Jeff Mitchell cf7c86e0f8 *Partially* revert "Remove now-unneeded PKCS8 code and update certutil tests for Go 1.10"
This partially reverts commit 83f6b21d3ef930df0352a4ae7b1e971790e3eb22.
2018-02-22 20:15:56 -05:00
Jeff Mitchell 0f26cb9b8d Fix PKI tests by generating on-demand 2018-02-20 00:23:37 -05:00
Jeff Mitchell aa9fc736c7 Sanitize pem encoding to Go default of a newline at the end rather than break backwards compat 2018-02-20 00:12:05 -05:00
Jeff Mitchell ce8f652ef9 Remove now-unneeded PKCS8 code and update certutil tests for Go 1.10 2018-02-19 22:46:17 -05:00
Jeff Mitchell f29bde0052
Support other names in SANs (#3889) 2018-02-16 17:19:34 -05:00
Seth Vargo b5e4db975e Add useragent helper (#3991)
* Add useragent package

This helper provides a consistent user-agent header for Vault, taking into account different versions.

* Add user-agent headers to spanner and gcs
2018-02-15 18:30:31 -05:00
Jeff Mitchell 35906aaa6c
Add ChaCha20-Poly1305 support to transit (#3975) 2018-02-14 11:59:46 -05:00
Jeff Mitchell 4969505c7e
Add transaction-like behavior for Transit persists. (#3959) 2018-02-12 17:27:28 -05:00
Jeff Mitchell 2ff01bb3ec Remove package level variables from transit policy test, makes it easier to parallelize later and less magical 2018-02-12 11:04:58 -05:00
Jeff Mitchell 642b88c76a go vet fixes 2018-02-05 14:26:31 -05:00
Jeff Mitchell f33563f667 Some vet fixes 2018-02-04 20:37:57 -05:00
Brian Kassouf 1cee2a1415
plugins/gRPC: fix issues with reserved keywords in response data (#3881)
* plugins/gRPC: fix issues with reserved keywords in response data

* Add the path raw file for mock plugin

* Fix panic when special paths is nil

* Add tests for Listing and raw requests from plugins

* Add json.Number case when decoding the status

* Bump the version required for gRPC defaults

* Fix test for gRPC version check
2018-02-01 14:30:17 -08:00
John Eismeier 6d18e0da3d Propose small spelling change (#3890) 2018-02-01 12:51:38 -05:00
Brian Kassouf dc326a83d4 helper/storagepacker: Fix panic when bucket doesn't exist (#3875) 2018-01-31 18:42:22 -05:00
Calvin Leung Huang 385140ee6b
Version protocol switch (#3833)
* Use version to determine plugin protocol to use

* Remove field from ServeOpts

* Fix missing assignment, handle errors

* contraint -> constraint

* Inject the version string from the vault side

* Fix the version check

* Add grpc support check to database plugins

* Default to use grpc unless missing env var or fail on contraint check

* Add GRPCSupport test

* Add greater than test case

* Add go-version dep
2018-01-23 17:29:26 -05:00
Jeff Mitchell feed3b9b95
Better duo status message handling (#3834) 2018-01-23 14:18:48 -05:00
Jeff Mitchell 85560b6295 Fix build 2018-01-23 11:33:49 -05:00
Jeff Mitchell bf05b5c482
Fix intermittent panic connecting to Duo (#3832)
Fixes #2030
2018-01-23 11:29:22 -05:00
Jeff Mitchell 8e8675053b Sync some bits over 2018-01-22 21:44:49 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Brian Kassouf 7050c1ca41
gRPC Backend Plugins (#3808)
* Add grpc plugins

* Add grpc plugins

* Translate wrap info to/from proto

* Add nil checks

* Fix nil marshaling errors

* Provide logging through the go-plugin logger

* handle errors in the messages

* Update the TLS config so bidirectional connections work

* Add connectivity checks

* Restart plugin and add timeouts where context is not availible

* Add the response wrap data into the grpc system implementation

* Add leaseoptions to pb.Auth

* Add an error translator

* Add tests for translating the proto objects

* Fix rename of function

* Add tracing to plugins for easier debugging

* Handle plugin crashes with the go-plugin context

* Add test for grpcStorage

* Add tests for backend and system

* Bump go-plugin for GRPCBroker

* Remove RegisterLicense

* Add casing translations for new proto messages

* Use doneCtx in grpcClient

* Use doneCtx in grpcClient

* s/shutdown/shut down/
2018-01-18 13:49:20 -08:00
Vishal Nayak b826c56686
SHA2-256 salting for AppID (#3806)
* Use SHA2-256 hash with prefix to upgrade the paths

* test the SHA1 upgrade to SHA256

* Remove hash identifier and the delimiter; use 's' instead

* Added API test to verify the correctness of the fix

* Fix broken test

* remove unneeded test
2018-01-17 19:48:32 -05:00
Chris Hoffman 5b2b168e97
Converting OU and Organization role fields to CommaStringSlice (#3804) 2018-01-17 11:53:49 -05:00
Jeff Mitchell d8009bced1 Merge branch 'master-oss' into sethvargo/cli-magic 2018-01-10 11:15:49 -05:00
Brian Kassouf 64da50c27c
Update plugin deps to include context changes (#3765)
* Update plugin deps to include context changes

* Fix tests
2018-01-08 12:26:13 -08:00
Brian Kassouf 1c190d4bda
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Jeff Mitchell d1803098ae Merge branch 'master-oss' into sethvargo/cli-magic 2018-01-03 14:02:31 -05:00
dmwilcox 39dd122663 Update docs to reflect ability to load cold CA certs to output full chains. (#3740) 2018-01-03 10:59:18 -05:00
Chris Hoffman 164849f056
Add support for encrypted TLS key files (#3685) 2017-12-15 17:33:55 -05:00
Brian Kassouf ccb7cdc3f5 Fix plaintext backup in transit (#3692) 2017-12-15 09:08:28 -05:00
Brian Kassouf afe53eb862
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Vishal Nayak 15b3d8738e Transit: backup/restore (#3637) 2017-12-14 12:51:50 -05:00
Vishal Nayak 48ac5caaa9
Transit: Refactor internal representation of key entry map (#3652)
* convert internal map to index by string

* Add upgrade test for internal key entry map

* address review feedback
2017-12-06 18:24:00 -05:00
Calvin Leung Huang 8f87854b86
Clarify api_addr related errors on VaultPluginTLSProvider (#3620)
* Mention api_addr on VaultPluginTLSProvider logs, update docs

* Clarify message and mention automatic api_address detection

* Change error message to use api_addr

* Change error messages to use api_addr
2017-12-05 12:01:35 -05:00