Commit Graph

1417 Commits

Author SHA1 Message Date
Jeff Mitchell 48cb30312a
Remove old workaround for a rollback error (#4206)
It can now cause problems in other situations
2018-03-27 16:34:06 -04:00
Jeff Mitchell 2f90e0c2e1 Merge branch 'master-oss' into 0.10-beta 2018-03-27 12:40:30 -04:00
Calvin Leung Huang 8d9295c539
Token store deleted parent (#4193)
* Handle removal of parent index on revoke-orphan and tidy operations

* Refactor handleTidy to use same for loop children deletion of invalid parent entry

* Update comments

* Add logic for revoke-orphan and tidy to turn no-parent tokens into orphans

* Add orphan check to test

* Update test comments

* Fix TestTokenStore_Revoke_Orphan test

* Address feedback, add explicit delete when parent prefix is empty

* Revert explicit delete, add comment on why it's not done

* Update comment to indicate ok on marking token as orphan

* Fix test
2018-03-27 11:12:06 -04:00
Jeff Mitchell a1d5defe01 Update tests to use the real accessors listing path 2018-03-26 14:21:36 -04:00
Jeff Mitchell 65d8eb0914 Add more docs around list paths in policies.
CC #4199
2018-03-26 11:30:58 -04:00
Calvin Leung Huang 25792df5a9
Passthrough request headers (#4172)
* Add passthrough request headers for secret/auth mounts

* Update comments

* Fix SyncCache deletion of passthrough_request_headers

* Remove debug line

* Case-insensitive header comparison

* Remove unnecessary allocation

* Short-circuit filteredPassthroughHeaders if there's nothing to filter

* Add whitelistedHeaders list

* Update router logic after merge

* Add whitelist test

* Add lowercase x-vault-kv-client to whitelist

* Add back const

* Refactor whitelist logic
2018-03-21 19:56:47 -04:00
Brian Kassouf 5c84c36915
command/kv: Add a "kv" subcommand for using the key-value store (#4168)
* Add more cli subcommands

* Add metadata commands

* Add more subcommands

* Update cli

* Move archive commands to delete

* Add helpers for making http calls to the kv backend

* rename cli header

* Format the various maps from kv

* Add list command

* Update help text

* Add a command to enable versioning on a backend

* Rename enable-versions command

* Some review feedback

* Fix listing of top level keys

* Fix issue when metadata is nil

* Add test for lising top level keys

* Fix some typos

* Add a note about deleting all versions
2018-03-21 15:02:41 -07:00
Brian Kassouf cc625e19ee
Add options to mount tune and mount endpoints in preparation for versioning (#4155)
* Add some requirements for versioned k/v

* Add a warning message when an upgrade is triggered

* Add path help values

* Make the kv header a const

* Add the uid to mount entry instead of options map

* Pass the backend aware uuid to the mounts and plugins

* Fix comment

* Add options to secret/auth enable and tune CLI commands (#4170)

* Switch mount/tune options to use TypeKVPairs (#4171)

* switching options to TypeKVPairs, adding bool parse for versioned flag

* flipping bool check

* Fix leases coming back from non-leased pluin kv store

* add a test for updating mount options

* Fix tests
2018-03-21 12:04:27 -07:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Jeff Mitchell e49c230f7b
Log revocations in revokeCommon rather than expireID (#4164)
Revocations that happen not as a result of direct expirations will
therefore be logged

Fixes #4156
2018-03-20 11:46:27 -04:00
Jeff Mitchell 933f1e4b87 Sync 2018-03-20 10:42:57 -04:00
Calvin Leung Huang f86881c295
Unauthenticated endpoint to list secret and auth mounts (#4134)
* Add audit hmac values to AuthConfigInput and AuthConfigOutput, fix docs

* docs: Add ttl params to auth enable endpoint

* Rewording of go string to simply string

* Add audit hmac keys as CLI flags on auth/secrets enable

* Fix copypasta mistake

* WIP on auth-list endpoint

* Rename variable to be singular, add CLI flag, show value in auth and secrets list

* Add audit hmac keys to auth and secrets list

* Only set config values if they exist

* Fix http sys/auth tests

* More auth plugin_name test fixes

* Rename tag internal_ui_show_mount to _ui_show_mount

* Add tests

* Make endpoint unauthed

* Rename field to listing_visibility

* Add listing-visibility to cli tune commands

* Use ListingVisiblityType

* Fix type conversion

* Do not actually change token's value on testHttpGet

* Remove unused ListingVisibilityAuth, use const in pathInternalUIMountsRead
2018-03-19 23:16:33 -04:00
Jeff Mitchell 3a5e1792c0 Update path-help to make clear you shouldn't put things in the URL.
Remove from website docs as those have been long deprecated.
2018-03-19 11:50:16 -04:00
Jeff Mitchell 60adb4297f Update SealWrapStorage in passthrough to use newer style glob for matching all 2018-03-18 14:25:15 -04:00
Jeff Mitchell 6e93a6c4fb
Properly forward (or specifically don't) sys calls that result in read only errors (#4129)
Prior to this policy writes against a performance secondary would not
succeed because the read-only error was swallowed by handleError. In
addition to fixing this, it adds a similar function that explicitly
doesn't trigger forwarding. This is useful for things that are local to
the secondary such as raw operations and lease management.
2018-03-17 21:29:17 -04:00
Jin-wook Jeong 92ea5f9d66 Make credential plugin to work that is modified before vault startup and reloaded after vault startup. (#4121)
Set routeEntry.rootPaths, loginPaths after plugin reload using atomic values.
2018-03-16 10:35:19 -07:00
Calvin Leung Huang 3108860d4b
Audit HMAC values on AuthConfig (#4077)
* Add audit hmac values to AuthConfigInput and AuthConfigOutput, fix docs

* docs: Add ttl params to auth enable endpoint

* Rewording of go string to simply string

* Add audit hmac keys as CLI flags on auth/secrets enable

* Fix copypasta mistake

* Add audit hmac keys to auth and secrets list

* Only set config values if they exist

* Fix http sys/auth tests

* More auth plugin_name test fixes

* Pass API values into MountEntry's config when creating auth/secrets mount

* Update usage wording
2018-03-09 14:32:28 -05:00
Jeff Mitchell 55187255bd
Truncate token store issued token periods when greater than tuned max at (#4112)
issue time, not just renew time.
2018-03-09 10:53:04 -05:00
Jeff Mitchell a82b43081c Fix output in warning displaying nanoseconds instead of seconds 2018-03-08 16:14:20 -05:00
Brian Kassouf 9dba3590ac
Add context to the NewSalt function (#4102) 2018-03-08 11:21:11 -08:00
Brian Kassouf 34d958968d Revert stopoptions (#4088)
* Use an atomic value instead to communicate whether to give up HA lock

* Remove now-unneeded StopOptions struct

* Add a channel based mutex acquisition function to avoid a deadlock

* Make periodic leader refresh only spawn a single waiting goroutine and
extend logic to the key upgrade check
2018-03-06 21:35:58 -05:00
Jeff Mitchell 6b777dc63e
Rearrange shutdown logic to remove some lock switching (#4083)
* Rearrange shutdown logic to remove some lock switching and to allow
giving up the HA lock to be skipped.
2018-03-06 18:06:09 -05:00
Calvin Leung Huang e2fb199ce5
Non-HMAC audit values (#4033)
* Add non-hmac request keys

* Update comment

* Initial audit request keys implementation

* Add audit_non_hmac_response_keys

* Move where req.NonHMACKeys gets set

* Minor refactor

* Add params to auth tune endpoints

* Sync cache on loadCredentials

* Explicitly unset req.NonHMACKeys

* Do not error if entry is nil

* Add tests

* docs: Add params to api sections

* Refactor audit.Backend and Formatter interfaces, update audit broker methods

* Add audit_broker.go

* Fix method call params in audit backends

* Remove fields from logical.Request and logical.Response, pass keys via LogInput

* Use data.GetOk to allow unsetting existing values

* Remove debug lines

* Add test for unsetting values

* Address review feedback

* Initialize values in FormatRequest and FormatResponse using input values

* Update docs

* Use strutil.StrListContains

* Use strutil.StrListContains
2018-03-02 12:18:39 -05:00
Jeff Mitchell 5034ae2dcb Add the ability to use multiple paths for capability checking (#3663)
* Add the ability to use multiple paths for capability checking. WIP
(tests, docs).

Fixes #3336

* Added tests

* added 'paths' field

* Update docs

* return error if paths is not supplied
2018-03-01 11:14:56 -05:00
Jeff Mitchell f1bd0cbe74
Use atomic values in seal to avoid some data races (#4040) 2018-02-23 17:18:48 -05:00
Jeff Mitchell d4a431b298
Move local cluster parameters to atomic values to fix some potential data races (#4036) 2018-02-23 14:47:07 -05:00
Brian Kassouf cb08fb92d2 Port some replicated cluster changes from ent (#4037) 2018-02-23 14:01:15 -05:00
Jeff Mitchell 1a814803d7 Add core object to policy store for some ent uses 2018-02-23 14:00:46 -05:00
Calvin Leung Huang 1bb4d165e7
Add TTL related config options on auth enable (#4019) 2018-02-22 10:26:29 -05:00
Jeff Mitchell 4669f37c78 Add four cluster flag 2018-02-22 00:23:37 -05:00
Gobin Sougrakpam 33e22c41ca Log warnings when too many leases are active (#3957) 2018-02-14 09:22:46 -05:00
Kevin Paulisse 2282fcef8a Decompress data before sending via sys/raw (#3954) 2018-02-09 18:43:48 -05:00
Jeff Mitchell 96ea0620fd
Add sealunwrapper to ease OSS downgrades (#3936) 2018-02-09 16:37:40 -05:00
Jeff Mitchell 847e499261 Simplify readonly view logic 2018-02-09 15:39:27 -05:00
Vishal Nayak bf66dc2841 Make mount view read only until after mount persist (#3910) 2018-02-09 14:04:25 -05:00
Vishal Nayak 81c66efd6d
AppRole/Identity: Fix for race when creating an entity during login (#3932)
* possible fix for race in approle login while creating entity

* Add a test that hits the login request concurrently

* address review comments
2018-02-09 10:40:56 -05:00
seanjfellows 4739342fdb Create audit failure metrics unconditionally (#3937) 2018-02-07 15:57:00 -05:00
Jeff Mitchell 1a8a843588
Change grpc's max sent/recv size to a very large value. (#3912) 2018-02-06 13:52:35 -05:00
Jeff Mitchell 642b88c76a go vet fixes 2018-02-05 14:26:31 -05:00
Jeff Mitchell 05723e1229 Revert cluster.go change 2018-02-04 20:42:45 -05:00
Jeff Mitchell f33563f667 Some vet fixes 2018-02-04 20:37:57 -05:00
Jeff Mitchell fc6564e4ee
Don't run rollback and upgrade functionality if we are a replication secondary (#3900)
* Don't run rollback and upgrade functionality if we are a replication
secondary, but do if the mount is local.
2018-02-02 20:28:25 -05:00
Jeff Mitchell 676b302dbf
Add a sysview call to determine if a mount is local. (#3899)
This is useful for deciding when to run upgrade logic, e.g. if on a
performance secondary but local it's fine to run.
2018-02-02 18:17:12 -05:00
Calvin Leung Huang 848ce6427b
Handle period's zero value in token store's token creation (#3880)
* Handle period's zero value on handleCreateCommon

* Add test for period zero value
2018-02-01 12:01:46 -05:00
Jeff Mitchell 60e2209532
Remove core restriction in cache and turn it into an active/standby restriction instead (#3849) 2018-01-25 22:21:51 -05:00
Vishal Nayak 150ad8405b
Remove logical.Initialize() method (#3848)
* Remove logical.Initialize() method

* More cleanup

* Fix test
2018-01-25 20:19:27 -05:00
Jeff Mitchell ad9da2e0b8
Fix intermittent panic by storing a reference to the grpc server (#3842)
* Fix intermittent panic by storing a reference to the grpc server and
using that to ensure it will never be nil.

* Just get rid of c.rpcServer
2018-01-24 20:23:08 -05:00
Jeff Mitchell cf87cc54fc Fix build 2018-01-22 21:45:22 -05:00
Jeff Mitchell 8e8675053b Sync some bits over 2018-01-22 21:44:49 -05:00
Jeff Mitchell 7d6fed2e86
Use a separate var for active node replication state (#3819) 2018-01-19 19:24:04 -05:00
Jeff Mitchell 43617619ad Embed derived contexts into replication clients 2018-01-19 07:22:31 -05:00
Jeff Mitchell 123e22cd7e Fix compile 2018-01-19 05:31:55 -05:00
Jeff Mitchell b4be030d07
Add context to barrier encryptor interface 2018-01-19 05:28:47 -05:00
Jeff Mitchell 31a7eb1168
Add context to barrier encryptor access 2018-01-19 05:24:40 -05:00
Jeff Mitchell 69aead14f0 A bit more context plumbing 2018-01-19 04:11:59 -05:00
Jeff Mitchell 33b68ebf3d Remove context from a few extraneous places 2018-01-19 03:44:06 -05:00
Jeff Mitchell 6be5b8e8a1 Don't use context in barrier type/recovery type 2018-01-19 03:17:36 -05:00
Jeff Mitchell 0f7e3bb79b Add context to performPolicyChecks 2018-01-19 02:43:39 -05:00
Jeff Mitchell c97e73ce70
Don't check recovery seal in Initialized()
It doesn't actually matter and can give false positives.
2018-01-19 01:55:33 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Jeff Mitchell 842a3a4a05 Remove RegisterLicense from logical.Backend
It's almost certainly the wrong signature and nothing uses it currently
anyways.
2018-01-18 13:44:29 -05:00
Calvin Leung Huang f59069c22f
Don't call LeaseExtend on login renewal paths when period is provided (#3803)
* Don't call LeaseExtend on login renewal paths when period is provided

* WIP tests

* NoopBackend accept backend ttl values

* Test period value on credentials backend

* Use t.Fatalf instead

* Remove mockCoreExpiration

* Add login renewal test for approle backend

* Add resp.Auth.Period check on aws and cert backend tests

* Pass in approle's period via role's period

* Correctly set period in valid-role's role

* Add period renewal test using TestCluster and approle backend

* Check for ttl values after renewals on test
2018-01-18 12:19:18 -05:00
Jeff Mitchell cf895fc19a Actually print out forwarded stacktrace 2018-01-18 11:40:59 -05:00
Calvin Leung Huang bffaf9b0c3 Remove vlogger, add log.Logger to PolicyStore struct (#3813) 2018-01-18 01:01:51 -05:00
Jeff Mitchell 5371b0489f Rename var from last commit 2018-01-17 23:08:35 -05:00
Jeff Mitchell 307a177540 Make heartbeatInterval a package var to allow tests to modify it 2018-01-17 23:05:11 -05:00
Jeff Mitchell a71c74aa3f
Add replication state to EchoReply (#3810) 2018-01-17 22:17:47 -05:00
Tom Wilkie e06cb595fc Make the plugin catalog endpoint roundtrip so we can use terraform to manage them. (#3778) 2018-01-17 19:19:28 -05:00
Jeff Mitchell 88313f0398 Update replication state logic.
Fixes #3727
2018-01-16 13:51:55 -05:00
Vishal Nayak 8ef51c0065
Delete group alias upon group deletion (#3773) 2018-01-11 10:58:05 -05:00
Jeff Mitchell d8009bced1 Merge branch 'master-oss' into sethvargo/cli-magic 2018-01-10 11:15:49 -05:00
Brian Kassouf 64da50c27c
Update plugin deps to include context changes (#3765)
* Update plugin deps to include context changes

* Fix tests
2018-01-08 12:26:13 -08:00
Brian Kassouf 1c190d4bda
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Jeff Mitchell 6681329041 Update redirect address error to be more clear 2018-01-04 10:45:40 -05:00
Jeff Mitchell 1fe494e8e1
Port IsDRSecondary over and enable returning it via sys_health (#3749) 2018-01-03 15:07:13 -05:00
Jeff Mitchell d1803098ae Merge branch 'master-oss' into sethvargo/cli-magic 2018-01-03 14:02:31 -05:00
Jeff Mitchell 92aaebd20e Bust recovery cache after unsealed is set true to force migration 2017-12-19 16:22:09 -05:00
Jeff Mitchell 9630f93845
Fix audited request header lookup (#3707)
The headers are stored lowercased but the lookup function wasn't
properly lowercasing when indexing in the header map.

Fixes #3701
2017-12-18 10:05:51 -05:00
Chris Hoffman 164849f056
Add support for encrypted TLS key files (#3685) 2017-12-15 17:33:55 -05:00
Calvin Leung Huang 57bc19c169 Defer reader.Close that is used to determine sha256 2017-12-15 14:04:09 -05:00
Calvin Leung Huang 7c0b4f1333 Avoid unseal failure if plugin backends fail to setup during postUnseal (#3686) 2017-12-15 13:31:57 -05:00
Calvin Leung Huang 79cb82e133
Add logic for using Auth.Period when handling auth login/renew requests (#3677)
* Add logic for using Auth.Period when handling auth login/renew requests

* Set auth.TTL if not set in handleLoginRequest

* Always set auth.TTL = te.TTL on handleLoginRequest, check TTL and period against sys values on RenewToken

* Get sysView from le.Path, revert tests

* Add back auth.Policies

* Fix TokenStore tests, add resp warning when capping values

* Use switch for ttl/period check on RenewToken

* Move comments around
2017-12-15 13:30:05 -05:00
Brian Kassouf afe53eb862
Database gRPC plugins (#3666)
* Start work on context aware backends

* Start work on moving the database plugins to gRPC in order to pass context

* Add context to builtin database plugins

* use byte slice instead of string

* Context all the things

* Move proto messages to the dbplugin package

* Add a grpc mechanism for running backend plugins

* Serve the GRPC plugin

* Add backwards compatibility to the database plugins

* Remove backend plugin changes

* Remove backend plugin changes

* Cleanup the transport implementations

* If grpc connection is in an unexpected state restart the plugin

* Fix tests

* Fix tests

* Remove context from the request object, replace it with context.TODO

* Add a test to verify netRPC plugins still work

* Remove unused mapstructure call

* Code review fixes

* Code review fixes

* Code review fixes
2017-12-14 14:03:11 -08:00
Brian Kassouf de9b7d779d
Fix leaking connections on cluster port (#3680) 2017-12-12 17:18:04 -08:00
Chris Hoffman 822ce95dc4
adding ability to override temp dir in dev cluster (#3673) 2017-12-11 18:02:35 -05:00
lemondrank 255212af23 Non-recursive DFS token tree revoke (#2478) 2017-12-11 16:51:37 -05:00
Vishal Nayak 0928a65c38
remove unused function (#3657) 2017-12-06 18:55:43 -05:00
Chris Hoffman f966d20225
Adding ability to cache core values, cache transaction improvements (#3640) 2017-12-06 12:25:17 -05:00
Jeff Mitchell 8f159b12b1
allowed/disallowed_policies as TypeCommaStringSlice (#3641)
Our docs apparently claim that this is a list, but the code is
string-only. This fixes that discrepancy.
2017-12-04 12:47:05 -05:00
Jeff Mitchell 20320cd71e
Move location of quit channel closing in exp manager (#3638)
* Move location of quit channel closing in exp manager

If it happens after stopping timers any timers firing before all timers
are stopped will still run the revocation function. With plugin
auto-crash-recovery this could end up instantiating a plugin that could
then try to unwrap a token from a nil token store.

This also plumbs in core so that we can grab a read lock during the
operation and check standby/sealed status before running it (after
grabbing the lock).

* Use context instead of checking core values directly

* Use official Go context in a few key places
2017-12-01 17:08:38 -05:00
Jeff Mitchell 548629e8ef Port over some changes 2017-11-30 09:43:07 -05:00
Jeff Mitchell eef6afec3e
Fix mount path for credential values in aliases (#3580) 2017-11-14 01:31:10 -05:00
Calvin Leung Huang 2f9ff5c062 Acquire state lock at the start of UnsealWithRecoveryKeys (#3579)
* Acquire state lock from the start of UnsealWithRecoveryKeys, add metrics

* Change metrics variable
2017-11-13 22:05:22 -05:00
Vishal Nayak 5d976794d4
API refactoring and doc updates (#3577)
* Doc updates and API refactoring

* fix tests

* change metadata fieldtype to TypeKVPairs

* Give example for TypeKVPairs in CLI for metadata

* Update API docs examples to reflect the native expected value for TypeKVPairs

* Don't mention comma separation in the docs for TypeCommaStringSlice

* s/groups/group; s/entities/entity; s/entity-aliases/entity-alias; s/group-aliases/group-alias

* Address review feedback

* Fix formatting

* fix sidebar links
2017-11-13 20:59:42 -05:00
Chris Hoffman 9e79e9b397
generate token functions to share common names (#3576) 2017-11-13 15:44:26 -05:00
Jeff Mitchell 2b78bc2a9b
Port over bits (#3575) 2017-11-13 15:31:32 -05:00
Jeff Mitchell d55f94f4a3
Plumb more seal wrap stuff through and move to outside layer of mount options (#3572) 2017-11-13 11:22:22 -05:00
Jeff Mitchell 0798ccdd7c Populate config error in three node test function 2017-11-10 17:44:25 -05:00
Jeff Mitchell e66c4b11d5 Add core numbers to output in dev three node 2017-11-10 16:21:46 -05:00
Brian Kassouf ab3b625a3b
Add API methods for creating a DR Operation Token and make generate root accept strategy types (#3565)
* Add API and Command code for generating a DR Operation Token

* Update generate root to accept different token strategies
2017-11-10 10:19:42 -08:00