Commit graph

140 commits

Author SHA1 Message Date
Calvin Leung Huang 8cefbca1c9
Refactor service registration (#8976)
* serivceregistration: refactor service registration logic to run later

* move state check to the internal func

* sr/kubernetes: update setInitialStateInternal godoc

* sr/kubernetes: remove return in setInitialState

* core/test: fix mockServiceRegistration

* address review feedback
2020-05-15 11:06:58 -07:00
Joe Dollard 8f74b4d2b6
provide vault server flag to exit on core shutdown (#7561)
* provide vault server flag to exit on core shutdown

* Update command/server.go

Co-Authored-By: Jeff Mitchell <jeffrey.mitchell@gmail.com>

Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-02-14 18:07:31 -08:00
Becca Petrin 759f9b38f7
Observer pattern for service registration interface (#8123)
* use observer pattern for service discovery

* update perf standby method

* fix test

* revert usersTags to being called serviceTags

* use previous consul code

* vault isnt a performance standby before starting

* log err

* changes from feedback

* add Run method to interface

* changes from feedback

* fix core test

* update example
2020-01-24 09:42:03 -08:00
Mike Jarmy e42bc0ffc0
Introduce optional service_registration stanza (#7887)
* move ServiceDiscovery into methods

* add ServiceDiscoveryFactory

* add serviceDiscovery field to vault.Core

* refactor ConsulServiceDiscovery into separate struct

* cleanup

* revert accidental change to go.mod

* cleanup

* get rid of un-needed struct tags in vault.CoreConfig

* add service_discovery parser

* add ServiceDiscovery to config

* cleanup

* cleanup

* add test for ConfigServiceDiscovery to Core

* unit testing for config service_discovery stanza

* cleanup

* get rid of un-needed redirect_addr stuff in service_discovery stanza

* improve test suite

* cleanup

* clean up test a bit

* create docs for service_discovery

* check if service_discovery is configured, but storage does not support HA

* tinker with test

* tinker with test

* tweak docs

* move ServiceDiscovery into its own package

* tweak a variable name

* fix comment

* rename service_discovery to service_registration

* tweak service_registration config

* Revert "tweak service_registration config"

This reverts commit 5509920a8ab4c5a216468f262fc07c98121dce35.

* simplify naming

* refactor into ./serviceregistration/consul
2019-12-06 09:46:39 -05:00
Calvin Leung Huang bbaa7f8ea9
core: revoke the proper token on partial failures from token-related requests (#7835)
* core: revoke the proper token on partial failures from token-related requests

* move test to vault package, move test trigger to expiration manager

* update logging messages for clarity

* docstring fix
2019-11-08 13:14:03 -08:00
Calvin Leung Huang 5428ab50ee audit: log invalid wrapping token request/response (#6541)
* audit: log invalid wrapping token request/response

* Update helper/consts/error.go

Co-Authored-By: calvn <cleung2010@gmail.com>

* update error comments

* Update vault/wrapping.go

Co-Authored-By: calvn <cleung2010@gmail.com>

* update comment

* move validateWrappingToken out of http and into logical

* minor refactor, add test cases

* comment rewording

* refactor validateWrappingToken to perform audit logging

* move ValidateWrappingToken back to wrappingVerificationFunc

* Fix tests

* Review feedback
2019-07-05 14:15:14 -07:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Jeff Mitchell e5ca13d0be
Don't add kv by default in non-dev scenarios (#6109)
It's retained for tests though since most rely on it.
2019-02-14 11:55:32 -08:00
Jeff Mitchell 5f249d4005
Add allowed_response_headers (#6115) 2019-02-05 16:02:15 -05:00
Jeff Mitchell 27c960d8df
Split SubView functionality into logical.StorageView (#6141)
This lets other parts of Vault that can't depend on the vault package
take advantage of the subview functionality.

This also allows getting rid of BarrierStorage and vault.Entry, two
totally redundant abstractions.
2019-01-31 09:25:18 -05:00
Jim Kalafut d0e2badbae Run goimports across the repository (#6010)
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
2019-01-08 16:48:57 -08:00
Becca Petrin 7bd22e6779
Run all builtins as plugins (#5536) 2018-11-06 17:21:24 -08:00
Jeff Mitchell 82992d6097
Seal migration (OSS) (#781) 2018-10-22 23:34:02 -07:00
Jeff Mitchell a64fc7d7cb
Batch tokens (#755) 2018-10-15 12:56:24 -04:00
Calvin Leung Huang 37c0b83669
Add denylist check when filtering passthrough headers (#5436)
* Add denylist check when filtering passthrough headers

* Minor comment update
2018-10-01 12:20:31 -07:00
Jeff Mitchell 919b968c27
The big one (#5346) 2018-09-17 23:03:00 -04:00
Jeff Mitchell 4261618d10 Add request timeouts in normal request path and to expirations (#4971)
* Add request timeouts in normal request path and to expirations

* Add ability to adjust default max request duration

* Some test fixes

* Ensure tests have defaults set for max request duration

* Add context cancel checking to inmem/file

* Fix tests

* Fix tests

* Set default max request duration to basically infinity for this release for BC

* Address feedback
2018-07-24 14:50:49 -07:00
Jeff Mitchell 9687ccc8fa Tackle #4929 a different way (#4932)
* Tackle #4929 a different way

This turns c.sealed into an atomic, which allows us to call sealInternal
without a lock. By doing so we can better control lock grabbing when a
condition causing the standby loop to get out of active happens. This
encapsulates that logic into two distinct pieces (although they could
be combined into one), and makes lock guarding more understandable.

* Re-add context canceling to the non-HA version of sealInternal

* Return explicitly after stopCh triggered
2018-07-24 13:57:25 -07:00
Jeff Mitchell 575a606db7
Move TokenEntry into logical. (#4729)
This allows the HTTP logicalAuth handler to cache the value in the
logical.Request, avoiding a lookup later when performing acl
checks/counting a use.
2018-06-08 17:24:27 -04:00
Jeff Mitchell 5207099042 Offline token revocation fix 2018-06-05 18:53:27 -04:00
Brian Kassouf c7f9d185b0
Kv preflight (#4430)
* Update kv command to use a preflight check

* Make the existing ui endpoint return the allowed mounts

* Add kv subcommand tests

* Enable `-field` in `vault kv get/put` (#4426)

* Enable `-field` in `vault kv get/put`

Fixes #4424

* Unify nil value handling

* Use preflight helper

* Update vkv plugin

* Add all the mount info when authenticated

* Add fix the error message on put

* add metadata test

* No need to sort the capabilities

* Remove the kv client header

* kv patch command (#4432)

* Fix test

* Fix tests

* Use permission denied instead of entity disabled
2018-04-23 15:00:02 -07:00
Chris Hoffman a7ada08b3b
Core handling of TTLs (#4230)
* govet cleanup in token store

* adding general ttl handling to login requests

* consolidating TTL calculation to system view

* deprecate LeaseExtend

* deprecate LeaseExtend

* set the increment to the correct value

* move calculateTTL out of SystemView

* remove unused value

* add back clearing of lease id

* implement core ttl in some backends

* removing increment and issue time from lease options

* adding ttl tests, fixing some compile issue

* adding ttl tests

* fixing some explicit max TTL logic

* fixing up some tests

* removing unneeded test

* off by one errors...

* adding back some logic for bc

* adding period to return on renewal

* tweaking max ttl capping slightly

* use the appropriate precision for ttl calculation

* deprecate proto fields instead of delete

* addressing feedback

* moving TTL handling for backends to core

* mongo is a secret backend not auth

* adding estimated ttl for backends that also manage the expiration time

* set the estimate values before calling the renew request

* moving calculate TTL to framework, revert removal of increment and issue time from logical

* minor edits

* addressing feedback

* address more feedback
2018-04-03 12:20:20 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Calvin Leung Huang 25792df5a9
Passthrough request headers (#4172)
* Add passthrough request headers for secret/auth mounts

* Update comments

* Fix SyncCache deletion of passthrough_request_headers

* Remove debug line

* Case-insensitive header comparison

* Remove unnecessary allocation

* Short-circuit filteredPassthroughHeaders if there's nothing to filter

* Add whitelistedHeaders list

* Update router logic after merge

* Add whitelist test

* Add lowercase x-vault-kv-client to whitelist

* Add back const

* Refactor whitelist logic
2018-03-21 19:56:47 -04:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Calvin Leung Huang e2fb199ce5
Non-HMAC audit values (#4033)
* Add non-hmac request keys

* Update comment

* Initial audit request keys implementation

* Add audit_non_hmac_response_keys

* Move where req.NonHMACKeys gets set

* Minor refactor

* Add params to auth tune endpoints

* Sync cache on loadCredentials

* Explicitly unset req.NonHMACKeys

* Do not error if entry is nil

* Add tests

* docs: Add params to api sections

* Refactor audit.Backend and Formatter interfaces, update audit broker methods

* Add audit_broker.go

* Fix method call params in audit backends

* Remove fields from logical.Request and logical.Response, pass keys via LogInput

* Use data.GetOk to allow unsetting existing values

* Remove debug lines

* Add test for unsetting values

* Address review feedback

* Initialize values in FormatRequest and FormatResponse using input values

* Update docs

* Use strutil.StrListContains

* Use strutil.StrListContains
2018-03-02 12:18:39 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Jeff Mitchell 1ba4e828b4
Ensure revocation happens before seal/step-down since token store isn't (#3500)
available after when using single-use tokens.

Fixes #3497
2017-11-02 08:47:02 -05:00
Jeff Mitchell b7ffe7ec99 Sync over 2017-10-23 16:03:36 -04:00
Jeff Mitchell fdaaaadee2 Migrate physical backends into separate packages (#3106) 2017-08-03 13:24:27 -04:00
Jeff Mitchell d0f329e124 Add leader cluster address to status/leader output. (#3061)
* Add leader cluster address to status/leader output. This helps in
identifying a particular node when all share the same redirect address.

Fixes #3042
2017-07-31 18:25:27 -04:00
Chris Hoffman 7e7d766e21 Exclude /sys/leases/renew from registering with expiration manager (#2891)
* exclude /sys/leases/renew from registering with expiration manager

* adding sys/leases/renew to return full secret object, adding tests to catch renew errors
2017-06-20 12:34:00 -04:00
Joel Thompson e06a78a474 Create unified aws auth backend (#2441)
* Rename builtin/credential/aws-ec2 to aws

The aws-ec2 authentication backend is being expanded and will become the
generic aws backend. This is a small rename commit to keep the commit
history clean.

* Expand aws-ec2 backend to more generic aws

This adds the ability to authenticate arbitrary AWS IAM principals using
AWS's sts:GetCallerIdentity method. The AWS-EC2 auth backend is being to
just AWS with the expansion.

* Add missing aws auth handler to CLI

This was omitted from the previous commit

* aws auth backend general variable name cleanup

Also fixed a bug where allowed auth types weren't being checked upon
login, and added tests for it.

* Update docs for the aws auth backend

* Refactor aws bind validation

* Fix env var override in aws backend test

Intent is to override the AWS environment variables with the TEST_*
versions if they are set, but the reverse was happening.

* Update docs on use of IAM authentication profile

AWS now allows you to change the instance profile of a running instance,
so the use case of "a long-lived instance that's not in an instance
profile" no longer means you have to use the the EC2 auth method. You
can now just change the instance profile on the fly.

* Fix typo in aws auth cli help

* Respond to PR feedback

* More PR feedback

* Respond to additional PR feedback

* Address more feedback on aws auth PR

* Make aws auth_type immutable per role

* Address more aws auth PR feedback

* Address more iam auth PR feedback

* Rename aws-ec2.html.md to aws.html.md

Per PR feedback, to go along with new backend name.

* Add MountType to logical.Request

* Make default aws auth_type dependent upon MountType

When MountType is aws-ec2, default to ec2 auth_type for backwards
compatibility with legacy roles. Otherwise, default to iam.

* Pass MountPoint and MountType back up to the core

Previously the request router reset the MountPoint and MountType back to
the empty string before returning to the core. This ensures they get set
back to the correct values.
2017-04-24 15:15:50 -04:00
Jeff Mitchell 36c84df326 Large update to request forwarding handling. (#2426) 2017-03-02 10:03:49 -05:00
Jeff Mitchell 0c39b613c8 Port some replication bits to OSS (#2386) 2017-02-16 15:15:02 -05:00
Jeff Mitchell 69eb5066dd Multi value test seal (#2281) 2017-01-17 15:43:10 -05:00
Jeff Mitchell dd0e44ca10 Add nonce to unseal to allow seeing if the operation has reset (#2276) 2017-01-17 11:47:06 -05:00
Jeff Mitchell b45a481365 Wrapping enhancements (#1927) 2016-09-28 21:01:28 -07:00
Jeff Mitchell f0203741ff Change default TTL from 30 to 32 to accommodate monthly operations (#1942) 2016-09-28 18:32:49 -04:00
Jeff Mitchell 722e26f27a Add support for PGP encrypting the initial root token. (#1883) 2016-09-13 18:42:24 -04:00
Jeff Mitchell 9fee9ce8ff Don't allow tokens in paths. (#1783) 2016-08-24 15:59:43 -04:00
Jeff Mitchell 58b32e5432 Convert to logxi 2016-08-21 18:13:37 -04:00
Jeff Mitchell 37320f8798 Request forwarding (#1721)
Add request forwarding.
2016-08-15 09:42:42 -04:00
Jeff Mitchell 4f0310ed96 Don't allow root from authentication backends either.
We've disabled this in the token store, but it makes no sense to have
that disabled but have it enabled elsewhere. It's the same issue across
all, so simply remove the ability altogether.
2016-08-08 17:32:37 -04:00
Laura Bennett e5737b6789 initial local commit 2016-07-23 21:46:28 -04:00
Jeff Mitchell 05b0e0a866 Enable audit-logging of seal and step-down commands.
This pulls the logical request building code into its own function so
that it's accessible from other HTTP handlers, then uses that with some
added logic to the Seal() and StepDown() commands to have meaningful
audit log entries.
2016-05-20 17:03:54 +00:00
Jeff Mitchell c4431a7e30 Address most review feedback. Change responses to multierror to better return more useful values when there are multiple errors 2016-05-16 16:11:33 -04:00
Sean Chittenden af4e2feda7
When testing, increase the time we wait for the stepdown to occur.
2s -> 5s, no functional change.
2016-05-15 07:30:40 -07:00
Sean Chittenden aeea7628d6 Add a *log.Logger argument to physical.Factory
Logging in the backend is a good thing.  This is a noisy interface change but should be a functional noop.
2016-04-25 20:10:32 -07:00