Commit Graph

162 Commits

Author SHA1 Message Date
Michel Vocks 83d38480fc Backport OSS changes from enterprise (#7864) 2019-11-12 09:17:37 -08:00
Michel Vocks e5a921d277
Harden mount/auth filter evaluation error handling (#7754) 2019-10-29 17:42:13 +01:00
Jeff Mitchell 0c88218dd4 Port some changes that got out of sync 2019-10-28 11:38:14 -04:00
Brian Kassouf a20e73c2da
Port filtered paths changes back to OSS (#7741)
* Port filtered paths changes back to OSS

* Fix build
2019-10-27 13:30:38 -07:00
Lexman c86fe212c0
oss changes for entropy augmentation feature (#7670)
* oss changes for entropy augmentation feature

* fix oss command/server/config tests

* update go.sum

* fix logical_system and http/ tests

* adds vendored files

* removes unused variable
2019-10-17 10:33:00 -07:00
Jeff Mitchell 9816963355
Move SudoPrivilege out of SystemView (#7266)
* Move SudoPrivilege out of SystemView

We only use this in token store and it literally doesn't work anything
that isn't the token store or system mount, so we should stop exposing
something that doesn't work.

* Reconcile extended system view with sdk/logical a bit and put an explanation for why SudoPrivilege isn't moved over
2019-08-26 10:23:46 -04:00
Brian Kassouf b0cfcc003d Bind entry to initalize locally 2019-07-05 18:37:10 -07:00
Mike Jarmy e0ce2195cc AWS upgrade role entries (#7025)
* upgrade aws roles

* test upgrade aws roles

* Initialize aws credential backend at mount time

* add a TODO

* create end-to-end test for builtin/credential/aws

* fix bug in initializer

* improve comments

* add Initialize() to logical.Backend

* use Initialize() in Core.enableCredentialInternal()

* use InitializeRequest to call Initialize()

* improve unit testing for framework.Backend

* call logical.Backend.Initialize() from all of the places that it needs to be called.

* implement backend.proto changes for logical.Backend.Initialize()

* persist current role storage version when upgrading aws roles

* format comments correctly

* improve comments

* use postUnseal funcs to initialize backends

* simplify test suite

* improve test suite

* simplify logic in aws role upgrade

* simplify aws credential initialization logic

* simplify logic in aws role upgrade

* use the core's activeContext for initialization

* refactor builtin/plugin/Backend

* use a goroutine to upgrade the aws roles

* misc improvements and cleanup

* do not run AWS role upgrade on DR Secondary

* always call logical.Backend.Initialize() when loading a plugin.

* improve comments

* on standbys and DR secondaries we do not want to run any kind of upgrade logic

* fix awsVersion struct

* clarify aws version upgrade

* make the upgrade logic for aws auth more explicit

* aws upgrade is now called from a switch

* fix fallthrough bug

* simplify logic

* simplify logic

* rename things

* introduce currentAwsVersion const to track aws version

* improve comments

* rearrange things once more

* conglomerate things into one function

* stub out aws auth initialize e2e test

* improve aws auth initialize e2e test

* finish aws auth initialize e2e test

* tinker with aws auth initialize e2e test

* tinker with aws auth initialize e2e test

* tinker with aws auth initialize e2e test

* fix typo in test suite

* simplify logic a tad

* rearrange assignment

* Fix a few lifecycle related issues in #7025 (#7075)

* Fix panic when plugin fails to load
2019-07-05 16:55:40 -07:00
Jeff Mitchell a3fc497fec
Fix batch token test (#7047)
At the level of role config it doesn't mean anything to use
default-service or default-batch; that's for mount tuning. So disallow
it in tokenutil. This also fixes the fact that the switch statement
wasn't right.
2019-07-02 22:16:43 -04:00
Jeff Mitchell 07dcdc8b79 Sync 2019-06-20 20:55:10 -04:00
Jeff Mitchell 7966231d88
Port some stuff (#6939)
* Port some fixes

* Sync some updates
2019-06-20 16:02:11 -04:00
ncabatoff 006bab1e49
Add new license callback init step for logical backends. (#6887) 2019-06-17 14:11:35 -04:00
ncabatoff cc41e608dc
Add core.coreNumber field, used to differentiate multiple cores/clusters when running tests (#6855)
This is not used or exposed in prod.

Remove some test-specific code from the cluster-building helpers. The corresponding additions go on the ent side.
2019-06-10 14:07:16 -04:00
Jeff Mitchell 5f7321dcc7 Fix a case where mounts could be duplicated (#6771)
When unmounting, the router entry would be tainted, preventing routing.
However, we would then unmount the router before clearing storage, so if
an error occurred the router would have forgotten the path. For auth
mounts this isn't a problem since they had a secondary check, but
regular mounts didn't (not sure why, but this is true back to at least
0.2.0). This meant you could then create a duplicate mount using the
same path which would then not conflict in the router until postUnseal.

This adds the extra check to regular mounts, and also moves the location
of the router unmount.

This also ensures that on the next router.Mount, tainted is set to the
mount entry's tainted status.

Fixes #6769
2019-06-04 10:33:36 -07:00
ncabatoff ad28263b69
Allow plugins to submit audit requests/responses via extended SystemView (#6777)
Move audit.LogInput to sdk/logical.  Allow the Data values in audited
logical.Request and Response to implement OptMarshaler, in which case
we delegate hashing/serializing responsibility to them.  Add new
ClientCertificateSerialNumber audit request field.

SystemView can now be cast to ExtendedSystemView to expose the Auditor
interface, which allows submitting requests and responses to the audit
broker.
2019-05-22 18:52:53 -04:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Jeff Mitchell 4aa4a0665f Sync over 2019-04-11 11:12:37 -04:00
Martin 9044173d6e Prometheus support on v1/sys/metrics endpoint (#5308)
* initial commit for prometheus and sys/metrics support

* Throw an error if prometheusRetentionTime is 0,add prometheus in devmode

* return when format=prometheus is used and prom is disable

* parse prometheus_retention_time from string instead of int

* Initialize config.Telemetry if nil

* address PR issues

* add sys/metrics framework.Path in a factory

* Apply requiredMountTable entries's MountConfig to existing core table

* address pr comments

* enable prometheus sink by default

* Move Metric-related code in a separate metricsutil helper
2019-02-14 12:46:59 -08:00
Jeff Mitchell e5ca13d0be
Don't add kv by default in non-dev scenarios (#6109)
It's retained for tests though since most rely on it.
2019-02-14 11:55:32 -08:00
Jeff Mitchell 5f249d4005
Add allowed_response_headers (#6115) 2019-02-05 16:02:15 -05:00
Jeff Mitchell 27c960d8df
Split SubView functionality into logical.StorageView (#6141)
This lets other parts of Vault that can't depend on the vault package
take advantage of the subview functionality.

This also allows getting rid of BarrierStorage and vault.Entry, two
totally redundant abstractions.
2019-01-31 09:25:18 -05:00
Jim Kalafut d0e2badbae Run goimports across the repository (#6010)
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
2019-01-08 16:48:57 -08:00
Calvin Leung Huang 907dd834ff Revert deprecated plugin var names (#5822)
* Revert field back to ListPluginsResponse.Names

* Revert field back to MountConfig.PluginName and APIMountConfig.PluginName
2018-11-19 15:23:48 -08:00
Jeff Mitchell a742857edb
Fix some remount logic within namespaces (#5722) 2018-11-07 14:56:24 -05:00
Becca Petrin 7bd22e6779
Run all builtins as plugins (#5536) 2018-11-06 17:21:24 -08:00
Jeff Mitchell a64fc7d7cb
Batch tokens (#755) 2018-10-15 12:56:24 -04:00
Jeff Mitchell 919b968c27
The big one (#5346) 2018-09-17 23:03:00 -04:00
Jeff Mitchell c28ed23972
Allow most parts of Vault's logging to have its level changed on-the-fly (#5280)
* Allow most parts of Vault's logging to have its level changed on-the-fly

* Use a const for not set
2018-09-05 15:52:54 -04:00
Jeff Mitchell 362a92945e Don't resetnamed 2018-08-23 15:04:18 -04:00
Chris Hoffman d8b1d19ed6
Plumbing request context through to expiration manager (#5021)
* plumbing request context to expiration manager

* moar context

* address feedback

* only using active context for revoke prefix

* using active context for revoke commands

* cancel tidy on active context

* address feedback
2018-08-01 21:39:39 -04:00
Jeff Mitchell 98bf463a65 Make single-lease revocation behave like expiration (#4883)
This change makes it so that if a lease is revoked through user action,
we set the expiration time to now and update pending, just as we do with
tokens. This allows the normal retry logic to apply in these cases as
well, instead of just erroring out immediately. The idea being that once
you tell Vault to revoke something it should keep doing its darndest to
actually make that happen.
2018-07-11 15:45:35 -04:00
Jeff Mitchell 8f45bc69ba Fix tuning visibility in CLI (#4827)
The API elides the value if it's empty, but empty has meaning. This adds
"hidden" as an option which is fundamentally identical to the default.
2018-07-02 12:13:25 -04:00
Martin a259c16c8b Typo in Visibility + Add accepted values in Description (#4638) 2018-05-25 13:39:35 -04:00
Jeff Mitchell 1414eb5e32 Add a switch for the demo server to use version 2 kv mount by default 2018-05-22 17:31:36 -04:00
Jeff Mitchell e69845b458
Defer setting views read/write until the end of postUnseal (#4392)
A few notes:

* We exert positive control over singletons and they usually need to
perform some (known, validated) writes, so this excludes singletons --
they are simply limited to the end of the mount function as before.

* I'm not sure how to test this _specifically_; I've done some testing
of e.g. sealing vault and unsealing and ensuring that I can write to a
KV mount. I think this is tested by every dev server though, since for a
dev server Vault is inited, the default mounts are mounted, then it's
sealed, then it's unsealed for the user, so it already goes through this
code path. The mere fact that you can write to secret/ on a dev server
means it was successfully set read-write.
2018-04-19 13:29:43 -04:00
Brian Kassouf 6ca3ae4007 Port some ent mount changes (#4330) 2018-04-11 14:32:55 -04:00
Jeff Mitchell 0535f46e27 Make standard secret/ mount version 1, but upgrade to v2 in dev mode. 2018-04-09 15:37:36 -04:00
Brian Kassouf a8b8ca136e
KV: Update 'versioned' naming to 'v2' (#4293)
* Update 'versioned' naming to 'v2'

* Make sure options are set

* Fix description of auth flag

* Review feedback
2018-04-09 09:39:32 -07:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Calvin Leung Huang 25792df5a9
Passthrough request headers (#4172)
* Add passthrough request headers for secret/auth mounts

* Update comments

* Fix SyncCache deletion of passthrough_request_headers

* Remove debug line

* Case-insensitive header comparison

* Remove unnecessary allocation

* Short-circuit filteredPassthroughHeaders if there's nothing to filter

* Add whitelistedHeaders list

* Update router logic after merge

* Add whitelist test

* Add lowercase x-vault-kv-client to whitelist

* Add back const

* Refactor whitelist logic
2018-03-21 19:56:47 -04:00
Brian Kassouf cc625e19ee
Add options to mount tune and mount endpoints in preparation for versioning (#4155)
* Add some requirements for versioned k/v

* Add a warning message when an upgrade is triggered

* Add path help values

* Make the kv header a const

* Add the uid to mount entry instead of options map

* Pass the backend aware uuid to the mounts and plugins

* Fix comment

* Add options to secret/auth enable and tune CLI commands (#4170)

* Switch mount/tune options to use TypeKVPairs (#4171)

* switching options to TypeKVPairs, adding bool parse for versioned flag

* flipping bool check

* Fix leases coming back from non-leased pluin kv store

* add a test for updating mount options

* Fix tests
2018-03-21 12:04:27 -07:00
Calvin Leung Huang f86881c295
Unauthenticated endpoint to list secret and auth mounts (#4134)
* Add audit hmac values to AuthConfigInput and AuthConfigOutput, fix docs

* docs: Add ttl params to auth enable endpoint

* Rewording of go string to simply string

* Add audit hmac keys as CLI flags on auth/secrets enable

* Fix copypasta mistake

* WIP on auth-list endpoint

* Rename variable to be singular, add CLI flag, show value in auth and secrets list

* Add audit hmac keys to auth and secrets list

* Only set config values if they exist

* Fix http sys/auth tests

* More auth plugin_name test fixes

* Rename tag internal_ui_show_mount to _ui_show_mount

* Add tests

* Make endpoint unauthed

* Rename field to listing_visibility

* Add listing-visibility to cli tune commands

* Use ListingVisiblityType

* Fix type conversion

* Do not actually change token's value on testHttpGet

* Remove unused ListingVisibilityAuth, use const in pathInternalUIMountsRead
2018-03-19 23:16:33 -04:00
Calvin Leung Huang e2fb199ce5
Non-HMAC audit values (#4033)
* Add non-hmac request keys

* Update comment

* Initial audit request keys implementation

* Add audit_non_hmac_response_keys

* Move where req.NonHMACKeys gets set

* Minor refactor

* Add params to auth tune endpoints

* Sync cache on loadCredentials

* Explicitly unset req.NonHMACKeys

* Do not error if entry is nil

* Add tests

* docs: Add params to api sections

* Refactor audit.Backend and Formatter interfaces, update audit broker methods

* Add audit_broker.go

* Fix method call params in audit backends

* Remove fields from logical.Request and logical.Response, pass keys via LogInput

* Use data.GetOk to allow unsetting existing values

* Remove debug lines

* Add test for unsetting values

* Address review feedback

* Initialize values in FormatRequest and FormatResponse using input values

* Update docs

* Use strutil.StrListContains

* Use strutil.StrListContains
2018-03-02 12:18:39 -05:00
Jeff Mitchell 847e499261 Simplify readonly view logic 2018-02-09 15:39:27 -05:00
Vishal Nayak bf66dc2841 Make mount view read only until after mount persist (#3910) 2018-02-09 14:04:25 -05:00
Vishal Nayak 150ad8405b
Remove logical.Initialize() method (#3848)
* Remove logical.Initialize() method

* More cleanup

* Fix test
2018-01-25 20:19:27 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Jeff Mitchell 88313f0398 Update replication state logic.
Fixes #3727
2018-01-16 13:51:55 -05:00