Commit Graph

123 Commits

Author SHA1 Message Date
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Calvin Leung Huang 25792df5a9
Passthrough request headers (#4172)
* Add passthrough request headers for secret/auth mounts

* Update comments

* Fix SyncCache deletion of passthrough_request_headers

* Remove debug line

* Case-insensitive header comparison

* Remove unnecessary allocation

* Short-circuit filteredPassthroughHeaders if there's nothing to filter

* Add whitelistedHeaders list

* Update router logic after merge

* Add whitelist test

* Add lowercase x-vault-kv-client to whitelist

* Add back const

* Refactor whitelist logic
2018-03-21 19:56:47 -04:00
Brian Kassouf cc625e19ee
Add options to mount tune and mount endpoints in preparation for versioning (#4155)
* Add some requirements for versioned k/v

* Add a warning message when an upgrade is triggered

* Add path help values

* Make the kv header a const

* Add the uid to mount entry instead of options map

* Pass the backend aware uuid to the mounts and plugins

* Fix comment

* Add options to secret/auth enable and tune CLI commands (#4170)

* Switch mount/tune options to use TypeKVPairs (#4171)

* switching options to TypeKVPairs, adding bool parse for versioned flag

* flipping bool check

* Fix leases coming back from non-leased pluin kv store

* add a test for updating mount options

* Fix tests
2018-03-21 12:04:27 -07:00
Calvin Leung Huang f86881c295
Unauthenticated endpoint to list secret and auth mounts (#4134)
* Add audit hmac values to AuthConfigInput and AuthConfigOutput, fix docs

* docs: Add ttl params to auth enable endpoint

* Rewording of go string to simply string

* Add audit hmac keys as CLI flags on auth/secrets enable

* Fix copypasta mistake

* WIP on auth-list endpoint

* Rename variable to be singular, add CLI flag, show value in auth and secrets list

* Add audit hmac keys to auth and secrets list

* Only set config values if they exist

* Fix http sys/auth tests

* More auth plugin_name test fixes

* Rename tag internal_ui_show_mount to _ui_show_mount

* Add tests

* Make endpoint unauthed

* Rename field to listing_visibility

* Add listing-visibility to cli tune commands

* Use ListingVisiblityType

* Fix type conversion

* Do not actually change token's value on testHttpGet

* Remove unused ListingVisibilityAuth, use const in pathInternalUIMountsRead
2018-03-19 23:16:33 -04:00
Calvin Leung Huang e2fb199ce5
Non-HMAC audit values (#4033)
* Add non-hmac request keys

* Update comment

* Initial audit request keys implementation

* Add audit_non_hmac_response_keys

* Move where req.NonHMACKeys gets set

* Minor refactor

* Add params to auth tune endpoints

* Sync cache on loadCredentials

* Explicitly unset req.NonHMACKeys

* Do not error if entry is nil

* Add tests

* docs: Add params to api sections

* Refactor audit.Backend and Formatter interfaces, update audit broker methods

* Add audit_broker.go

* Fix method call params in audit backends

* Remove fields from logical.Request and logical.Response, pass keys via LogInput

* Use data.GetOk to allow unsetting existing values

* Remove debug lines

* Add test for unsetting values

* Address review feedback

* Initialize values in FormatRequest and FormatResponse using input values

* Update docs

* Use strutil.StrListContains

* Use strutil.StrListContains
2018-03-02 12:18:39 -05:00
Jeff Mitchell 847e499261 Simplify readonly view logic 2018-02-09 15:39:27 -05:00
Vishal Nayak bf66dc2841 Make mount view read only until after mount persist (#3910) 2018-02-09 14:04:25 -05:00
Vishal Nayak 150ad8405b
Remove logical.Initialize() method (#3848)
* Remove logical.Initialize() method

* More cleanup

* Fix test
2018-01-25 20:19:27 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Jeff Mitchell 88313f0398 Update replication state logic.
Fixes #3727
2018-01-16 13:51:55 -05:00
Calvin Leung Huang 7c0b4f1333 Avoid unseal failure if plugin backends fail to setup during postUnseal (#3686) 2017-12-15 13:31:57 -05:00
Vishal Nayak 0928a65c38
remove unused function (#3657) 2017-12-06 18:55:43 -05:00
Jeff Mitchell d55f94f4a3
Plumb more seal wrap stuff through and move to outside layer of mount options (#3572) 2017-11-13 11:22:22 -05:00
Jeff Mitchell 0c9e692414
Minor mount logic updates (#3553) 2017-11-07 20:30:02 -05:00
Jeff Mitchell febda30382
Fix potential panic reading local mount tables (#3552) 2017-11-07 18:04:37 -05:00
Jonathan Freedman 4109473134 More Mount Conflict Detection (#2919) 2017-11-06 15:29:09 -05:00
Jeff Mitchell d38a699c32 Make compile 2017-10-23 17:15:56 -04:00
Jeff Mitchell 3c6fe40a91 Sync 2017-10-23 15:35:28 -04:00
Vishal Nayak f7ed6732a5 Porting identity store (#3419)
* porting identity to OSS

* changes that glue things together

* add testing bits

* wrapped entity id

* fix mount error

* some more changes to core

* fix storagepacker tests

* fix some more tests

* fix mount tests

* fix http mount tests

* audit changes for identity

* remove upgrade structs on the oss side

* added go-memdb to vendor
2017-10-11 10:21:20 -07:00
Chris Hoffman 1029ad3b33 Rename "generic" secret backend to "kv" (#3292) 2017-09-15 09:02:29 -04:00
Brian Kassouf a8d9426d9f Update locking components from DR replication changes (#3283)
* Update locking components from DR replication changes

* Fix plugin backend test

* Add a comment about needing the statelock:
2017-09-04 19:38:37 -04:00
Calvin Leung Huang 48263622cb Add comment to the reason behind mounting a nil backend for plugin mounts during setup 2017-09-01 12:26:31 -04:00
Calvin Leung Huang a581e96b78 Lazy-load plugin mounts (#3255)
* Lazy load plugins to avoid setup-unwrap cycle

* Remove commented blocks

* Refactor NewTestCluster, use single core cluster on basic plugin tests

* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly

* Add special path to mock plugin

* Move ensureCoresSealed to vault/testing.go

* Use same method for EnsureCoresSealed and Cleanup

* Bump ensureCoresSealed timeout to 60s

* Correctly handle nil opts on NewTestCluster

* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap

* Check metadata flag directly on the plugin process

* Plumb isMetadataMode down to PluginRunner

* Add NOOP shims when running in metadata mode

* Remove unused flag from the APIMetadata object

* Remove setupSecretPlugins and setupCredentialPlugins functions

* Move when we setup rollback manager to after the plugins are initialized

* Fix tests

* Fix merge issue

* start rollback manager after the credential setup

* Add guards against running certain client and server functions while in metadata mode

* Call initialize once a plugin is loaded on the fly

* Add more tests, update basic secret/auth plugin tests to trigger lazy loading

* Skip mount if plugin removed from catalog

* Fixup

* Remove commented line on LookupPlugin

* Fail on mount operation if plugin is re-added to catalog and mount is on existing path

* Check type and special paths on startBackend

* Fix merge conflicts

* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
2017-09-01 01:02:03 -04:00
Calvin Leung Huang 6853d80212 Add plugin backend reload capability (#3112)
* Add plugin reload capability on all mounts for a specific plugin type

* Comments cleanup

* Add per-mount plugin backend reload, add tests

* Fix typos

* Remove old comment

* Reuse existing storage view in reloadPluginCommon

* Correctly handle reloading auth plugin backends

* Update path to plugin/backend/reload

* Use multierrors on reloadMatchingPluginMounts, attempt to reload all mounts provided

* Use internal value as check to ensure plugin backend reload

* Remove connection state from request for plugins at the moment

* Minor cleanup

* Refactor tests
2017-08-08 00:18:59 -04:00
Calvin Leung Huang bb54e9c131 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Chris Hoffman d481e65c5a Cleaning up logical and auth unmount functions (#2994) 2017-07-13 10:57:14 -07:00
Jeff Mitchell d169918465 Create and persist human-friendly-ish mount accessors (#2918) 2017-06-26 18:14:36 +01:00
Jeff Mitchell d25aa9fc21 Don't write salts in initialization, look up on demand (#2702) 2017-05-09 17:51:09 -04:00
Jeff Mitchell e3c8be72cc Fix local check on singleton required mounts 2017-05-09 08:36:10 -04:00
Jeff Mitchell e0c2b37c2a Add commenting to singletonMountTables 2017-05-08 13:39:18 -04:00
Jeff Mitchell f50d345fa3 Fix imports.
Closes #2688
2017-05-08 10:23:29 -04:00
Jeff Mitchell 726bd6f379 Don't load a required mount if in secondary mode, let sync invalidate do that 2017-05-05 19:40:26 -04:00
Jeff Mitchell 3aafb3270c Move singleton mount fetching function to mount.go and fix tests 2017-05-05 17:20:30 -04:00
Jeff Mitchell b53331d345 Add token to singleton mounts 2017-05-05 16:45:48 -04:00
Jeff Mitchell 6f6f242061 Add logic to skip initialization in some cases and some invalidation logic 2017-05-05 15:01:52 -04:00
Jeff Mitchell f03d500808 Add option to disable caching per-backend. (#2455) 2017-03-08 09:20:09 -05:00
Jeff Mitchell 8462d945d3 Add some nil checks to mounting 2017-03-04 16:43:18 -05:00
Jeff Mitchell a585f709d3 Understand local when persisting mount tables, to avoid invalidations when not necessary (#2427) 2017-03-02 14:37:59 -05:00
Jeff Mitchell 4e045d000c Create upgrade path for cubbyhole's local status 2017-02-24 10:05:44 -05:00
Jeff Mitchell 496420a5ab Make cubbyhole local instead of replicated. (#2397)
This doesn't really change behavior, just what it looks like in the UX.
However, it does make tests more complicated. Most were fixed by adding
a sorting function, which is generally useful anyways.
2017-02-18 13:51:05 -05:00
Jeff Mitchell 494b4c844b More porting from rep (#2389)
* More porting from rep

* Address feedback
2017-02-16 20:13:19 -05:00
Jeff Mitchell 47274eca88 Add cleanup functions to multiple DB backends. (#2313)
Ensure it's called on unmount, not just for seal.
2017-02-01 14:05:25 -05:00
Jeff Mitchell 2052e406d2 Move router mount back below table persistence 2017-01-17 15:15:28 -05:00
Jeff Mitchell 8e62acbd59 Sync the locking behavior between logical/auth backend (#2280) 2017-01-17 13:02:29 -05:00
vishalnayak 0778a2eba7 core: adding error server logs for failure to update mount table 2017-01-11 20:21:34 -05:00
Vishal Nayak 7367158a2a Merge pull request #2252 from hashicorp/mountentry-clone
Adding Tainted to MountEntry.Clone
2017-01-10 10:28:13 -05:00
vishalnayak 28c3f4a192 Adding Tainted to MountEntry.Clone 2017-01-10 08:32:33 -05:00
Jeff Mitchell 9e5d1eaac9 Port some updates 2017-01-06 15:42:18 -05:00
Jeff Mitchell 6d00f0c483 Adds HUP support for audit log files to close and reopen. (#1953)
Adds HUP support for audit log files to close and reopen. This makes it
much easier to deal with normal log rotation methods.

As part of testing this I noticed that HUP and other items that come out
of command/server.go are going to stderr, which is where our normal log
lines go. This isn't so much problematic with our normal output but as
we officially move to supporting other formats this can cause
interleaving issues, so I moved those to stdout instead.
2016-09-30 12:04:50 -07:00