* Start work on passing context to backends
* More work on passing context
* Unindent logical system
* Unindent token store
* Unindent passthrough
* Unindent cubbyhole
* Fix tests
* use requestContext in rollback and expiration managers
* Move location of quit channel closing in exp manager
If it happens after stopping timers any timers firing before all timers
are stopped will still run the revocation function. With plugin
auto-crash-recovery this could end up instantiating a plugin that could
then try to unwrap a token from a nil token store.
This also plumbs in core so that we can grab a read lock during the
operation and check standby/sealed status before running it (after
grabbing the lock).
* Use context instead of checking core values directly
* Use official Go context in a few key places
* Capabilities endpoint will now return considering policies on entities and groups
* refactor the policy derivation into a separate function
* Docs: Update docs to reflect the change in capabilities endpoint
* external identity groups
* add local LDAP groups as well to group aliases
* add group aliases for okta credential backend
* Fix panic in tests
* fix build failure
* remove duplicated struct tag
* add test steps to test out removal of group member during renewals
* Add comment for having a prefix check in router
* fix tests
* s/parent_id/canonical_id
* s/parent/canonical in comments and errors
* porting identity to OSS
* changes that glue things together
* add testing bits
* wrapped entity id
* fix mount error
* some more changes to core
* fix storagepacker tests
* fix some more tests
* fix mount tests
* fix http mount tests
* audit changes for identity
* remove upgrade structs on the oss side
* added go-memdb to vendor
* disable raw endpoint by default
* adding docs
* config option raw -> raw_storage_endpoint
* docs updates
* adding listing on raw endpoint
* reworking tests for enabled raw endpoints
* root protecting base raw endpoint
* Lazy load plugins to avoid setup-unwrap cycle
* Remove commented blocks
* Refactor NewTestCluster, use single core cluster on basic plugin tests
* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly
* Add special path to mock plugin
* Move ensureCoresSealed to vault/testing.go
* Use same method for EnsureCoresSealed and Cleanup
* Bump ensureCoresSealed timeout to 60s
* Correctly handle nil opts on NewTestCluster
* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap
* Check metadata flag directly on the plugin process
* Plumb isMetadataMode down to PluginRunner
* Add NOOP shims when running in metadata mode
* Remove unused flag from the APIMetadata object
* Remove setupSecretPlugins and setupCredentialPlugins functions
* Move when we setup rollback manager to after the plugins are initialized
* Fix tests
* Fix merge issue
* start rollback manager after the credential setup
* Add guards against running certain client and server functions while in metadata mode
* Call initialize once a plugin is loaded on the fly
* Add more tests, update basic secret/auth plugin tests to trigger lazy loading
* Skip mount if plugin removed from catalog
* Fixup
* Remove commented line on LookupPlugin
* Fail on mount operation if plugin is re-added to catalog and mount is on existing path
* Check type and special paths on startBackend
* Fix merge conflicts
* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
* Add automatic plugin reload
* Refactor builtin/backend
* Remove plugin reload at the core level
* Refactor plugin tests
* Add auto-reload test case
* Change backend to use sync.RWMutex, fix dangling test plugin processes
* Add a canary to plugin backends to avoid reloading many times (#3174)
* Call setupPluginCatalog before mount-related operations in postUnseal
* Don't create multiple system backends since core only holds a reference (#3176)
to one.
* Add backend plugin changes
* Fix totp backend plugin tests
* Fix logical/plugin InvalidateKey test
* Fix plugin catalog CRUD test, fix NoopBackend
* Clean up commented code block
* Fix system backend mount test
* Set plugin_name to omitempty, fix handleMountTable config parsing
* Clean up comments, keep shim connections alive until cleanup
* Include pluginClient, disallow LookupPlugin call from within a plugin
* Add wrapper around backendPluginClient for proper cleanup
* Add logger shim tests
* Add logger, storage, and system shim tests
* Use pointer receivers for system view shim
* Use plugin name if no path is provided on mount
* Enable plugins for auth backends
* Add backend type attribute, move builtin/plugin/package
* Fix merge conflict
* Fix missing plugin name in mount config
* Add integration tests on enabling auth backend plugins
* Remove dependency cycle on mock-plugin
* Add passthrough backend plugin, use logical.BackendType to determine lease generation
* Remove vault package dependency on passthrough package
* Add basic impl test for passthrough plugin
* Incorporate feedback; set b.backend after shims creation on backendPluginServer
* Fix totp plugin test
* Add plugin backends docs
* Fix tests
* Fix builtin/plugin tests
* Remove flatten from PluginRunner fields
* Move mock plugin to logical/plugin, remove totp and passthrough plugins
* Move pluginMap into newPluginClient
* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck
* Change shim logger's Fatal to no-op
* Change BackendType to uint32, match UX backend types
* Change framework.Backend Setup signature
* Add Setup func to logical.Backend interface
* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments
* Remove commented var in plugin package
* RegisterLicense on logical.Backend interface (#3017)
* Add RegisterLicense to logical.Backend interface
* Update RegisterLicense to use callback func on framework.Backend
* Refactor framework.Backend.RegisterLicense
* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs
* plugin: Revert BackendType to remove TypePassthrough and related references
* Fix typo in plugin backends docs
* Add /sys/config/audited-headers endpoint for configuring the headers that will be audited
* Remove some debug lines
* Add a persistant layer and refactor a bit
* update the api endpoints to be more restful
* Add comments and clean up a few functions
* Remove unneeded hash structure functionaility
* Fix existing tests
* Add tests
* Add test for Applying the header config
* Add Benchmark for the ApplyConfig method
* ResetTimer on the benchmark:
* Update the headers comment
* Add test for audit broker
* Use hyphens instead of camel case
* Add size paramater to the allocation of the result map
* Fix the tests for the audit broker
* PR feedback
* update the path and permissions on config/* paths
* Add docs file
* Fix TestSystemBackend_RootPaths test
* Provide base64 keys in addition to hex encoded.
Accept these at unseal/rekey time.
Also fix a bug where backup would not be honored when doing a rekey with
no operation currently ongoing.
This pulls the logical request building code into its own function so
that it's accessible from other HTTP handlers, then uses that with some
added logic to the Seal() and StepDown() commands to have meaningful
audit log entries.
Add locking around UseToken and Lookup. Have UseToken flag an entry that
needs to be revoked so that it can be done at the appropriate time, but
so that Lookup in the interm doesn't return a value.
The locking is a map of 4096 locks keyed off of the first three
characters of the token ID which should provide good distribution.
If the local Consul agent is not available while attempting to step down from active or up to active, retry once a second. Allow for concurrent changes to the state with a single registration updater. Fix standby initialization.
1: Fix path check in core to handle renew paths from the token store
that aren't simply renew/
2: Use token policy logic if token store role policies are empty
This endpoint causes the node it's hit to step down from active duty.
It's a noop if the node isn't active or not running in HA mode. The node
will wait one second before attempting to reacquire the lock, to give
other nodes a chance to grab it.
Fixes#1093
This commit splits ACL policies into more fine-grained capabilities.
This both drastically simplifies the checking code and makes it possible
to support needed workflows that are not possible with the previous
method. It is backwards compatible; policies containing a "policy"
string are simply converted to a set of capabilities matching previous
behavior.
Fixes#724 (and others).
The rollback manager was using a saved MountTable rather than the
current table, causing it to attempt to rollback unmounted mounts, and
never rollback new mounts.
In fixing this, it became clear that bad things could happen to the
mount table...the table itself could be locked, but the table pointer
(which is what the rollback manager needs) could be modified at any time
without locking. This commit therefore also returns locking to a mutex
outside the table instead of inside, and plumbs RLock/RUnlock through to
the various places that are reading the table but not holding a write
lock.
Both unit tests and race detection pass.
Fixes#771
You can now turn on and off the lease behavior in the generic backend by
using one of two factories. Core uses the normal one if it's not already
set, so unit tests can use the custom one and all stay working.
This also adds logic into core to check, when the response is coming
from a generic backend, whether that backend has leases enabled. This
adds some slight overhead.
fields in 'lookup-self'. Importantly, this also makes credential
backends use the SystemView per-backend TTL values and fixes unit tests
to expect this.
Fully fixes#527
In order to implement this efficiently, I have introduced the concept of
"singleton" backends -- currently, 'sys' and 'cubbyhole'. There isn't
much reason to allow sys to be mounted at multiple places, and there
isn't much reason you'd need multiple per-token storage areas. By
restricting it to just one, I can store that particular mount instead of
iterating through them in order to call the appropriate revoke function.
Additionally, because revocation on the backend needs to be triggered by
the token store, the token store's salt is kept in the router and
client tokens going to the cubbyhole backend are double-salted by the
router. This allows the token store to drive when revocation happens
using its salted tokens.
return a descriptive error indicating that the Secret cannot be returned
because when the token was revoked the secret was too. This prevents
confusion where credentials come back but cannot be used.
Fixes#615
/cc @armon - This is a reasonably major refactor that I think cleans up
a lot of the logic with secrets in responses. The reason for the
refactor is that while implementing Renew/Revoke in logical/framework I
found the existing API to be really awkward to work with.
Primarily, we needed a way to send down internal data for Vault core to
store since not all the data you need to revoke a key is always sent
down to the user (for example the user than AWS key belongs to).
At first, I was doing this manually in logical/framework with
req.Storage, but this is going to be such a common event that I think
its something core should assist with. Additionally, I think the added
context for secrets will be useful in the future when we have a Vault
API for returning orphaned out keys: we can also return the internal
data that might help an operator.
So this leads me to this refactor. I've removed most of the fields in
`logical.Response` and replaced it with a single `*Secret` pointer. If
this is non-nil, then the response represents a secret. The Secret
struct encapsulates all the lease info and such.
It also has some fields on it that are only populated at _request_ time
for Revoke/Renew operations. There is precedent for this sort of
behavior in the Go stdlib where http.Request/http.Response have fields
that differ based on client/server. I copied this style.
All core unit tests pass. The APIs fail for obvious reasons but I'll fix
that up in the next commit.
