* port SSCT OSS
* port header hmac key to ent and generate token proto without make command
* remove extra nil check in request handling
* add changelog
* add comment to router.go
* change test var to use length constants
* remove local index is 0 check and extra defer which can be removed after use of ExternalID
* store unauthenticated path wildcards in map
* working unauthenticated paths with basic unit tests
* refactor wildcard logic
* add parseUnauthenticatedPaths unit tests
* use parseUnauthenticatedPaths when reloading backend
* add more wildcard test cases
* update special paths doc; add changelog
* remove buggy prefix check; add test cases
* prevent false positives for prefix matches
If we ever encounter a mismatched segment, break and set a flag to
prevent false positives for prefix matches.
If it is a match we need to do a prefix check. But we should not return
unless HasPrefix also evaluates to true. Otherwise we should let the for
loop continue to check other possibilities and only return false once
all wildcard paths have been evaluated.
* refactor switch and add more test cases
* remove comment leftover from debug session
* add more wildcard path validation and test cases
* update changelong; feature -> improvement
* simplify wildcard segment matching logic
* refactor wildcard matching into func
* fix glob matching, add more wildcard validation, refactor
* refactor common wildcard errors to func
* move doc comment to logical.Paths
* optimize wildcard paths storage with pre-split slices
* fix comment typo
* fix test case after changing wildcard paths storage type
* move prefix check to parseUnauthenticatedPaths
* tweak regex, remove unneeded array copy, refactor
* add test case around wildcard and glob matching
* plugin: fix panic on router.MatchingSystemView if backend is nil
* correctly determine the plugin binary file in the directory
* docs: simplify plugin file removal
At the level of role config it doesn't mean anything to use
default-service or default-batch; that's for mount tuning. So disallow
it in tokenutil. This also fixes the fact that the switch statement
wasn't right.
When unmounting, the router entry would be tainted, preventing routing.
However, we would then unmount the router before clearing storage, so if
an error occurred the router would have forgotten the path. For auth
mounts this isn't a problem since they had a secondary check, but
regular mounts didn't (not sure why, but this is true back to at least
0.2.0). This meant you could then create a duplicate mount using the
same path which would then not conflict in the router until postUnseal.
This adds the extra check to regular mounts, and also moves the location
of the router unmount.
This also ensures that on the next router.Mount, tainted is set to the
mount entry's tainted status.
Fixes#6769
* First pass at filtered-path endpoint. It seems to be working, but there are tests missing, and possibly some optimization to handle large key sets.
* Vendor go-cmp.
* Fix incomplete vendoring of go-cmp.
* Improve test coverage. Fix bug whereby access to a subtree named X would expose existence of a the key named X at the same level.
* Add benchmarks, which showed that hasNonDenyCapability would be "expensive" to call for every member of a large folder. Made a couple of minor tweaks so that now it can be done without allocations.
* Comment cleanup.
* Review requested changes: rename some funcs, use routeCommon instead of
querying storage directly.
* Keep the same endpoint for now, but move it from a LIST to a POST and allow multiple paths to be queried in one operation.
* Modify test to pass multiple paths in at once.
* Add endpoint to default policy.
* Move endpoint to /sys/access/filtered-path.
This lets other parts of Vault that can't depend on the vault package
take advantage of the subview functionality.
This also allows getting rid of BarrierStorage and vault.Entry, two
totally redundant abstractions.
The result will still pass gofmtcheck and won't trigger additional
changes if someone isn't using goimports, but it will avoid the
piecemeal imports changes we've been seeing.
If we have a panic defer functions are run but unlocks aren't. Since we
can't really trust plugins and storage, this backs out the changes for
those parts of the request path.
* Remove a lot of deferred functions in the request path.
There is an interesting benchmark at https://www.reddit.com/r/golang/comments/3h21nk/simple_micro_benchmark_to_measure_the_overhead_of/
It shows that defer actually adds quite a lot of overhead -- maybe 100ns
per call but we defer a *lot* of functions in the request path. So this
removes some of the ones in request handling, ha, barrier, router, and
physical cache.
One meta-note: nearly every metrics function is in a defer which means
every metrics call we add could add a non-trivial amount of time, e.g.
for every 10 extra metrics statements we add 1ms to a request. I don't
know how to solve this right now without doing what I did in some of
these cases and putting that call into a simple function call that then
goes before each return.
* Simplify barrier defer cleanup
* Update kv command to use a preflight check
* Make the existing ui endpoint return the allowed mounts
* Add kv subcommand tests
* Enable `-field` in `vault kv get/put` (#4426)
* Enable `-field` in `vault kv get/put`
Fixes#4424
* Unify nil value handling
* Use preflight helper
* Update vkv plugin
* Add all the mount info when authenticated
* Add fix the error message on put
* add metadata test
* No need to sort the capabilities
* Remove the kv client header
* kv patch command (#4432)
* Fix test
* Fix tests
* Use permission denied instead of entity disabled
* Add some requirements for versioned k/v
* Add a warning message when an upgrade is triggered
* Add path help values
* Make the kv header a const
* Add the uid to mount entry instead of options map
* Pass the backend aware uuid to the mounts and plugins
* Fix comment
* Add options to secret/auth enable and tune CLI commands (#4170)
* Switch mount/tune options to use TypeKVPairs (#4171)
* switching options to TypeKVPairs, adding bool parse for versioned flag
* flipping bool check
* Fix leases coming back from non-leased pluin kv store
* add a test for updating mount options
* Fix tests
* Start work on passing context to backends
* More work on passing context
* Unindent logical system
* Unindent token store
* Unindent passthrough
* Unindent cubbyhole
* Fix tests
* use requestContext in rollback and expiration managers
* external identity groups
* add local LDAP groups as well to group aliases
* add group aliases for okta credential backend
* Fix panic in tests
* fix build failure
* remove duplicated struct tag
* add test steps to test out removal of group member during renewals
* Add comment for having a prefix check in router
* fix tests
* s/parent_id/canonical_id
* s/parent/canonical in comments and errors
* porting identity to OSS
* changes that glue things together
* add testing bits
* wrapped entity id
* fix mount error
* some more changes to core
* fix storagepacker tests
* fix some more tests
* fix mount tests
* fix http mount tests
* audit changes for identity
* remove upgrade structs on the oss side
* added go-memdb to vendor
* Lazy load plugins to avoid setup-unwrap cycle
* Remove commented blocks
* Refactor NewTestCluster, use single core cluster on basic plugin tests
* Set c.pluginDirectory in TestAddTestPlugin for setupPluginCatalog to work properly
* Add special path to mock plugin
* Move ensureCoresSealed to vault/testing.go
* Use same method for EnsureCoresSealed and Cleanup
* Bump ensureCoresSealed timeout to 60s
* Correctly handle nil opts on NewTestCluster
* Add metadata flag to APIClientMeta, use meta-enabled plugin when mounting to bootstrap
* Check metadata flag directly on the plugin process
* Plumb isMetadataMode down to PluginRunner
* Add NOOP shims when running in metadata mode
* Remove unused flag from the APIMetadata object
* Remove setupSecretPlugins and setupCredentialPlugins functions
* Move when we setup rollback manager to after the plugins are initialized
* Fix tests
* Fix merge issue
* start rollback manager after the credential setup
* Add guards against running certain client and server functions while in metadata mode
* Call initialize once a plugin is loaded on the fly
* Add more tests, update basic secret/auth plugin tests to trigger lazy loading
* Skip mount if plugin removed from catalog
* Fixup
* Remove commented line on LookupPlugin
* Fail on mount operation if plugin is re-added to catalog and mount is on existing path
* Check type and special paths on startBackend
* Fix merge conflicts
* Refactor PluginRunner run methods to use runCommon, fix TestSystemBackend_Plugin_auth
Josh Black