Commit graph

141 commits

Author SHA1 Message Date
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Calvin Leung Huang f59069c22f
Don't call LeaseExtend on login renewal paths when period is provided (#3803)
* Don't call LeaseExtend on login renewal paths when period is provided

* WIP tests

* NoopBackend accept backend ttl values

* Test period value on credentials backend

* Use t.Fatalf instead

* Remove mockCoreExpiration

* Add login renewal test for approle backend

* Add resp.Auth.Period check on aws and cert backend tests

* Pass in approle's period via role's period

* Correctly set period in valid-role's role

* Add period renewal test using TestCluster and approle backend

* Check for ttl values after renewals on test
2018-01-18 12:19:18 -05:00
Brian Kassouf 1c190d4bda
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Chris Hoffman 91338d7aa2 Adding latency injector option to -dev mode for storage operations (#3289) 2017-09-11 14:49:08 -04:00
Chris Hoffman 71952b7738 ExpirationManager restoration to load in the background (#3260) 2017-09-05 11:09:00 -04:00
Jeff Mitchell fdaaaadee2 Migrate physical backends into separate packages (#3106) 2017-08-03 13:24:27 -04:00
Calvin Leung Huang bb54e9c131 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Lars Lehtonen d0a3f07b07 Stop Vault Package Tests From Swallowing Errors (#2960)
* Fix ignored error in TestAESGCMBarrier_MoveIntegrityV1().

* Fix ignored error in TestAESGCMBarrier_MoveIntegrityV2().

* Fix ignored err in TestExpiration_Tidy().

* Fix ignored error in TestSystemBackend_policyCRUD().
2017-07-04 13:58:28 -04:00
vishalnayak 3f9ec7a2e2 fix expiration tests 2017-06-30 21:31:27 -04:00
Jeff Mitchell 858deb9ca4 Don't allow parent references in file paths 2017-05-12 13:52:33 -04:00
Jeff Mitchell d25aa9fc21 Don't write salts in initialization, look up on demand (#2702) 2017-05-09 17:51:09 -04:00
Jeff Mitchell 3aafb3270c Move singleton mount fetching function to mount.go and fix tests 2017-05-05 17:20:30 -04:00
vishalnayak 1a02f9be11 Fix up the tests 2017-05-04 12:41:15 -04:00
Jeff Mitchell f8295a301d Merge branch 'master-oss' into sys-tidy-leases 2017-05-04 09:37:52 -04:00
Chris Hoffman 3d9cf89ad6 Add the ability to view and list of leases metadata (#2650) 2017-05-03 22:03:42 -04:00
vishalnayak 9f682eb9cd Test to check that leases with valid tokens are not being cleaned up 2017-05-02 18:12:03 -04:00
vishalnayak 850cda7861 Added test to check the atomicity of the lease tidy operation 2017-05-02 18:06:59 -04:00
vishalnayak 875658531b Do not duplicate log lines for invalid leases 2017-05-02 17:56:15 -04:00
vishalnayak 403fd62c61 Check if multiple leases with same invalid token is getting cleaned up 2017-05-02 17:15:26 -04:00
vishalnayak 5f70576715 Added steps to check if invalid token is properly cleaned up 2017-05-02 17:11:35 -04:00
vishalnayak 668595b902 Added a test for tidying of empty token 2017-05-02 16:54:03 -04:00
Brian Kassouf 13ec9c5dbf Load leases into the expiration manager in parallel (#2370)
* Add a benchmark for exiration.Restore

* Add benchmarks for consul Restore functions

* Add a parallel version of expiration.Restore

* remove debug code

* Up the MaxIdleConnsPerHost

* Add tests for etcd

* Return errors and ensure go routines are exited

* Refactor inmem benchmark

* Add s3 bench and refactor a bit

* Few tweaks

* Fix race with waitgroup.Add()

* Fix waitgroup race condition

* Move wait above the info log

* Add helper/consts package to store consts that are needed in cyclic packages

* Remove not used benchmarks
2017-02-16 10:16:06 -08:00
Jeff Mitchell 2c8d18ad8d Attempt to fix expiration test again 2017-01-24 11:17:48 -05:00
Jeff Mitchell 77bc6fa481 Use time.Now rather than using time as a struct 2017-01-24 10:21:41 -05:00
Jeff Mitchell 43acbea6a9 Add some newlines to a failing test to make it easier to spot differences 2017-01-23 14:08:29 -05:00
Jeff Mitchell f6044764c0 Fix revocation of leases when num_uses goes to 0 (#2190) 2016-12-16 13:11:55 -05:00
Vishal Nayak 9f1e6c7b26 Merge pull request #1607 from hashicorp/standardize-time
Remove redundant invocations of UTC() call on `time.Time` objects
2016-07-13 10:19:23 -06:00
vishalnayak fcb0b580ab Fix broken build 2016-07-08 23:16:58 -04:00
vishalnayak 55a667b8cd Fix broken build 2016-07-08 20:30:27 -04:00
vishalnayak dc690d6233 Place error check before the response check in expiration test 2016-07-08 19:01:36 -04:00
vishalnayak e09b40e155 Remove Unix() invocations on 'time.Time' objects and removed conversion of time to UTC 2016-07-08 18:30:18 -04:00
Jeff Mitchell c7d72fea90 Do some extra checking in the modified renewal check 2016-07-08 10:34:49 -04:00
Jeff Mitchell 7023eafc67 Make the API client retry on 5xx errors.
This should help with transient issues. Full control over min/max delays
and number of retries (and ability to turn off) is provided in the API
and via env vars.

Fix tests.
2016-07-06 16:50:23 -04:00
Jeff Mitchell 475b0e2d33 Add table/type checking to mounts table. 2016-05-26 12:55:00 -04:00
Adam Shannon fb07d07ad9 all: Cleanup from running go vet 2016-04-13 14:38:29 -05:00
Jeff Mitchell 75650ec1ad Keep the expiration manager from keeping old token entries.
The expiration manager would never be poked to remove token entries upon
token revocation, if that revocation was initiated in the token store
itself. It might have been to avoid deadlock, since during revocation of
tokens the expiration manager is called, which then calls back into the
token store, and so on.

This adds a way to skip that last call back into the token store if we
know that we're on the revocation path because we're in the middle of
revoking a token. That way the lease is cleaned up. This both prevents
log entries appearing for already-revoked tokens, and it also releases
timer/memory resources since we're not keeping the leases around.
2016-03-31 15:10:25 -04:00
Jeff Mitchell 6df72e6efd Merge pull request #1168 from hashicorp/revoke-force
Add forced revocation.
2016-03-09 16:59:52 -05:00
Jeff Mitchell d171931e59 Add unit test for forced revocation 2016-03-09 16:47:58 -05:00
vishalnayak 86dca39141 Fix testcase 2016-03-04 15:03:01 -05:00
Jeff Mitchell 4c87c101f7 Fix tests 2016-02-26 16:44:35 -05:00
Jeff Mitchell b2bde47b01 Pull out setting the root token ID; use the new ParseUUID method in
go-uuid instead, and revoke if there is an error.
2016-01-19 19:44:33 -05:00
Jeff Mitchell 3b994dbc7f Add the ability to generate root tokens via unseal keys. 2016-01-19 18:28:10 -05:00
Jeff Mitchell 9c5ad28632 Update deps, and adjust usage of go-uuid to match new return values 2016-01-13 13:40:08 -05:00
Jeff Mitchell e990b77d6e Address review feedback; move storage of these values to the expiration manager 2016-01-04 16:43:07 -05:00
Jeff Mitchell f2da5b639f Migrate 'uuid' to 'go-uuid' to better fit HC naming convention 2015-12-16 12:56:20 -05:00
Jeff Mitchell 395d6bead4 Fix removing secondary index from exp manager.
Due to a typo, revoking ensures that index entries are created rather
than removed. This adds a failing, then fixed test case (and helper
function) to ensure that index entries are properly removed on revoke.

Fixes #749
2015-11-04 10:50:31 -05:00
Jeff Mitchell 636d57a026 Make the token store's Create and RootToken functions non-exported.
Nothing requires them to be exported, and I don't want anything in the
future to think it's okay to simply create a root token when it likes.
2015-10-30 10:59:26 -04:00
Jeff Mitchell a9155ef85e Use split-out hashicorp/uuid 2015-10-12 14:07:12 -04:00
Jeff Mitchell c460ff10ca Push a lot of logic into Router to make a bunch of it nicer and enable a
lot of cleanup. Plumb config and calls to framework.Backend.Setup() into
logical_system and elsewhere, including tests.
2015-09-10 15:09:54 -04:00
Jeff Mitchell ea9fbb90bc Rejig Lease terminology internally; also, put a few JSON names back to their original values 2015-08-20 22:27:01 -07:00
Jeff Mitchell 93ef9a54bd Internally refactor Lease/LeaseGracePeriod into TTL/GracePeriod 2015-08-20 18:00:51 -07:00
Armon Dadgar 8bc99f8c23 helper/uuid: single generateUUID definition 2015-06-30 12:38:32 -07:00
Armon Dadgar e2b0f5dae8 vault: improve lease error message. Fixes #338 2015-06-18 15:37:08 -07:00
Armon Dadgar c849aba53a vault: Adding InternalData to Auth 2015-05-09 11:39:54 -07:00
Seth Vargo f17d65507f Use UTC in tests 2015-04-28 22:18:00 -04:00
Seth Vargo 95c8001388 Disable mlock in tests 2015-04-28 22:18:00 -04:00
Mitchell Hashimoto eef1a10e8e vault: fix more test race conditions 2015-04-28 19:17:45 -07:00
Mitchell Hashimoto 992028e23e vault: the expiration time should be relative to the issue time 2015-04-10 21:21:06 -07:00
Armon Dadgar 1e2863e2b8 vault: remove unused RevokeAll method 2015-04-10 14:59:49 -07:00
Armon Dadgar b10fbc4d83 vault: Adding token based revocation 2015-04-10 14:48:08 -07:00
Armon Dadgar 39c51ede2e vault: testing renewAuthEntry 2015-04-10 14:07:06 -07:00
Armon Dadgar 13836e8612 vault: groundwork to allow auth renew 2015-04-10 13:59:49 -07:00
Armon Dadgar 4679febdf3 logical: Refactor LeaseOptions to share between Secret and Auth 2015-04-09 12:14:04 -07:00
Armon Dadgar 7df486482b vault: Adding LeaseIssue for renew to allow limiting maximum lease length 2015-04-09 11:54:32 -07:00
Armon Dadgar 82c5d9c478 vault: Enforce non-renewability 2015-04-08 17:03:46 -07:00
Armon Dadgar 429ad7e5cb vault: Handle auth entry without lease 2015-04-08 15:43:26 -07:00
Armon Dadgar 466c7575d3 Replace VaultID with LeaseID for terminology simplification 2015-04-08 13:35:32 -07:00
Armon Dadgar 493ee49e4d vault: unify the token renew response 2015-04-06 16:35:39 -07:00
Armon Dadgar 002b2ad589 vault: Provide salted client token to logical backends 2015-04-03 14:42:39 -07:00
Armon Dadgar adaa83b48c vault: Adding RenewToken to expiration manager 2015-04-03 11:58:10 -07:00
Armon Dadgar c82fbbb8c3 vault: Support prefix based token revocation 2015-04-03 11:40:08 -07:00
Armon Dadgar 1b19a8ee1b vault: Rename RegisterLogin to RegisterAuth 2015-04-02 17:45:42 -07:00
Mitchell Hashimoto 69593cde56 remove credential/ lots of tests faililng 2015-03-30 18:07:05 -07:00
Armon Dadgar 6fd3cae2c2 vault: Adding auth/token/create endpoint 2015-03-24 15:10:46 -07:00
Armon Dadgar 65ef4f1032 vault: wire tokens into expiration manager 2015-03-23 18:11:15 -07:00
Armon Dadgar 86c9bd9083 vault: Give expiration manager a token store reference 2015-03-23 18:00:14 -07:00
Mitchell Hashimoto c349e97168 vault: clean up VaultID duplications, make secret responses clearer
/cc @armon - This is a reasonably major refactor that I think cleans up
a lot of the logic with secrets in responses. The reason for the
refactor is that while implementing Renew/Revoke in logical/framework I
found the existing API to be really awkward to work with.

Primarily, we needed a way to send down internal data for Vault core to
store since not all the data you need to revoke a key is always sent
down to the user (for example the user than AWS key belongs to).

At first, I was doing this manually in logical/framework with
req.Storage, but this is going to be such a common event that I think
its something core should assist with. Additionally, I think the added
context for secrets will be useful in the future when we have a Vault
API for returning orphaned out keys: we can also return the internal
data that might help an operator.

So this leads me to this refactor. I've removed most of the fields in
`logical.Response` and replaced it with a single `*Secret` pointer. If
this is non-nil, then the response represents a secret. The Secret
struct encapsulates all the lease info and such.

It also has some fields on it that are only populated at _request_ time
for Revoke/Renew operations. There is precedent for this sort of
behavior in the Go stdlib where http.Request/http.Response have fields
that differ based on client/server. I copied this style.

All core unit tests pass. The APIs fail for obvious reasons but I'll fix
that up in the next commit.
2015-03-19 23:11:42 +01:00
Mitchell Hashimoto 8039fc5c63 logical/framework: support renew 2015-03-19 20:20:57 +01:00
Armon Dadgar 421f73d332 vault: Removing mtype from router 2015-03-18 15:48:14 -07:00
Armon Dadgar 5f1e3e5986 vault: Testing restore 2015-03-16 15:11:47 -07:00
Armon Dadgar 703bcd8190 vault: Testing revoke and renew 2015-03-16 15:11:47 -07:00
Armon Dadgar b203c27326 vault: testing internal expiration manager methods 2015-03-16 15:11:47 -07:00
Armon Dadgar 11552f132b vault: testing expiration manager persistence 2015-03-16 15:11:46 -07:00
Armon Dadgar e85cd66b30 all: Removing fields from Lease 2015-03-16 13:29:51 -07:00
Armon Dadgar bdfa320e01 vault: First pass at expiration manager 2015-03-16 11:35:43 -07:00
Mitchell Hashimoto 12b12e578c vault: fix merge conflict + pass tests 2015-03-15 19:38:23 -07:00
Mitchell Hashimoto d1d1929192 vault: convert to logical.Request and friends 2015-03-15 14:53:41 -07:00
Armon Dadgar 9d5db1286d vault: Track the renew time 2015-03-13 11:36:24 -07:00
Armon Dadgar f0d00e77ec vault: Adding start/stop to expiration manager 2015-03-13 11:31:43 -07:00
Armon Dadgar 5ce63ea7cd vault: Adding lease registration 2015-03-13 10:56:03 -07:00