Commit graph

164 commits

Author SHA1 Message Date
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Josh Soref 73b1fde82f Spelling (#4119) 2018-03-20 14:54:10 -04:00
Brian Nuszkowski 76be90f384 Add PKCS1v15 as a RSA signature and verification option on the Transit secret engine (#4018)
Option to specify the RSA signature type, in specific add support for PKCS1v15
2018-03-15 09:17:02 -07:00
Jeff Mitchell 35906aaa6c
Add ChaCha20-Poly1305 support to transit (#3975) 2018-02-14 11:59:46 -05:00
Chris Hoffman 898026c58f Fix auditing for transit keys with backup/restore info (#3919) 2018-02-09 13:54:18 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Brian Kassouf 1c190d4bda
Pass context to backends (#3750)
* Start work on passing context to backends

* More work on passing context

* Unindent logical system

* Unindent token store

* Unindent passthrough

* Unindent cubbyhole

* Fix tests

* use requestContext in rollback and expiration managers
2018-01-08 10:31:38 -08:00
Vishal Nayak 15b3d8738e Transit: backup/restore (#3637) 2017-12-14 12:51:50 -05:00
Vishal Nayak 48ac5caaa9
Transit: Refactor internal representation of key entry map (#3652)
* convert internal map to index by string

* Add upgrade test for internal key entry map

* address review feedback
2017-12-06 18:24:00 -05:00
Vishal Nayak 52df62d4ff
Encrypt/Decrypt/Sign/Verify using RSA in Transit backend (#3489)
* encrypt/decrypt/sign/verify RSA

* update path-help and doc

* Fix the bug which was breaking convergent encryption

* support both 2048 and 4096

* update doc to contain both 2048 and 4096

* Add test for encrypt, decrypt and rotate on RSA keys

* Support exporting RSA keys

* Add sign and verify test steps

* Remove 'RSA' from PEM header

* use the default salt length

* Add 'RSA' to PEM header since openssl is expecting that

* export rsa keys as signing-key as well

* Comment the reasoning behind the PEM headers

* remove comment

* update comment

* Parameterize hashing for RSA signing and verification

* Added test steps to check hash algo choice for RSA sign/verify

* fix test by using 'prehashed'
2017-11-03 10:45:53 -04:00
Jeff Mitchell 3a2440a651
Check input size to avoid a panic (#3521) 2017-11-02 16:40:52 -05:00
Jeff Mitchell a25dae82dd Final sync 2017-10-23 17:39:21 -04:00
Vishal Nayak 2ede750c78 return the actual error for base64 decoding failure (#3397) 2017-10-20 11:21:45 -04:00
Jeremy Voorhis af24163abd Implement signing of pre-hashed data (#3448)
Transit backend sign and verify endpoints now support algorithm=none
2017-10-11 11:48:51 -04:00
Jeff Mitchell 1bfc6d4fe7 Add a -dev-three-node option for devs. (#3081) 2017-07-31 11:28:06 -04:00
Calvin Leung Huang 3e8aecc7d5 Add BackendType to existing backends (#3078) 2017-07-28 14:04:46 -04:00
Jeff Mitchell 45fd7dad60 Add note about ed25519 hashing to docs and path help.
Fixes #3074
Closes #3076
2017-07-28 09:30:27 -04:00
Calvin Leung Huang bb54e9c131 Backend plugin system (#2874)
* Add backend plugin changes

* Fix totp backend plugin tests

* Fix logical/plugin InvalidateKey test

* Fix plugin catalog CRUD test, fix NoopBackend

* Clean up commented code block

* Fix system backend mount test

* Set plugin_name to omitempty, fix handleMountTable config parsing

* Clean up comments, keep shim connections alive until cleanup

* Include pluginClient, disallow LookupPlugin call from within a plugin

* Add wrapper around backendPluginClient for proper cleanup

* Add logger shim tests

* Add logger, storage, and system shim tests

* Use pointer receivers for system view shim

* Use plugin name if no path is provided on mount

* Enable plugins for auth backends

* Add backend type attribute, move builtin/plugin/package

* Fix merge conflict

* Fix missing plugin name in mount config

* Add integration tests on enabling auth backend plugins

* Remove dependency cycle on mock-plugin

* Add passthrough backend plugin, use logical.BackendType to determine lease generation

* Remove vault package dependency on passthrough package

* Add basic impl test for passthrough plugin

* Incorporate feedback; set b.backend after shims creation on backendPluginServer

* Fix totp plugin test

* Add plugin backends docs

* Fix tests

* Fix builtin/plugin tests

* Remove flatten from PluginRunner fields

* Move mock plugin to logical/plugin, remove totp and passthrough plugins

* Move pluginMap into newPluginClient

* Do not create storage RPC connection on HandleRequest and HandleExistenceCheck

* Change shim logger's Fatal to no-op

* Change BackendType to uint32, match UX backend types

* Change framework.Backend Setup signature

* Add Setup func to logical.Backend interface

* Move OptionallyEnableMlock call into plugin.Serve, update docs and comments

* Remove commented var in plugin package

* RegisterLicense on logical.Backend interface (#3017)

* Add RegisterLicense to logical.Backend interface

* Update RegisterLicense to use callback func on framework.Backend

* Refactor framework.Backend.RegisterLicense

* plugin: Prevent plugin.SystemViewClient.ResponseWrapData from getting JWTs

* plugin: Revert BackendType to remove TypePassthrough and related references

* Fix typo in plugin backends docs
2017-07-20 13:28:40 -04:00
Lars Lehtonen 3f0b15826a Fix swallowed errors in builtin (#2977) 2017-07-07 08:23:12 -04:00
Jeff Mitchell 873aacf23f Don't panic in audit logs when reading transit keys. (#2970) 2017-07-05 11:25:10 -04:00
Matthew Irish d26a8ebf5e add min_encryption_version to the transit key response (#2838) 2017-06-08 13:07:18 -05:00
Jeff Mitchell a7fca34076 Add ability to specify encryption key version in transit (#2821) 2017-06-06 16:02:54 -04:00
Jeff Mitchell 3eebd5cf5a ed25519 support in transit (#2778) 2017-06-05 15:00:39 -04:00
Jeff Mitchell f7df60b131 Allow accessing Warnings directly in Response. (#2806)
A change in copystructure has caused some panics due to the custom copy
function. I'm more nervous about production panics than I am about
keeping some bad code wiping out some existing warnings, so remove the
custom copy function and just allow direct setting of Warnings.
2017-06-05 10:52:43 -04:00
Jeff Mitchell 4693881fe9 Update some path-help in datakey 2017-05-23 10:04:32 -04:00
Brian Kassouf 533dbe5d4c Update the error when no key can be found to a more clear error text (#2720) 2017-05-12 14:14:00 -04:00
Jeff Mitchell c81582fea0 More porting from rep (#2388)
* More porting from rep

* Address review feedback
2017-02-16 16:29:30 -05:00
Vishal Nayak 7f2717b74a transit: change batch input format (#2331)
* transit: change batch input format

* transit: no json-in-json for batch response

* docs: transit: update batch input format

* transit: fix tests after changing response format
2017-02-06 14:56:16 -05:00
Vishal Nayak 5fb28f53cb Transit: Support batch encryption and decryption (#2143)
* Transit: Support batch encryption

* Address review feedback

* Make the normal flow go through as a batch request

* Transit: Error out if encryption fails during batch processing

* Transit: Infer the 'derived' parameter based on 'context' being set

* Transit: Batch encryption doc updates

* Transit: Return a JSON string instead of []byte

* Transit: Add batch encryption tests

* Remove plaintext empty check

* Added tests for batch encryption, more coming..

* Added more batch encryption tests

* Check for base64 decoding of plaintext before encrypting

* Transit: Support batch decryption

* Transit: Added tests for batch decryption

* Transit: Doc update for batch decryption

* Transit: Sync the path-help and website docs for decrypt endpoint

* Add batch processing for rewrap

* transit: input validation for context

* transit: add rewrap batch option to docs

* Remove unnecessary variables from test

* transit: Added tests for rewrap use cases

* Address review feedback

* Address review feedback

* Address review feedback

* transit: move input checking out of critical path

* transit: allow empty plaintexts for batch encryption

* transit: use common structs for batch processing

* transit: avoid duplicate creation of structs; add omitempty to response structs

* transit: address review feedback

* transit: fix tests

* address review feedback

* transit: fix tests

* transit: rewrap encrypt user error should not error out

* transit: error out for internal errors
2017-02-02 14:24:20 -05:00
Jeff Mitchell f1a5a858d3 Make export errors a bit more meaningful 2017-01-30 09:25:50 -05:00
Jeff Mitchell 2e15dc93df Have transit exporting return the same structure regardless of one key or many 2017-01-28 10:37:35 -05:00
Chris Hoffman 7568a212b1 Adding support for exportable transit keys (#2133) 2017-01-23 11:04:43 -05:00
Matthew Irish cb8bbc4fbd Transit key actions (#2254)
* add supports_* for transit key reads

* update transit docs with new supports_* fields
2017-01-11 10:05:06 -06:00
vishalnayak c14a6c8666 Move policy test to keysutil package 2016-10-26 19:57:28 -04:00
vishalnayak 6d1e1a3ba5 Pulled out transit's lock manager and policy structs into a helper 2016-10-26 19:52:31 -04:00
Chris Hoffman 4b6e82afcb Add ability to list keys in transit backend (#1987) 2016-10-18 10:13:01 -04:00
Jeff Mitchell 676e7e0f07 Ensure upgrades have a valid HMAC key 2016-09-21 11:10:57 -04:00
Jeff Mitchell 0ff76e16d2 Transit and audit enhancements 2016-09-21 10:49:26 -04:00
Jeff Mitchell 197c7eae5f Allow encrypting empty ciphertext values. (#1881)
Replaces #1874
2016-09-13 12:00:04 -04:00
vishalnayak b599948e1c Use uuid.GenerateRandomBytes 2016-09-09 14:17:09 -04:00
vishalnayak 127f61473b Not exposing structs from the backend's package 2016-09-01 11:57:28 -04:00
Jeff Mitchell 1db0544b7a Use unexported kdf const names 2016-08-31 07:19:58 -04:00
Jeff Mitchell d2239d22d9 Use hkdf for transit key derivation for new keys (#1812)
Use hkdf for transit key derivation for new keys
2016-08-30 16:29:09 -04:00
Jeff Mitchell 0b113f7916 Derive nonce fully in convergent mode (#1796)
Ping #1794
2016-08-26 17:01:56 -04:00
Jeff Mitchell 2f5876dfe9 Use key derivation for convergent nonce. (#1794)
Use key derivation for convergent nonce.

Fixes #1792
2016-08-26 14:11:03 -04:00
Jeff Mitchell b69ed7ea93 Fix build 2016-08-08 17:00:59 -04:00
Jeff Mitchell 7f6c58b807 Address review feedback 2016-08-08 16:30:48 -04:00
Jeff Mitchell 606ba64e23 Remove context-as-nonce, add docs, and properly support datakey 2016-08-07 15:53:40 -04:00
Jeff Mitchell 1976bc0534 Add unit tests for convergence in non-context mode 2016-08-07 15:16:36 -04:00
Jeff Mitchell 8b1d47037e Refactor convergent encryption to make specifying a nonce in addition to context possible 2016-08-05 17:52:44 -04:00
vishalnayak cff7aada7a Fix invalid input getting marked as internal error 2016-07-28 16:23:11 -04:00
Jeff Mitchell e0c5f5f5fa Add convergence tests to transit backend 2016-07-28 11:30:52 -04:00
vishalnayak c14235b206 Merge branch 'master-oss' into json-use-number
Conflicts:
	http/handler.go
	logical/framework/field_data.go
	logical/framework/wal.go
	vault/logical_passthrough.go
2016-07-15 19:21:55 -04:00
Jeff Mitchell 57cdb58374 Switch to pester from go-retryablehttp to avoid swallowing 500 error messages 2016-07-11 21:37:46 +00:00
vishalnayak db3670c353 Fix transit tests 2016-07-06 22:04:08 -04:00
vishalnayak ad7cb2c8f1 Added JSON Decode and Encode helpers.
Changed all the occurances of Unmarshal to use the helpers.
Fixed http/ package tests.
2016-07-06 12:25:40 -04:00
Jeff Mitchell e50e331ffc Always run transit acceptance tests 2016-07-01 11:45:56 -04:00
Jeff Mitchell 2e7704ea7e Add convergent encryption option to transit.
Fixes #1537
2016-06-20 13:17:48 -04:00
Jeff Mitchell 8f437d6142 Make logical.InmemStorage a wrapper around physical.InmemBackend.
This:

* Allows removing LockingInmemStorage since the physical backend already
  locks properly
* Makes listing work properly by adhering to expected semantics of only
  listing up to the next prefix separator
* Reduces duplicated code
2016-06-06 12:03:08 -04:00
Jeff Mitchell f21b88802f Add some more tests around deletion and fix upsert status returning 2016-05-03 00:19:18 -04:00
Jeff Mitchell 7e1bdbe924 Massively simplify lock handling based on feedback 2016-05-02 23:47:18 -04:00
Jeff Mitchell 7f3613cc6e Remove some deferring 2016-05-02 22:36:44 -04:00
Jeff Mitchell fa0d389a95 Change use-hint of lockAll and lockPolicy 2016-05-02 22:36:44 -04:00
Jeff Mitchell 49c56f05e8 Address review feedback 2016-05-02 22:36:44 -04:00
Jeff Mitchell 3e5391aa9c Switch to lockManager 2016-05-02 22:36:44 -04:00
Jeff Mitchell 08b91b776d Address feedback 2016-05-02 22:36:44 -04:00
Jeff Mitchell fedc8711a7 Fix up commenting and some minor tidbits 2016-05-02 22:36:44 -04:00
Jeff Mitchell fe1f56de40 Make a non-caching but still locking variant of transit for when caches are disabled 2016-05-02 22:36:44 -04:00
Adam Shannon fb07d07ad9 all: Cleanup from running go vet 2016-04-13 14:38:29 -05:00
vishalnayak 06eeaecef6 Skip acceptance tests if VAULT_ACC is not set 2016-04-11 20:00:15 -04:00
vishalnayak fd8b023655 s/TF_ACC/VAULT_ACC 2016-04-05 15:24:59 -04:00
vishalnayak 95abdebb06 Added AcceptanceTest boolean to logical.TestCase 2016-04-05 15:10:44 -04:00
Jeff Mitchell f6092f8311 Don't run transit fuzzing if not during acceptance tests 2016-02-29 14:44:04 -05:00
Tom Ritter e5952a1c28 Typo in policy.go 2016-02-08 12:00:06 -06:00
Jeff Mitchell eb1deefac1 Introduce a locking inmem storage for unit tests that are doing concurrent things 2016-02-04 09:40:35 -05:00
Jeff Mitchell 70eeaa1519 Add transit fuzz test 2016-02-03 17:36:15 -05:00
Jeff Mitchell 159754acf2 Use capabilities to determine upsert-ability in transit. 2016-02-02 10:03:14 -05:00
Jeff Mitchell 5ef8839e48 Revert "Re-add upsert into transit. Defaults to off and a new endpoint /config"
This reverts commit dc27d012c0357f93bfd5bd8d480f3e229166307a.
2016-02-02 09:26:25 -05:00
Jeff Mitchell 1d385b4de3 Re-add upsert into transit. Defaults to off and a new endpoint /config
can be used to turn it on for a given mount.
2016-02-01 20:13:57 -05:00
Jeff Mitchell 073e755aa6 Update error return strings 2016-01-29 14:40:13 -05:00
Jeff Mitchell 3396b42c6c Address final review feedback 2016-01-29 14:33:51 -05:00
Jeff Mitchell 12bd2f430b Ensure generatePolicy checks disk, not just the cache, now that we aren't eager loading 2016-01-28 13:10:59 -05:00
Jeff Mitchell dd1b94fbd6 Remove eager loading 2016-01-28 08:59:05 -05:00
Jeff Mitchell be83340b14 Embed the cache directly 2016-01-27 21:59:20 -05:00
Jeff Mitchell 48c9f79896 Implement locking in the transit backend.
This ensures that we can safely rotate and modify configuration
parameters with multiple requests in flight.

As a side effect we also get a cache, which should provide a nice
speedup since we don't need to decrypt/deserialize constantly, which
would happen even with the physical LRU.
2016-01-27 17:03:21 -05:00
Jeff Mitchell d1b2bf3183 Move archive location; also detect first load of a policy after archive
is added and cause the keys to be copied to the archive.
2016-01-27 13:41:37 -05:00
Jeff Mitchell 369d0bbad0 Address review feedback 2016-01-27 13:41:37 -05:00
Jeff Mitchell e5a58109ec Store all keys in archive always 2016-01-27 13:41:37 -05:00
Jeff Mitchell 30ffc18c19 Add unit tests 2016-01-27 13:41:37 -05:00
Jeff Mitchell 5000711a67 Force min decrypt version to 1 if it's zero, which allows fixing problematic archiving logic 2016-01-27 13:41:37 -05:00
Jeff Mitchell 7a27dd5cb3 Fix logic bug when restoring keys 2016-01-27 13:41:37 -05:00
Jeff Mitchell 004b35be36 Fix decrementing instead of incrementing 2016-01-27 13:41:37 -05:00
Jeff Mitchell beafe25508 Initial transit key archiving work 2016-01-27 13:41:37 -05:00
Jeff Mitchell f3ce90164f WriteOperation -> UpdateOperation 2016-01-08 13:03:03 -05:00
Jeff Mitchell 3eb38d19ba Update transit backend documentation, and also return the min decryption
value in a read operation on the key.
2015-09-21 16:13:43 -04:00
Jeff Mitchell 01ee6c4fe1 Move no_plaintext to two separate paths for datakey. 2015-09-18 14:41:05 -04:00
Jeff Mitchell 448249108c Add datakey generation to transit.
Can specify 128 bits (defaults to 256) and control whether or not
plaintext is returned (default true).

Unit tests for all of the new functionality.
2015-09-18 14:41:05 -04:00
Jeff Mitchell 61398f1b01 Remove enable/disable and make deletion_allowed a configurable property. On read, return the version and creation time of each key 2015-09-18 14:41:05 -04:00
Jeff Mitchell 801e531364 Enhance transit backend:
* Remove raw endpoint from transit
* Add multi-key structure
* Add enable, disable, rewrap, and rotate functionality
* Upgrade functionality, and record creation time of keys in metadata. Add flag in config function to control the minimum decryption version, and enforce that in the decrypt function
* Unit tests for everything
2015-09-18 14:41:05 -04:00