Commit Graph

32 Commits

Author SHA1 Message Date
Brian Kassouf ed14061578
Raft Storage Backend (#6888)
* Work on raft backend

* Add logstore locally

* Add encryptor and unsealable interfaces

* Add clustering support to raft

* Remove client and handler

* Bootstrap raft on init

* Cleanup raft logic a bit

* More raft work

* Work on TLS config

* More work on bootstrapping

* Fix build

* More work on bootstrapping

* More bootstrapping work

* fix build

* Remove consul dep

* Fix build

* merged oss/master into raft-storage

* Work on bootstrapping

* Get bootstrapping to work

* Clean up FMS and node-id

* Update local node ID logic

* Cleanup node-id change

* Work on snapshotting

* Raft: Add remove peer API (#906)

* Add remove peer API

* Add some comments

* Fix existing snapshotting (#909)

* Raft get peers API (#912)

* Read raft configuration

* address review feedback

* Use the Leadership Transfer API to step-down the active node (#918)

* Raft join and unseal using Shamir keys (#917)

* Raft join using shamir

* Store AEAD instead of master key

* Split the raft join process to answer the challenge after a successful unseal

* get the follower to standby state

* Make unseal work

* minor changes

* Some input checks

* reuse the shamir seal access instead of new default seal access

* refactor joinRaftSendAnswer function

* Synchronously send answer in auto-unseal case

* Address review feedback

* Raft snapshots (#910)

* Fix existing snapshotting

* implement the noop snapshotting

* Add comments and switch log libraries

* add some snapshot tests

* add snapshot test file

* add TODO

* More work on raft snapshotting

* progress on the ConfigStore strategy

* Don't use two buckets

* Update the snapshot store logic to hide the file logic

* Add more backend tests

* Cleanup code a bit

* [WIP] Raft recovery (#938)

* Add recovery functionality

* remove fmt.Printfs

* Fix a few fsm bugs

* Add max size value for raft backend (#942)

* Add max size value for raft backend

* Include physical.ErrValueTooLarge in the message

* Raft snapshot Take/Restore API  (#926)

* Inital work on raft snapshot APIs

* Always redirect snapshot install/download requests

* More work on the snapshot APIs

* Cleanup code a bit

* On restore handle special cases

* Use the seal to encrypt the sha sum file

* Add sealer mechanism and fix some bugs

* Call restore while state lock is held

* Send restore cb trigger through raft log

* Make error messages nicer

* Add test helpers

* Add snapshot test

* Add shamir unseal test

* Add more raft snapshot API tests

* Fix locking

* Change working to initalize

* Add underlying raw object to test cluster core

* Move leaderUUID to core

* Add raft TLS rotation logic (#950)

* Add TLS rotation logic

* Cleanup logic a bit

* Add/Remove from follower state on add/remove peer

* add comments

* Update more comments

* Update request_forwarding_service.proto

* Make sure we populate all nodes in the followerstate obj

* Update times

* Apply review feedback

* Add more raft config setting (#947)

* Add performance config setting

* Add more config options and fix tests

* Test Raft Recovery (#944)

* Test raft recovery

* Leave out a node during recovery

* remove unused struct

* Update physical/raft/snapshot_test.go

* Update physical/raft/snapshot_test.go

* fix vendoring

* Switch to new raft interface

* Remove unused files

* Switch a gogo -> proto instance

* Remove unneeded vault dep in go.sum

* Update helper/testhelpers/testhelpers.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* Update vault/cluster/cluster.go

* track active key within the keyring itself (#6915)

* track active key within the keyring itself

* lookup and store using the active key ID

* update docstring

* minor refactor

* Small text fixes (#6912)

* Update physical/raft/raft.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* review feedback

* Move raft logical system into separate file

* Update help text a bit

* Enforce cluster addr is set and use it for raft bootstrapping

* Fix tests

* fix http test panic

* Pull in latest raft-snapshot library

* Add comment
2019-06-20 12:14:58 -07:00
Jeff Mitchell 8bcb533a1b
Create sdk/ and api/ submodules (#6583) 2019-04-12 17:54:35 -04:00
Brian Kassouf aaca35be94
Updates to recovery keys (#6152) 2019-02-01 11:29:55 -08:00
Jeff Mitchell 41649c1511 Clean up stored barrier keys after migration to shamir (#5671) 2018-11-05 14:06:39 -05:00
Jeff Mitchell 82992d6097
Seal migration (OSS) (#781) 2018-10-22 23:34:02 -07:00
Jeff Mitchell 919b968c27
The big one (#5346) 2018-09-17 23:03:00 -04:00
Jeff Mitchell 7e7163f826 Factor out a bunch of shared code 2018-05-21 17:46:32 -04:00
Jeff Mitchell 8ad0bbbc44 Address feedback 2018-05-21 16:13:38 -04:00
Jeff Mitchell 462afbd0b9 Address review feedback 2018-05-21 14:47:00 -04:00
Jeff Mitchell acce3997a8 Start of seal improvements for testing 2018-05-20 17:49:37 -04:00
Jeff Mitchell 6340add8c1 Finish non-recovery test 2018-05-20 02:42:15 -04:00
Jeff Mitchell b5868a1de7 Add some more test structure and update test 2018-05-20 00:02:45 -04:00
Jeff Mitchell 72af2d49f9 Update rekey methods to indicate proper error codes in responses 2018-05-19 23:43:48 -04:00
Jeff Mitchell a9d8be3c4d WIP 2018-05-19 21:31:45 -04:00
Vishal Nayak 28e3eb9e2c
Errwrap everywhere (#4252)
* package api

* package builtin/credential

* package builtin/logical

* package command

* package helper

* package http and logical

* package physical

* package shamir

* package vault

* package vault

* address feedback

* more fixes
2018-04-05 11:49:21 -04:00
Becca Petrin 03cf302e9a Move to "github.com/hashicorp/go-hclog" (#4227)
* logbridge with hclog and identical output

* Initial search & replace

This compiles, but there is a fair amount of TODO
and commented out code, especially around the
plugin logclient/logserver code.

* strip logbridge

* fix majority of tests

* update logxi aliases

* WIP fixing tests

* more test fixes

* Update test to hclog

* Fix format

* Rename hclog -> log

* WIP making hclog and logxi love each other

* update logger_test.go

* clean up merged comments

* Replace RawLogger interface with a Logger

* Add some logger names

* Replace Trace with Debug

* update builtin logical logging patterns

* Fix build errors

* More log updates

* update log approach in command and builtin

* More log updates

* update helper, http, and logical directories

* Update loggers

* Log updates

* Update logging

* Update logging

* Update logging

* Update logging

* update logging in physical

* prefixing and lowercase

* Update logging

* Move phyisical logging name to server command

* Fix som tests

* address jims feedback so far

* incorporate brians feedback so far

* strip comments

* move vault.go to logging package

* update Debug to Trace

* Update go-plugin deps

* Update logging based on review comments

* Updates from review

* Unvendor logxi

* Remove null_logger.go
2018-04-02 17:46:59 -07:00
Jeff Mitchell f1bd0cbe74
Use atomic values in seal to avoid some data races (#4040) 2018-02-23 17:18:48 -05:00
Jeff Mitchell 33b68ebf3d Remove context from a few extraneous places 2018-01-19 03:44:06 -05:00
Jeff Mitchell 6be5b8e8a1 Don't use context in barrier type/recovery type 2018-01-19 03:17:36 -05:00
Brian Kassouf 2f19de0305 Add context to storage backends and wire it through a lot of places (#3817) 2018-01-19 01:44:44 -05:00
Jeff Mitchell bb924a404f Update seal type names 2017-10-27 17:28:50 -04:00
Jeff Mitchell d9bef0abca Update storedBarrierKeysPath name 2017-10-25 11:59:02 -04:00
Jeff Mitchell d38a699c32 Make compile 2017-10-23 17:15:56 -04:00
Jeff Mitchell 47dae8ffc7 Sync 2017-10-23 14:59:37 -04:00
Jeff Mitchell 69eb5066dd Multi value test seal (#2281) 2017-01-17 15:43:10 -05:00
Jeff Mitchell b5225fd000 Add KeyNotFoundError to seal file 2016-10-05 17:17:33 -04:00
Jeff Mitchell 58b32e5432 Convert to logxi 2016-08-21 18:13:37 -04:00
Jeff Mitchell c1a46349fa Change to keybase openpgp fork as it has important fixes 2016-08-11 08:31:43 -04:00
vishalnayak ad7cb2c8f1 Added JSON Decode and Encode helpers.
Changed all the occurances of Unmarshal to use the helpers.
Fixed http/ package tests.
2016-07-06 12:25:40 -04:00
Jeff Mitchell b4620d5d04 Add check against seal type to catch errors before we attempt to use the data 2016-04-15 18:16:48 -04:00
Jeff Mitchell 119238149b Add Finalize method to seal. 2016-04-14 20:37:34 +00:00
Jeff Mitchell afae46feb7 SealInterface 2016-04-04 10:44:22 -04:00