Commit graph

327 commits

Author SHA1 Message Date
Jeff Mitchell a8aa5f0dd3 go-ldap update; bump dep, api, secrets-ad plugin, and vendor 2019-11-08 11:24:46 -05:00
Jeff Mitchell 6a95e8465b
Update go-metrics (#7794) 2019-11-05 15:28:51 -05:00
Jim Kalafut a8a50df7fc
Update azure secrets plugin (#7788) 2019-11-05 10:43:28 -08:00
Calvin Leung Huang 8c31e45860
hostutil: query stats with context, update gopsutil, refactor tests (#7769)
* hostutil: query stats with context, update gopsutil, refactor tests

* go mod vendor

* minor comment wording
2019-11-01 10:12:22 -07:00
Lukasz Jagiello 05fdb2287e Correct version of vault-plugin-auth-jwt (#7773)
Based on Vault changelog JWT-71 and JWT-77 should be included in Vault
1.3.0. Unfortunetly there was a wrong version of vault-plugin-auth-jwt
included.

This fix #7771
2019-10-31 12:17:37 -07:00
Pascal Enz 33c1b7150f Rabbitmq topic permissions (#7751)
* Upgraded rabbit hole library to 2.0

* Added RabbitMQ topic permission support.

* Updated docs to cover RabbitMQ topic permissions.

* Improved comments and docs as suggested.
2019-10-30 14:19:49 -07:00
Sam Salisbury 8f0c38f78d
run go mod vendor (#7736) 2019-10-25 13:35:22 +01:00
Clint 245935447b
Vault Agent Template (#7652)
* Vault Agent Template: parse templates  (#7540)

* add template config parsing, but it's wrong b/c it's not using mapstructure

* parsing consul templates in agent config

* add additional test to configuration parsing, to cover basics

* another test fixture, rework simple test into table

* refactor into table test

* rename test

* remove flattenKeys and add other test fixture

* Update command/agent/config/config.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* return the decode error instead of swallowing it

* Update command/agent/config/config_test.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* go mod tidy

* change error checking style

* Add agent template doc

* TemplateServer: render secrets with Consul Template (#7621)

* add template config parsing, but it's wrong b/c it's not using mapstructure

* parsing consul templates in agent config

* add additional test to configuration parsing, to cover basics

* another test fixture, rework simple test into table

* refactor into table test

* rename test

* remove flattenKeys and add other test fixture

* add template package

* WIP: add runner

* fix panic, actually copy templates, etc

* rework how the config.Vault is created and enable reading from the environment

* this was supposed to be a part of the prior commit

* move/add methods to testhelpers for converting some values to pointers

* use new methods in testhelpers

* add an unblock channel to block agent until a template has been rendered

* add note

* unblock if there are no templates

* cleanups

* go mod tidy

* remove dead code

* simple test to starT

* add simple, empty templates test

* Update package doc, error logs, and add missing close() on channel

* update code comment to be clear what I'm referring to

* have template.NewServer return a (<- chan) type, even though it's a normal chan, as a better practice to enforce reading only

* Update command/agent.go

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>

* update with test

* Add README and doc.go to the command/agent directory (#7503)

* Add README and doc.go to the command/agent directory

* Add link to website

* address feedback for agent.go

* updated with feedback from Calvin

* Rework template.Server to export the unblock channel, and remove it from the NewServer function

* apply feedback from Nick

* fix/restructure rendering test

* Add pointerutil package for converting types to their pointers

* Remove pointer helper methods; use sdk/helper/pointerutil instead

* update newRunnerConfig to use pointerutil and empty strings

* only wait for unblock if template server is initialized

* drain the token channel in this test

* conditionally send on channel
2019-10-18 16:21:46 -05:00
Jeff Mitchell c4df00f193 Fix kv mod import and vendoring 2019-10-18 08:57:32 -04:00
Lexman c86fe212c0
oss changes for entropy augmentation feature (#7670)
* oss changes for entropy augmentation feature

* fix oss command/server/config tests

* update go.sum

* fix logical_system and http/ tests

* adds vendored files

* removes unused variable
2019-10-17 10:33:00 -07:00
Calvin Leung Huang d2dbb8c963
Vault Debug (#7375)
* cli: initial work on debug; server-status target

* debug: add metrics capture target (#7376)

* check against DR secondary

* debug: add compression

* refactor check into preflight func

* debug: set short test time on tests, fix exit code bug

* debug: use temp dir for output on tests

* debug: use mholt/archiver for compression

* first pass on adding pprof

* use logger for output

* refactor polling target capture logic

* debug: poll and collect replication status

* debug: poll and collect host-info; rename output files and collection refactor

* fix comments

* add archive test; fix bugs found

* rename flag name to singular target

* add target output test; scaffold other tests cases

* debug/test: add pprof and index file tests

* debug/test: add min timing check tests

* debug: fix index gen race and collection goroutine race

* debug: extend archive tests, handle race between program exit and polling goroutines

* update docstring

* debug: correctly add to pollingWg

* debug: add config target support

* debug: don't wait on interrupt shutdown; add file exists unit tests

* move pprof bits into its goroutine

* debug: skip empty metrics and some pprof file creation if permission denied, add matching unit test

* address comments and feedback

* Vault debug using run.Group (#7658)

* debug: switch to use oklog/run.Group

* debug: use context to cancel requests and interrupt rungroups.

* debug: trigger the first interval properly

* debug: metrics collection should use metrics interval

* debug: add missing continue on metrics error

* debug: remove the use of buffered chan to trigger first interval

* debug: don't shadow BaseCommand's client, properly block on interval capture failures

* debug: actually use c.cachedClient everywhere

* go mod vendor

* debug: run all pprof in goroutines; bump pprof timings in tests to reduce flakiness

* debug: update help text
2019-10-15 15:39:19 -07:00
Becca Petrin e8432f1ebe
update ad secrets plugin for check-out feature (#7617) 2019-10-14 11:17:05 -07:00
Brian Kassouf 1167fad704
Improve raft write performance by utilizing FSM Batching (#7527)
* Start benchmark work

* Add batching FSM function

* dedupe some code

* Update dependency on chunking FSM

* fix raft external tests

* fix go.mod

* Add batching test

* uncomment test

* update raft deps

* update vendor

* Update physical/raft/fsm.go

Co-Authored-By: Michel Vocks <michelvocks@gmail.com>

* Update physical/raft/fsm.go
2019-10-14 09:25:07 -06:00
ncabatoff c16e3bbceb
Cache whether we've been initialized to reduce load on storage (#7549) 2019-10-08 17:52:38 -04:00
Nick Cabatoff 85e387439e go mod vendor and go mod tidy 2019-10-04 09:14:37 -04:00
Calvin Leung Huang 3f1c7c86a0
sys: add host-info endpoint (#7330)
* sys: add host-info endpoint, add client API method

* remove old commented handler

* add http tests, fix bugs

* query all partitions for disk usage

* fix Timestamp decoding

* add comments for clarification

* dont append a nil entry on disk usage query error

* remove HostInfo from the sdk api

We can use Logical().Read(...) to query this endpoint since the payload is contained with the data object. All warnings are preserved under Secret.Warnings.

* ensure that we're testing failure case against a standby node

* add and use TestWaitStandby to ensure core is on standby

* remove TestWaitStandby

* respond with local-only error

* move HostInfo into its own helper package

* fix imports; use new no-forward handler

* add cpu times to collection

* emit clearer multierrors/warnings by collection type

* add comments on HostInfo fields
2019-10-03 09:43:52 -07:00
Jeff Mitchell 86d14691f4 Bump sdk and vendoring 2019-09-17 11:38:03 -04:00
Brian Kassouf c2905773e4
Add download headers to snapshot take API (#7369)
* Add download headers to snapshot take API

* Add content type
2019-09-06 10:34:36 -07:00
Jim Kalafut b3fbcb2809
Update JWT auth dep (#7427) 2019-09-05 10:24:08 -07:00
ncabatoff 17695e4927
Use replace to force github.com source for thrift, since git.apache.org is flaky. (#7420) 2019-09-05 10:46:08 -04:00
Jim Kalafut 051bc15da3
Bundle OCI Auth method (#7422) 2019-09-04 16:46:00 -07:00
Vu Pham a47b2faf34 Added OCI Object Storage Plugin (#6985) 2019-09-04 11:33:16 -07:00
Jeff Mitchell d2376354f7 Update API and revendor 2019-09-04 12:46:30 -04:00
Aaron Bennett 9994307c6c update dependencies (#7390) 2019-09-03 16:08:50 -04:00
Jeff Malnick 8fdb5f62c4
feat: bump vault-plugin-auth-kubernetes to pick up support for projected tokens feature (#7386) 2019-08-30 11:53:06 -07:00
Jeff Mitchell 0d39d0507a Update api 2019-08-26 15:49:51 -04:00
Becca Petrin 64ecf46fb6
rename pcf to cf maintaining backwards compat (#7346) 2019-08-26 09:55:08 -07:00
Tommy Murphy fc3f1896ad telemetry: add stackdriver metrics sink (#6957)
* telemetry: add stackdriver metrics sink

* telemetry: stackdriver go mod tidy
2019-08-20 14:47:08 -07:00
Jeff Mitchell 88e1885c1c Updating plugin deps 2019-08-14 17:23:29 -04:00
Jim Kalafut 3e7a2211bf Update PCF Auth plugin (#7306) 2019-08-14 09:43:04 -04:00
Jeff Mitchell cfffaa5f09 Updating plugin deps 2019-07-30 00:26:33 -04:00
Jeff Mitchell e3ef0d3051 Pull in updated secrets-ad plugin 2019-07-29 18:10:13 -04:00
Brian Kassouf bdfa2c7828
Add additional raft chunk test (#7192)
* Add an end-to-end raft chunk test

* Apply suggestions from code review

Co-Authored-By: Jim Kalafut <jkalafut@hashicorp.com>
2019-07-29 14:11:46 -07:00
Jeff Mitchell abcae3830f Updating plugin deps 2019-07-25 13:01:47 -04:00
Jeff Mitchell 6e22d14ff7 Updating plugin deps 2019-07-25 12:38:07 -04:00
Jeff Mitchell 0425db59ab
Raft chunk snapshotting (#7185)
Support chunking, including snapshot handling
2019-07-24 20:44:13 -04:00
Nick Cabatoff b9c9148cff Update to latest sdk. 2019-07-24 11:47:20 -04:00
Nick Cabatoff 3f1d1765ec Update sdk to grpc 1.22 for https://github.com/grpc/grpc-go/pull/2818 which caused intermittent ent test failures. 2019-07-24 11:45:58 -04:00
Jeff Mitchell fd376b4bdf Use ChunkingConfigurationStore for raft 2019-07-23 10:59:21 -04:00
Jeff Mitchell 20f2b1097e Updating plugin deps 2019-07-22 12:55:11 -04:00
Jeff Mitchell 3b22ab2486 Add chunking support to raft 2019-07-22 12:17:58 -04:00
Jim Kalafut d23ea4fdac
Update JWT dep (#7152) 2019-07-19 15:30:30 -07:00
Jeff Mitchell e88ba8b29b Updating plugin deps 2019-07-19 10:59:39 -04:00
Jeff Mitchell 0811f983be Bump raft and vendoring 2019-07-19 10:40:25 -04:00
Jeff Mitchell 03c6090b95 Mod tidy 2019-07-05 20:26:12 -04:00
Brian Kassouf 4d7d0d729a
storage/raft: When restoring a snapshot preseal first (#7011)
* storage/raft: When restoring a snapshot preseal first

* best-effort allow standbys to apply the restoreOp before sealing active node

* Don't cache the raft tls key

* Update physical/raft/raft.go

* Move pending raft peers to core

* Fix race on close bool

* Extend the leaderlease time for tests

* Update raft deps

* Fix audit hashing

* Fix race with auditing
2019-07-03 13:56:30 -07:00
Jeff Mitchell b35aa24c7f Bump auth plugins 2019-07-03 00:47:07 -04:00
Jeff Mitchell 9baf59dcdc Update auth plugins 2019-07-02 18:40:41 -04:00
Jeff Mitchell e691f7a4e9 Bump secrets-ad plugin 2019-07-01 16:14:36 -04:00
Michael Gaffney ce9f8a72b6
Bump KV dep 2019-06-27 12:08:11 -04:00
Jeff Mitchell cd8058d498 More plugin updates 2019-06-20 23:37:41 -04:00
Jeff Mitchell 6b276e8cc0 Bump some more plugins 2019-06-20 23:26:39 -04:00
Jeff Mitchell 2f4f1558cf Bump sdk/api deps 2019-06-20 23:21:52 -04:00
Jeff Mitchell 8db677ce27 Bump some plugin versions 2019-06-20 23:16:06 -04:00
Vishal Nayak 53035ce390
Raft CLI (#6893)
* raft cli

* Reuse the command's client

* Better response handling

* minor touchups
2019-06-20 21:32:00 -04:00
Jeff Mitchell 44ea96e6e6 Update go.mod for new raft 2019-06-20 18:08:32 -04:00
Brian Kassouf ed14061578
Raft Storage Backend (#6888)
* Work on raft backend

* Add logstore locally

* Add encryptor and unsealable interfaces

* Add clustering support to raft

* Remove client and handler

* Bootstrap raft on init

* Cleanup raft logic a bit

* More raft work

* Work on TLS config

* More work on bootstrapping

* Fix build

* More work on bootstrapping

* More bootstrapping work

* fix build

* Remove consul dep

* Fix build

* merged oss/master into raft-storage

* Work on bootstrapping

* Get bootstrapping to work

* Clean up FMS and node-id

* Update local node ID logic

* Cleanup node-id change

* Work on snapshotting

* Raft: Add remove peer API (#906)

* Add remove peer API

* Add some comments

* Fix existing snapshotting (#909)

* Raft get peers API (#912)

* Read raft configuration

* address review feedback

* Use the Leadership Transfer API to step-down the active node (#918)

* Raft join and unseal using Shamir keys (#917)

* Raft join using shamir

* Store AEAD instead of master key

* Split the raft join process to answer the challenge after a successful unseal

* get the follower to standby state

* Make unseal work

* minor changes

* Some input checks

* reuse the shamir seal access instead of new default seal access

* refactor joinRaftSendAnswer function

* Synchronously send answer in auto-unseal case

* Address review feedback

* Raft snapshots (#910)

* Fix existing snapshotting

* implement the noop snapshotting

* Add comments and switch log libraries

* add some snapshot tests

* add snapshot test file

* add TODO

* More work on raft snapshotting

* progress on the ConfigStore strategy

* Don't use two buckets

* Update the snapshot store logic to hide the file logic

* Add more backend tests

* Cleanup code a bit

* [WIP] Raft recovery (#938)

* Add recovery functionality

* remove fmt.Printfs

* Fix a few fsm bugs

* Add max size value for raft backend (#942)

* Add max size value for raft backend

* Include physical.ErrValueTooLarge in the message

* Raft snapshot Take/Restore API  (#926)

* Inital work on raft snapshot APIs

* Always redirect snapshot install/download requests

* More work on the snapshot APIs

* Cleanup code a bit

* On restore handle special cases

* Use the seal to encrypt the sha sum file

* Add sealer mechanism and fix some bugs

* Call restore while state lock is held

* Send restore cb trigger through raft log

* Make error messages nicer

* Add test helpers

* Add snapshot test

* Add shamir unseal test

* Add more raft snapshot API tests

* Fix locking

* Change working to initalize

* Add underlying raw object to test cluster core

* Move leaderUUID to core

* Add raft TLS rotation logic (#950)

* Add TLS rotation logic

* Cleanup logic a bit

* Add/Remove from follower state on add/remove peer

* add comments

* Update more comments

* Update request_forwarding_service.proto

* Make sure we populate all nodes in the followerstate obj

* Update times

* Apply review feedback

* Add more raft config setting (#947)

* Add performance config setting

* Add more config options and fix tests

* Test Raft Recovery (#944)

* Test raft recovery

* Leave out a node during recovery

* remove unused struct

* Update physical/raft/snapshot_test.go

* Update physical/raft/snapshot_test.go

* fix vendoring

* Switch to new raft interface

* Remove unused files

* Switch a gogo -> proto instance

* Remove unneeded vault dep in go.sum

* Update helper/testhelpers/testhelpers.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* Update vault/cluster/cluster.go

* track active key within the keyring itself (#6915)

* track active key within the keyring itself

* lookup and store using the active key ID

* update docstring

* minor refactor

* Small text fixes (#6912)

* Update physical/raft/raft.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

* review feedback

* Move raft logical system into separate file

* Update help text a bit

* Enforce cluster addr is set and use it for raft bootstrapping

* Fix tests

* fix http test panic

* Pull in latest raft-snapshot library

* Add comment
2019-06-20 12:14:58 -07:00
Clint 5309609758
Bump the Elasticsearch db dependency to latest, to pull in fixes to satisfy dbplugin.Database interface (#6929)
Merging despite the TravisCI tests failing, which do not support go modules
2019-06-19 17:40:07 -05:00
Becca Petrin 8782f2f8bb revert more unnecessary dep changes 2019-06-19 10:47:25 -07:00
Becca Petrin 75d15ae627 revert unnecessary dep updates from bad merge 2019-06-19 10:43:09 -07:00
Becca Petrin ab156603bd merge master 2019-06-19 10:24:45 -07:00
Becca Petrin 8bbf6e6fc3 update to latest plugin dependencies 2019-06-19 10:04:49 -07:00
Jeff Mitchell 659f97a0b7 Update api and sdk to remove direct gogo dep 2019-06-18 14:51:13 -04:00
Jeff Mitchell 6ddac3ad33 Bump api/sdk dep 2019-06-18 11:19:15 -04:00
Evan Phoenix 29a287648e Upgrade github.com/lib/pq to pickup SCRAM-SHA-* methods (#6895) 2019-06-17 14:45:27 -04:00
Becca Petrin 5b9d49fc2d add elasticsearch database engine 2019-06-10 09:19:11 -07:00
Becca Petrin 66aaa46588 add PCF auth method, agent, and cli handler 2019-06-06 12:26:04 -07:00
Jeff Mitchell 51eae6c3c1 Bump AWS plugin again 2019-05-28 17:37:47 -04:00
Joel Thompson 98ee4b84b4 Bump AWS SDK dependency to latest (#6788)
Also pull the latest into the local vendor
2019-05-28 16:36:32 -05:00
Jeff Mitchell ec3ea45858
Update grpc and protos (#6725)
gRPC updated to 1.20.1 which fixes a couple of important bugs.

Updates protos as well.
2019-05-13 12:09:30 -04:00
Jeff Mitchell fe8ead5f2d go mod tidy and go mod vendor 2019-04-23 20:58:55 -04:00
Jeff Mitchell ebc3fda060 Update deep to pull in default full-level-diff behavior 2019-04-19 19:52:03 -04:00
Jeff Mitchell 9a7eb54a68 Merge branch '1.1.2' into master-oss 2019-04-18 18:49:49 -04:00
Jeff Mitchell 6eaab11ab4 Bump secrets-gcp version 2019-04-15 17:54:29 -04:00
Jeff Mitchell 213b9fd1cf Update to api 1.0.1 and sdk 0.1.8 2019-04-15 14:10:07 -04:00
Jeff Mitchell cdf6f2e909 Add replace directives for api and sdk for easier dev work 2019-04-13 03:46:13 -04:00
Jeff Mitchell 9ebc57581d
Switch to go modules (#6585)
* Switch to go modules

* Make fmt
2019-04-13 03:44:06 -04:00