Commit Graph

279 Commits

Author SHA1 Message Date
Calvin Leung Huang 0a6e7ab94b
mod: update plugin versions for 1.7.0 (#11046)
* mod: update plugin versions for 1.7.0

* command/agent: fix TestCFEndToEnd test
2021-03-04 18:32:51 -08:00
Vishal Nayak 3e55e79a3f
Autopilot: Server Stabilization, State and Dead Server Cleanup (#10856)
* k8s doc: update for 0.9.1 and 0.8.0 releases (#10825)

* k8s doc: update for 0.9.1 and 0.8.0 releases

* Update website/content/docs/platform/k8s/helm/configuration.mdx

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Autopilot initial commit

* Move autopilot related backend implementations to its own file

* Abstract promoter creation

* Add nil check for health

* Add server state oss no-ops

* Config ext stub for oss

* Make way for non-voters

* s/health/state

* s/ReadReplica/NonVoter

* Add synopsis and description

* Remove struct tags from AutopilotConfig

* Use var for config storage path

* Handle nin-config when reading

* Enable testing autopilot by using inmem cluster

* First passing test

* Only report the server as known if it is present in raft config

* Autopilot defaults to on for all existing and new clusters

* Add locking to some functions

* Persist initial config

* Clarify the command usage doc

* Add health metric for each node

* Fix audit logging issue

* Don't set DisablePerformanceStandby to true in test

* Use node id label for health metric

* Log updates to autopilot config

* Less aggressively consume config loading failures

* Return a mutable config

* Return early from known servers if raft config is unable to be pulled

* Update metrics name

* Reduce log level for potentially noisy log

* Add knob to disable autopilot

* Don't persist if default config is in use

* Autopilot: Dead server cleanup (#10857)

* Dead server cleanup

* Initialize channel in any case

* Fix a bunch of tests

* Fix panic

* Add follower locking in heartbeat tracker

* Add LastContactFailureThreshold to config

* Add log when marking node as dead

* Update follower state locking in heartbeat tracker

* Avoid follower states being nil

* Pull test to its own file

* Add execution status to state response

* Optionally enable autopilot in some tests

* Updates

* Added API function to fetch autopilot configuration

* Add test for default autopilot configuration

* Configuration tests

* Add State API test

* Update test

* Added TestClusterOptions.PhysicalFactoryConfig

* Update locking

* Adjust locking in heartbeat tracker

* s/last_contact_failure_threshold/left_server_last_contact_threshold

* Add disabling autopilot as a core config option

* Disable autopilot in some tests

* s/left_server_last_contact_threshold/dead_server_last_contact_threshold

* Set the lastheartbeat of followers to now when setting up active node

* Don't use config defaults from CLI command

* Remove config file support

* Remove HCL test as well

* Persist only supplied config; merge supplied config with default to operate

* Use pointer to structs for storing follower information

* Test update

* Retrieve non voter status from configbucket and set it up when a node comes up

* Manage desired suffrage

* Consider bucket being created already

* Move desired suffrage to its own entry

* s/DesiredSuffrageKey/LocalNodeConfigKey

* s/witnessSuffrage/recordSuffrage

* Fix test compilation

* Handle local node config post a snapshot install

* Commit to storage first; then record suffrage in fsm

* No need of local node config being nili case, post snapshot restore

* Reconcile autopilot config when a new leader takes over duty

* Grab fsm lock when recording suffrage

* s/Suffrage/DesiredSuffrage in FollowerState

* Instantiate autopilot only in leader

* Default to old ways in more scenarios

* Make API gracefully handle 404

* Address some feedback

* Make IsDead an atomic.Value

* Simplify follower hearbeat tracking

* Use uber.atomic

* Don't have multiple causes for having autopilot disabled

* Don't remove node from follower states if we fail to remove the dead server

* Autopilot server removals map (#11019)

* Don't remove node from follower states if we fail to remove the dead server

* Use map to track dead server removals

* Use lock and map

* Use delegate lock

* Adjust when to remove entry from map

* Only hold the lock while accessing map

* Fix race

* Don't set default min_quorum

* Fix test

* Ensure follower states is not nil before starting autopilot

* Fix race

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-03-03 13:59:50 -05:00
Hridoy Roy 2da7de2fec
Minimal changes to solve Dependency CVEs [VAULT-871] (#11015)
* minimal changes to solve most of the cves

* cleanup

* finished go mod vendor upgrades
2021-03-01 14:35:40 -08:00
Michael Golowka 00c1acf0e1
Vendor OpenLDAP v0.4.0 (#10996) 2021-02-25 13:00:00 -07:00
Michael Golowka eb891db72d
Vendor Couchbase DB plugin v0.3.0 (#10995) 2021-02-25 12:59:45 -07:00
Michel Vocks 0273647f97
Update vault/api version (#10965)
* Update vault/api version

* go mod vendor
2021-02-24 18:19:05 +01:00
Hridoy Roy 0c17cb5a79
upgrade consul template (#10973) 2021-02-22 14:24:27 -08:00
Clint 2aff402279
Bundle new Vault plugin: Terraform secrets (#10931)
* Bundle Terraform secrets engine

* update go.mod/sum

* vendor update

* add changelog entry

* add secrets terraform
2021-02-19 16:38:56 -06:00
Hridoy Roy 4a96126d5a
Revert "Vault Dependency Upgrades [VAULT-871] (#10903)" (#10939)
This reverts commit eb74ca61fc4dcb7038f39defb127d5d639ba0ca1.
2021-02-18 15:40:18 -05:00
Hridoy Roy a26d1300e8
Vault Dependency Upgrades [VAULT-871] (#10903)
* upgrade vault dependency set

* etcd and grpc issues:

* better for tests

* testing

* all upgrades for hashicorp deps

* kubernetes plugin upgrade seems to work

* kubernetes plugin upgrade seems to work

* etcd and a bunch of other stuff

* all vulnerable packages upgraded

* k8s is broken in linux env but not locally

* test fixes

* fix testing

* fix etcd and grpc

* fix etcd and grpc

* use master branch of go-testing-interface

* roll back etcd upgrade

* have to fix grpc since other vendors pull in grpc 1.35.0 but we cant due to etcd

* rolling back in the replace directives

* a few more testing dependencies to clean up

* fix go mod vendor
2021-02-18 12:31:57 -08:00
Austin Gebauer a7531a11ea
Updates the JWT/OIDC auth plugin (#10919) 2021-02-16 17:21:35 -08:00
swayne275 6e1b183f79
Shutdown Test Cores when Tests Complete (#10912)
* Shutdown Test Cores when Tests Complete

* go mod vendor
2021-02-12 13:04:48 -07:00
Michael Golowka d22c6f9a7a
Update gopsutil & x/sys libraries (#10889) 2021-02-10 14:33:11 -07:00
Michael Golowka ec18926754
Vendor OpenLDAP dynamic secrets (#10818) 2021-02-02 11:41:47 -07:00
Hridoy Roy fa5784d789
Pull in newest consul-template from master and all corresponding dependencies [VAULT-1392] (#10756)
* pull in newest consul template with bugfix and all dependencies

* pull in newest consul template with bugfix and all dependencies

* Rename readme.md to README.md

* add changelog
2021-01-29 12:30:16 -08:00
Lauren Voswinkel 508d33e64a
Updating GCP secrets plugin (#10759)
* Update gcp secrets plugin pseudo tag
2021-01-26 09:35:49 -08:00
Lauren Voswinkel 5794c4e91e
Updating snowflake plugin to 0.1.1 (#10709) 2021-01-20 12:56:36 -08:00
Eugene R 331529fc94
Aerospike storage backend (#10131)
* add an Aerospike storage backend

* go mod vendor

* add Aerospike storage configuration docs

* review fixes

* bump aerospike client to v3.1.1

* rename the defaultHostname variable

* relocate the docs page
2021-01-12 15:26:07 -08:00
Lauren Voswinkel 7189a67a33
Adding snowflake as a bundled database secrets plugin (#10603)
* Adding snowflake as a bundled database secrets plugin

* Add snowflake-database-plugin to expected bundled plugins

* Add snowflake plugin name to the mockBuiltinRegistry
2021-01-07 09:30:24 -08:00
Calvin Leung Huang 058532406b
mod: update secrets-gcp to latest (#10558) 2020-12-14 11:42:53 -08:00
Austin Gebauer 747d49150b
Updates the OIDC/JWT auth plugin (#10546) 2020-12-14 10:07:07 -08:00
Nick Cabatoff a73e834fda
Fail build if go mod vendor changes anything. (#10524) 2020-12-10 10:09:21 -05:00
Hridoy Roy 0ada870a52
Only use entropy augmentation for root token creation [VAULT-670] (#10487)
* Only use entropy augmentation for root token creation

* changelog

* change wording of changelog entry
2020-12-04 09:44:04 -08:00
Michael Golowka cc7efd393d
MySQL - Fix username generation length bug (#10433) 2020-12-01 15:24:51 -07:00
Scott Miller 32cb144d0d
Update HCL dependency to fix ParseACLPolicy error on invalid syntax (#10156) 2020-11-30 09:17:33 -06:00
Josh Black b8ba047b35
Allow Vault Agent to run as a Windows service (#10231) 2020-11-23 14:24:32 -08:00
Calvin Leung Huang ca72dd4761
mod: update database-couchbase to v0.2.1 (#10286) 2020-10-30 14:29:54 -07:00
Calvin Leung Huang 531e2eb613
mod: update vault plugins (#10283) 2020-10-30 13:28:47 -07:00
Theron Voran a15236e664
Updating to jwt plugin@master (#10266) 2020-10-29 14:25:06 -07:00
Calvin Leung Huang 08486cdbb9
mod: update gopsutil@v2.20.9 (#10261) 2020-10-28 17:20:54 -07:00
Jonas-Taha El Sesiy b7cf4a05ff
Add support for Managed Identity auth for physical/Azure (#10189)
* Add support for Managed Identity auth for physical/Azure

Obtain OAuth token from IMDS to allow for access to Azure Blob with
short-lived dynamic credentials

Fix #7322

* add tests & update docs/dependencies
2020-10-28 15:04:26 -07:00
Michael Golowka bd79fbafb3
Add couchbase, elasticsearch, and mongodbatlas back (#10222)
Updated the `Serve` function so these can be added back into Vault
2020-10-22 17:20:17 -06:00
Michael Golowka e6c8ee24ea
DBPW - Enables AutoMTLS for DB plugins (#10220)
This also temporarily disables couchbase, elasticsearch, and
mongodbatlas because the `Serve` function needs to change signatures
and those plugins are vendored in from external repos, causing problems
when building.
2020-10-22 15:43:19 -06:00
Theron Voran 92fa04c910
Update auth/jwt to latest master (#10214)
Fixes oidc config UI, and adds EdDSA (ed25519) to supported algorithms
2020-10-22 13:59:37 -07:00
Michael Golowka f4a3bf46ed
Couchbase -> 0.2.0; Elasticsearch -> 0.6.0; MongoDBAtlas -> 0.2.0 (#10188) 2020-10-20 11:48:53 -06:00
Michael Golowka a9e7edab39
DBPW - Updated couchbase plugin (#10162) 2020-10-16 15:19:42 -06:00
Nick Cabatoff 66274607b7
OSS changes for enterprise automated snapshots (#10160) 2020-10-16 14:57:11 -04:00
Hridoy Roy bd2dc7734c
Backport leader status telemetry [VAULT-672] (#10147)
* backport VAULT-672

* backport VAULT-672

* go mod tidy

* go mod tidy

* add back indirect import

* replace go mod and go sum with master version

* go mod vendor

* more go mod vendor

Co-authored-by: Hridoy Roy <hridoyroy@Hridoys-MBP.hitronhub.home>
Co-authored-by: Hridoy Roy <hridoyroy@Hridoys-MacBook-Pro.local>
2020-10-15 14:15:58 -07:00
Brian Kassouf 84dbca38a1
Revert "Migrate internalshared out (#9727)" (#10141)
This reverts commit ee6391b691ac12ab6ca13c3912404f1d3a842bd6.
2020-10-13 16:38:21 -07:00
Aleksandr Bezobchuk d37be9af6e
Merge PR #10095: Integrated Storage Cloud Auto-Join 2020-10-13 16:26:39 -04:00
Jeff Mitchell e6881c8147
Migrate internalshared out (#9727)
* Migrate internalshared out

* fix merge issue

* fix merge issue

* go mod vendor

Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
2020-10-12 11:56:24 -07:00
Lauren Voswinkel 0b7c6d2f71
Update couchbase plugin to use v5 dbplugin (#10124)
* Update couchbase plugin to use v5 dbplugin
2020-10-09 13:47:13 -07:00
Michael Golowka 5705133c2b
Fix checksum for vault-plugin-secrets-ad (#10125) 2020-10-09 13:53:27 -06:00
Jason O'Donnell abcac87687
secrets/ad: update dependency (#10121) 2020-10-09 14:07:04 -04:00
Jason O'Donnell cf9a7373bb
Update ad plugin to v0.6.7 (#10116) 2020-10-08 17:00:45 -04:00
Jonas-Taha El Sesiy 9b599c8162
Migrate to azure-storage-blob-go (#9577)
The azure sdk for go is maintenance-only for storage, see https://github.com/Azure/azure-sdk-for-go/tree/master/storage\#azure-storage-sdk-for-go-preview
Migrate to new azure-storage-blob-go SDK
Minor test improvements

Fix #9661
2020-10-05 14:37:13 -07:00
Michel Vocks dc5a0da770
Pull latest raft updates (#10055)
* Implement raft peers metric

* Remove old peers metric

* Update vault raft dependency

* Add peer_id docs
2020-10-05 16:36:48 +02:00
Scott Miller b513af3851
Expose generic versions of KDF and symmetric crypto (#10076)
* Support salt in DeriveKey

* Revert "Support salt in DeriveKey"

This reverts commit b295ae42673308a2d66d66b53527c6f9aba92ac9.

* Refactor out key derivation, symmetric encryption, and symmetric decryption into generic functions

* comments

* comments

* go mod vendor

* bump both go.mods

* This one too

* bump

* bump

* bump

* Make the lesser used params of symmetric ops a struct

* go fmt

* Call GetKey instead of DeriveKey

* Address feedback

* Wrong rv

* Rename calls

* Assign the nonce field

* trivial change

* Check nonce len instead

* go mod vendor
2020-10-01 21:04:36 -05:00
Theron Voran 52581cd472
Add logging during awskms auto-unseal (#9794)
Adds debug and warn logging around AWS credential chain generation,
specifically to help users debugging auto-unseal problems on AWS, by
logging which role is being used in the case of a webidentity token.

Adds a deferred call to flush the log output as well, to ensure logs
are output in the event of an initialization failure.
2020-09-28 14:06:49 -07:00
Theron Voran 8b20c04eb1
Update to vault-plugin-auth-kubernetes@master (#10004) 2020-09-24 15:44:06 -07:00