Commit Graph

1957 Commits

Author SHA1 Message Date
ncabatoff 5376e87360
Don't audit replication status requests or responses. (#8877) 2020-05-21 09:20:05 -04:00
Brian Kassouf c8dde052f2
storage/raft: Advertise the configured cluster address (#9008)
* storage/raft: Advertise the configured cluster address

* Don't allow raft to start with unspecified IP

* Fix concurrent map write panic

* Add test file

* changelog++

* changelog++

* changelog++

* Update tcp_layer.go

* Update tcp_layer.go

* Only set the adverise addr if set
2020-05-18 18:22:25 -07:00
Brian Kassouf 1bb0bd489d
storage/raft: Add committed and applied indexes to the status output (#9011)
* storage/raft: Add committed and applied indexes to the status output

* Update api vendor

* changelog++

* Update http/sys_leader.go

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>

Co-authored-by: Jim Kalafut <jkalafut@hashicorp.com>
2020-05-18 16:07:27 -07:00
Calvin Leung Huang 8cefbca1c9
Refactor service registration (#8976)
* serivceregistration: refactor service registration logic to run later

* move state check to the internal func

* sr/kubernetes: update setInitialStateInternal godoc

* sr/kubernetes: remove return in setInitialState

* core/test: fix mockServiceRegistration

* address review feedback
2020-05-15 11:06:58 -07:00
Jeff Mitchell b4f5d38916
Update to latest go-kms-wrapping and fix protos/etcd (#8996) 2020-05-14 18:45:10 -04:00
Jeff Mitchell 1d3d89e2aa
Create configutil and move some common config and setup functions there (#8362) 2020-05-14 09:19:27 -04:00
Mike Jarmy 724af764bb
Test reusable storage (#8983)
* stub out reusable storage test

* implement reusable inmem test

* work on reusable raft test

* stub out simple raft test

* switch to reusable raft storage

* cleanup tests

* cleanup tests

* refactor tests

* verify raft configuration

* cleanup tests

* stub out reuseStorage

* use common base address across clusters

* attempt to reuse raft cluster

* tinker with test

* fix typo

* start debugging

* debug raft configuration

* add BaseClusterListenPort to TestCluster options

* use BaseClusterListenPort in test

* raft join works now

* misc cleanup of raft tests

* use configurable base port for raft test

* clean up raft tests

* add parallelized tests for all backends

* clean up reusable storage tests

* remove debugging code from startClusterListener()

* improve comments in testhelpers

* improve comments in teststorage

* improve comments and test logging

* fix typo in vault/testing

* fix typo in comments

* remove debugging code

* make number of cores parameterizable in test
2020-05-14 08:31:02 -04:00
Mark Gritter bd766d7bae
Metrics wrapper that adds the cluster name as a label. (#8961) 2020-05-12 21:00:59 -05:00
Scott Miller c3b6db2f26
Reject requests read and write requests to cubbyhole with an empty path (#8971)
* Reject requests read and write requests to cubbyhole with an empty path
2020-05-11 14:15:36 -05:00
Calvin Leung Huang cef02f4803
sys: use full mount path for checkAccess on internal/ui/mounts/:path (#8962) 2020-05-08 16:02:40 -07:00
ncabatoff 862beef59d
Factor out test logger creation so that it can be used when the caller wants to specify a logger explicitly, e.g. because they also want to use it when creating storage. (#8916) 2020-05-05 12:11:36 -04:00
ncabatoff a6809fdbaf
Write NewTestCluster logs to files, which get deleted if the test passes (#8911) 2020-05-01 12:26:26 -04:00
Calvin Leung Huang ec8448ab56
token: disallow periods on custom token IDs (#8646)
* token: disallow periods on custom token IDs

* docs: update token API docs
2020-04-27 09:39:33 -07:00
Dustin Decker 08571a0ac3
Add identity num_entities gauge metric (#8816)
Signed-off-by: Dustin Decker <dustindecker@protonmail.com>
2020-04-23 19:29:42 -05:00
Brian Kassouf 21cdba6fb5
identity: Add batch entity deletion endpoint (#8785)
* identity: Add batch entity deletion endpoint

* Update the parameter description

* Update error message

* Update helper/storagepacker/storagepacker.go

Co-Authored-By: Vishal Nayak <vishalnayak@users.noreply.github.com>

* Review feedback

* Update vault/identity_store_entities.go

Co-Authored-By: Calvin Leung Huang <cleung2010@gmail.com>

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-04-23 15:25:13 -07:00
Brian Kassouf 33b3e6857e
identity: group refresh shouldn't lock unless an update is needed (#8795) 2020-04-23 11:31:22 -07:00
Calvin Leung Huang 2659c34910
raft: check for nil on concrete type in SetupCluster (#8784)
* raft: check for nil on concrete type in SetupCluster

* raft: move check to its own func

* raft: func cleanup

* raft: disallow disable_clustering = true when raft storage is used

* docs: update disable_clustering to mention new behavior
2020-04-21 13:45:07 -07:00
Calvin Leung Huang df23b481a6
core: change rawConfig to be atomic.Value (#8755)
This avoids SetConfig from having to grab a write lock which is called on a SIGHUP, and may block, along with a long-running requests that has a read lock held, any other operation that requires a state lock.
2020-04-16 16:34:46 -07:00
Vishal Nayak b93bd39b97
Support unwrapping tokens that does not contain data (#8714)
* Support unwrapping tokens that does not contain data

* s/token/secret
2020-04-09 14:09:23 -04:00
Calvin Leung Huang c3caac76f1
token/renewal: return full set of token and identity policies in the … (#8535)
* token/renewal: return full set of token and identity policies in the policies field

* extend tests to cover additional token and identity policies on a token

* verify identity_policies returned on login and renewals
2020-03-31 13:36:45 -07:00
ncabatoff 236eb7e19f
Fix a test failure I observed on ent re cluster listener (#8647)
Panics when the cluster listener changes while we're setting up request forwarding.
2020-03-31 13:47:39 -04:00
Brian Kassouf 6b8b02f77a
identity: Fix potential deadlock on error to load groups (#8613) 2020-03-25 09:42:23 -07:00
ncabatoff c6518cc3f0
Make sure if a user gets removed from all groups in the external system, Vault updates itself accordingly. This is CVE-2020-10660. (#8606) 2020-03-23 18:00:26 -04:00
Brian Kassouf 05eea911bd
storage/raft: Refresh TLS keyring on snapshot restore (#8546) 2020-03-13 13:39:14 -07:00
ncabatoff 5fe1ab766b
Add option to detect deadlocks in Core.stateLock using build tag `deadlock` (#8524) 2020-03-10 16:01:20 -04:00
Jim Kalafut 90e8d9267d
Fix identity token caching (#8412)
The namespace-partitioned cache flushing was not being used correctly,
which could leave standby nodes with stale information.

Fixes #8284
2020-02-26 15:56:19 -05:00
ncabatoff e5721310ac
Add persistent feature flags to be used on enterprise non-primaries. (#8391) 2020-02-19 18:06:53 -05:00
Jeff Mitchell 844b2c3a5d Bump API/SDK and adapt to move from SDK stuff 2020-02-15 14:58:05 -05:00
Vitaly Velikodny 3ae6568512
Add missed description field for GET /sys/auth/:path/tune endpoint (#8193)
* fix #7623: add missed description field for GET /sys/auth/:path/tune endpoint

* fix #7623: allow empty description

* fix #7623: update tests with description field
2020-02-15 10:32:47 -08:00
ncabatoff 9dd18d8487
Unless we've been asked to stop, most failures should not result in (#7732)
waitForLeadership returning.
2020-02-14 18:28:37 -08:00
Joe Dollard 8f74b4d2b6
provide vault server flag to exit on core shutdown (#7561)
* provide vault server flag to exit on core shutdown

* Update command/server.go

Co-Authored-By: Jeff Mitchell <jeffrey.mitchell@gmail.com>

Co-authored-by: Jeff Mitchell <jeffrey.mitchell@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-02-14 18:07:31 -08:00
Brian Kassouf 3bbd9dc34c
Port some changes back to OSS (#8359) 2020-02-14 16:39:13 -08:00
Security Sauce d74463bbcc
Identity: Allow specifying a custom OIDC client_id field (#8165) 2020-02-13 23:15:35 -08:00
Vishal Nayak c87d34d1a4
Seal migration with Raft (#8103)
* Seal migration after unsealing

* Refactor migration fields migrationInformation in core

* Perform seal migration as part of postUnseal

* Remove the sleep logic

* Use proper seal in the unseal function

* Fix migration from Auto to Shamir

* Fix the recovery config missing issue

* Address the non-ha migration case

* Fix the multi cluster case

* Avoid re-running seal migration

* Run the post migration code in new leaders

* Fix the issue of wrong recovery being set

* Address review feedback

* Add more complete testing coverage for seal migrations.   (#8247)

* Add more complete testing coverage for seal migrations.  Also remove VAULT_ACC gate from some tests that just depend on docker, cleanup dangling recovery config in storage after migration, and fix a call in adjustCoreForSealMigration that seems broken.

* Fix the issue of wrong recovery key being set

* Adapt tests to work with multiple cores.

* Add missing line to disable raft join.

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* Fix all known issues

* Remove warning

* Review feedback.

* Revert my previous change that broke raft tests.  We'll need to come back and at least comment
this once we better understand why it's needed.

* Don't allow migration between same types for now

* Disable auto to auto tests for now since it uses migration between same types which is not allowed

* Update vault/core.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Add migration logs

* Address review comments

* Add the recovery config check back

* Skip a few steps if migration is already done

* Return from waitForLeadership if migration fails

Co-authored-by: ncabatoff <nick.cabatoff@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-02-13 16:27:31 -05:00
Calvin Leung Huang dac3382e15
external_tests: ensure derived cores are stable before proceeding on tests (#8342)
* external_tests: ensure derived cores are stable before proceeding on tests

* testhelpers: add min duration tolerance when checking stability on derived core
2020-02-13 13:18:53 -08:00
Jeff Malnick 27ad920399
Add redshift database plugin (#8299)
* feat: add redshift database plugin

* build: update vendored libraries

* docs: add reference doc for redshift variant of the database secrets engine

* feat: set middlewear type name for better metrics naming (#8346)

Co-authored-by: Becca Petrin <beccapetrin@gmail.com>
2020-02-13 09:42:30 -08:00
Jeff Mitchell 86327b8010 Bump api/sdk and fix imports 2020-02-13 10:41:16 -05:00
Jim Kalafut 2ee7b76469
Bundle MongoDB Atlas (#8309) 2020-02-07 14:09:39 -08:00
ncabatoff a0ac4bdd2a
Audit generate root requests and responses. (#8301) 2020-02-06 11:56:37 -05:00
ncabatoff 2c8b012d14
Changes needed so that benchmark-vault can run with Prometheus monitoring (#8295) 2020-02-05 13:45:16 -05:00
Sebastien Williams-Wynn 9f99ff4912
Fix minor typo in doc string (#8277) 2020-02-02 20:12:59 +01:00
ncabatoff 30d262f149
Fix flaky test of api renewer by moving away from legacy api. (#8265) 2020-01-30 15:12:21 -05:00
Jim Kalafut f17fc4e5c1
Run goimports (#8251) 2020-01-27 21:11:00 -08:00
Becca Petrin 759f9b38f7
Observer pattern for service registration interface (#8123)
* use observer pattern for service discovery

* update perf standby method

* fix test

* revert usersTags to being called serviceTags

* use previous consul code

* vault isnt a performance standby before starting

* log err

* changes from feedback

* add Run method to interface

* changes from feedback

* fix core test

* update example
2020-01-24 09:42:03 -08:00
Vishal Nayak fb93b0bf9c Use Shamir as KeK when migrating from auto-seal to shamir (#8172)
* Use Shamir as KeK when migrating from auto-seal to shamir

* Use the correct number of shares/threshold for the migrated seal.

* Fix log message

* Add WaitForActiveNode to test

* Make test fail

* Minor updates

* Test with more shares and a threshold

* Add seal/unseal step to the test

* Update the logic that prepares seal migration (#8187)

* Update the logic that preps seal migration

* Add test and update recovery logic

Co-authored-by: ncabatoff <nick.cabatoff@gmail.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-01-21 09:24:33 -08:00
Brian Kassouf 1e37eadfa0
Fix panic when listener fails to startup (#8174) 2020-01-17 09:07:53 -08:00
Brian Kassouf f32a86ee7a
Create network layer abstraction to allow in-memory cluster traffic (#8173) 2020-01-16 23:03:02 -08:00
Calvin Leung Huang 67c0773df9
ldap, okta: fix renewal when login policies are empty (#8072)
* ldap, okta: fix renewal when login policies are empty

* test/policy: add test for login renewal without configured policy

* test/policy: remove external dependency on tests, refactor lease duration check
2020-01-16 09:42:35 -08:00
Vishal Nayak 8891f2ba88 Raft retry join (#7856)
* Raft retry join

* update

* Make retry join work with shamir seal

* Return upon context completion

* Update vault/raft.go

Co-Authored-By: Brian Kassouf <briankassouf@users.noreply.github.com>

* Address some review comments

* send leader information slice as a parameter

* Make retry join work properly with Shamir case. This commit has a blocking issue

* Fix join goroutine exiting before the job is done

* Polishing changes

* Don't return after a successful join during unseal

* Added config parsing test

* Add test and fix bugs

* minor changes

* Address review comments

* Fix build error

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2020-01-13 17:02:16 -08:00
Jeff Mitchell a0694943cc
Migrate built in auto seal to go-kms-wrapping (#8118) 2020-01-10 20:39:52 -05:00