Commit graph

2009 commits

Author SHA1 Message Date
Vishal Nayak 4e4cb5289a
Fetch quota by name before updating it (#9466)
* Fix quotas update

* Update doc
2020-07-15 13:25:00 -04:00
Mark Gritter c4dbbccef3
Remove namespace from mount_point label. (#9436)
* Remove namespace from mount_point label.
* Fix the other two places where vault.token.creation is emitted.
2020-07-14 14:28:11 -05:00
Calvin Leung Huang cf354f6438
quotas: fix data race that could occur if ApplyQuota was called durin… (#9458)
* quotas: fix data race that could occur if ApplyQuota was called during a db reset

* Abstract out the locking caller

* Remove unneeded lock

* Update

Co-authored-by: Vishal Nayak <vishalnayakv@gmail.com>
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2020-07-13 11:42:39 -07:00
ncabatoff c822a4bffb
Modifying base can break tests that share the same coreConfig passed to NewTestCluster. (#9451) 2020-07-10 13:11:18 -04:00
Josh Black 38fc012817
Backport the pieces of the replication API changes (#9425) 2020-07-09 15:11:37 -07:00
Alexander Bezobchuk f7c1907683
Merge PR #9437: tests: Port #1315 from enterprise 2020-07-09 17:16:31 -04:00
Brian Kassouf f8df68b673
seal: Fix issue migrating from Auto->Shamir and improve tests (#9430)
* Fix issue migrating from Auto->Shamir and improve tests

* Undo newline

* fix panic in test

* Fix test panic
2020-07-09 12:28:17 -07:00
Alexander Bezobchuk dfb28a8fcc
Merge PR #9390: http: revert resource quota changes 2020-07-07 00:05:28 -04:00
Calvin Leung Huang 67444d85b8
test/migration: ensure that leader client is used for storage read check (#9403) 2020-07-06 16:22:07 -07:00
Scott Miller dfbc60cc49
Add mount type to request earlier, so it is populated on audit entries even if the request fails authorization (#9389) 2020-07-06 11:17:41 -05:00
Alexander Bezobchuk f1534a0ed0
Add nil check for quota manager (#9379)
* Add nil check for quota manager

* Add missing nil checks
2020-07-01 18:14:33 -07:00
Alexander Bezobchuk fb9cd9db2a
Merge PR #9372: Fix Unauthenticated list 2020-07-01 16:02:14 -04:00
Mark Gritter 707fdea702
Don't return quota error on revoke. (#9374)
Changed log messages to be clearer about quota operations.
This should fix enterprise unit test failures.
2020-07-01 14:41:42 -05:00
Calvin Leung Huang 37c0e51142
logical/system: re-introduce ns-awareness in pathInternalUIMountsRead (#9373) 2020-07-01 12:37:11 -07:00
Scott Miller a6f62359a9
Don't setup plugin reload on perf standbys (#9352) 2020-06-30 17:32:06 -05:00
Mike Jarmy 4b2cdfee72
re-enable seal migration (#9351)
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2020-06-30 18:21:18 -04:00
Scott Miller ad292bec73
Fix wrong err return value in plugin reload status command (#9348)
* Fix wrong return value (discovered when merging to ENT)

* go.mod

* go mod vendor

* Add setup plugin reload hook

* All reloads return something now
2020-06-30 13:33:30 -05:00
Scott Miller 001ee861bd
Global Plugin Reload: OSS Changes Take II (#9347)
* Carefully move changes from the plugin-cluster-reload branch into this clean branch off master.

* Don't test this at this level, adequately covered in the api level tests

* Change PR link

* go.mod

* Vendoring

* Vendor api/sys_plugins.go
2020-06-30 10:26:52 -05:00
Scott Miller e92f8f5a81
Revert global plugin reload commits (#9344)
* Revert "Some of the OSS changes were clobbered when merging with quotas out of, master (#9343)"

This reverts commit 8719a9b7c4d6ca7afb2e0a85e7c570cc17081f41.

* Revert "OSS side of Global Plugin Reload (#9340)"

This reverts commit f98afb998ae50346849050e882b6be50807983ad.
2020-06-29 17:36:22 -05:00
Scott Miller cc51427584
Some of the OSS changes were clobbered when merging with quotas out of, master (#9343)
* OSS side of Global Plugin Reload
2020-06-29 16:58:51 -05:00
Scott Miller a83fe0fc6d
OSS side of Global Plugin Reload (#9340)
* OSS side of Global Plugin Reload

* changelog++
2020-06-29 16:23:28 -05:00
ncabatoff d42ee4f7ef
Ensure "initialized" service registration tag is also present whenever Vault is unsealed, on both Consul and K8s (#8990)
* Add the initialized tag to Consul registration for parity with k8s (and for easy automated testing).  Ensure that whenever we flag Vault as unsealed, we also flag it as initialized.

* Update API docs.

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
2020-06-29 16:02:49 -04:00
Calvin Leung Huang babaa93a0f
monitor: watch for seal state during monitor request (#9341)
* monitor: watch for seal state during monitor request

* monitor: return error regardless of how upstream handles it
2020-06-29 12:58:41 -07:00
Mark Gritter 873acbefbd
Unit test that fails to demonstrate identity store problem. (#9339)
Fix test compilation error.
2020-06-29 13:33:25 -05:00
Vishal Nayak 6bd5674345
Reset quota manager during shutdown (#9331) 2020-06-29 13:23:10 -04:00
Vishal Nayak c6876fe00f
Resource Quotas: Rate Limiting (#9330) 2020-06-26 17:13:16 -04:00
Javier Ramos 16070564cb
Calculate percentage when displaying progress in tidy operation (#9233)
* Calculate percentage when displaying progress in tidy operation

* Update vault/token_store.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update vault/token_store.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-06-26 12:44:45 -07:00
Mark Gritter a5d2edece2
Enable identity entity collection. (#9324) 2020-06-25 18:54:38 -05:00
Scott Miller 57c6ae4233
Test for overflow of the capacity value (#9317) 2020-06-25 11:22:13 -05:00
Mark Gritter a54a3b6a66
Entity and alias counts (#9262)
* Added gauge collectors for entity counts.
* Entity and alias gauges.
* Locking around accessor to core.identityStore and core.mount.
2020-06-23 19:45:59 -05:00
Mark Gritter 97d415d024
Token gauge metrics implementation. (#9239)
* Token gauge metrics implementation.
* Enable gauges only when interval is nonzero.
* Added count by TTL
* Yandle "in restore mode" error specifically.
* Refactored initialization code for gauge collection processes.
* Fixed for multiple namespaces.
* Ability to disable individual gauges with environment variable.
* changelog++
2020-06-23 18:36:24 -05:00
Calvin Leung Huang c45bdca0b3
raft: add support for using backend for ha_storage (#9193)
* raft: initial work on raft ha storage support

* add note on join

* add todo note

* raft: add support for bootstrapping and joining existing nodes

* raft: gate bootstrap join by reading leader api address from storage

* raft: properly check for raft-only for certain conditionals

* raft: add bootstrap to api and cli

* raft: fix bootstrap cli command

* raft: add test for setting up new cluster with raft HA

* raft: extend TestRaft_HA_NewCluster to include inmem and consul backends

* raft: add test for updating an existing cluster to use raft HA

* raft: remove debug log lines, clean up verifyRaftPeers

* raft: minor cleanup

* raft: minor cleanup

* Update physical/raft/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/ha.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/ha.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/logical_system_raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* address feedback comments

* address feedback comments

* raft: refactor tls keyring logic

* address feedback comments

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* address feedback comments

* testing: fix import ordering

* raft: rename var, cleanup comment line

* docs: remove ha_storage restriction note on raft

* docs: more raft HA interaction updates with migration and recovery mode

* docs: update the raft join command

* raft: update comments

* raft: add missing isRaftHAOnly check for clearing out state set earlier

* raft: update a few ha_storage config checks

* Update command/operator_raft_bootstrap.go

Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>

* raft: address feedback comments

* raft: fix panic when checking for config.HAStorage.Type

* Update vault/raft.go

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* Update website/pages/docs/commands/operator/raft.mdx

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* raft: remove bootstrap cli command

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/raft.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* raft: address review feedback

* raft: revert vendored sdk

* raft: don't send applied index and node ID info if we're HA-only

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Vishal Nayak <vishalnayak@users.noreply.github.com>
2020-06-23 12:04:13 -07:00
Mark Gritter ce88903638
Add counter for creation of new entity. (#9263) 2020-06-19 18:24:05 -05:00
Mark Gritter 3e220f300a
Fix to failing test compilation. (#9272) 2020-06-19 15:19:21 -05:00
Mark Gritter b3c3635f49
Added gauges to count KV secrets. (#9250)
* Added gauges to count KV secrets.
* Use real KV implementation in test.
2020-06-19 14:01:35 -05:00
Mark Gritter 239b2375aa
Counter that increments on every secret engine lease creation. (#9244) 2020-06-18 15:36:21 -05:00
Mark Gritter cc6a64d4d8
Replaced ClusterMetricSink's cluster name with an atomic.Value. (#9252)
* Replaced ClusterMetricSink's cluster name with an atomic.Value.
This should permit go-race tests to pass which seal and unseal
the core.

* Replace metric sink before unseal to avoid data races.
2020-06-18 12:55:50 -05:00
Michael Golowka f77bcc53c4
Move sdk/helper/random -> helper/random (#9226)
* This package is new for 1.5 so this is not a breaking change.
* This is being moved because this code was originally intended to be used
within plugins, however the design of password policies has changed such
that this is no longer needed. Thus, this code doesn't need to be in the
public SDK.
2020-06-17 14:24:38 -06:00
Scott Miller 0b9a40a64e
Add a simple sealed gauge, updated when seal status changes (#9177)
* Add a simple unsealed gauge, updated when seal status changes
2020-06-17 09:50:28 -05:00
Mike Jarmy e608503139
Test Shamir-to-Transit and Transit-to-Shamir Seal Migration for post-1.4 Vault. (#9214)
* move adjustForSealMigration to vault package

* fix adjustForSealMigration

* begin working on new seal migration test

* create shamir seal migration test

* refactor testhelpers

* add VerifyRaftConfiguration to testhelpers

* stub out TestTransit

* Revert "refactor testhelpers"

This reverts commit 39593defd0d4c6fd79aedfd37df6298391abb9db.

* get shamir test working again

* stub out transit join

* work on transit join

* remove debug code

* initTransit now works with raft join

* runTransit works with inmem

* work on runTransit with raft

* runTransit works with raft

* cleanup tests

* TestSealMigration_TransitToShamir_Pre14

* TestSealMigration_ShamirToTransit_Pre14

* split for pre-1.4 testing

* add simple tests for transit and shamir

* fix typo in test suite

* debug wrapper type

* test debug

* test-debug

* refactor core migration

* Revert "refactor core migration"

This reverts commit a776452d32a9dca7a51e3df4a76b9234d8c0c7ce.

* begin refactor of adjustForSealMigration

* fix bug in adjustForSealMigration

* clean up tests

* clean up core refactoring

* fix bug in shamir->transit migration

* stub out test that brings individual nodes up and down

* refactor NewTestCluster

* pass listeners into newCore()

* simplify cluster address setup

* simplify extra test core setup

* refactor TestCluster for readability

* refactor TestCluster for readability

* refactor TestCluster for readability

* add shutdown func to TestCore

* add cleanup func to TestCore

* create RestartCore

* stub out TestSealMigration_ShamirToTransit_Post14

* refactor address handling in NewTestCluster

* fix listener setup in newCore()

* remove unnecessary lock from setSealsForMigration()

* rename sealmigration test package

* use ephemeral ports below 30000

* work on post-1.4 migration testing

* clean up pre-1.4 test

* TestSealMigration_ShamirToTransit_Post14 works for non-raft

* work on raft TestSealMigration_ShamirToTransit_Post14

* clean up test code

* refactor TestClusterCore

* clean up TestClusterCore

* stub out some temporary tests

* use HardcodedServerAddressProvider in seal migration tests

* work on raft for TestSealMigration_ShamirToTransit_Post14

* always use hardcoded raft address provider in seal migration tests

* debug TestSealMigration_ShamirToTransit_Post14

* fix bug in RestartCore

* remove debug code

* TestSealMigration_ShamirToTransit_Post14 works now

* clean up debug code

* clean up tests

* cleanup tests

* refactor test code

* stub out TestSealMigration_TransitToShamir_Post14

* set seals properly for transit->shamir migration

* migrateFromTransitToShamir_Post14 works for inmem

* migrateFromTransitToShamir_Post14 works for raft

* use base ports per-test

* fix seal verification test code

* simplify seal migration test suite

* simplify test suite

* cleanup test suite

* use explicit ports below 30000

* simplify use of numTestCores

* Update vault/external_tests/sealmigration/seal_migration_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* Update vault/external_tests/sealmigration/seal_migration_test.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* clean up imports

* rename to StartCore()

* Update vault/testing.go

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>

* simplify test suite

* clean up tests

Co-authored-by: Calvin Leung Huang <cleung2010@gmail.com>
2020-06-16 14:12:22 -04:00
Mark Gritter 50b388a93c
Changes to expiration manager to walk tokens (#9182)
* Changes to expiration manager to walk tokens (including non-expiring ones.)
* Count by namespace in token manager.
* Keep a dictionary of policy lists and deduplicate based on it.
2020-06-15 18:54:36 -05:00
Austin Gebauer a826bd1d23
fix: configutil redeclared as imported package name (#9211) 2020-06-11 18:08:46 -07:00
Michael Golowka a89f09802d
Integrate password policies into RabbitMQ secret engine (#9143)
* Add password policies to RabbitMQ & update docs
* Also updates some parts of the password policies to aid/fix testing
2020-06-11 16:08:20 -06:00
Mike Jarmy 4303790aae
Test pre-1.4 seal migration (#9085)
* enable seal wrap in all seal migration tests

* move adjustForSealMigration to vault package

* fix adjustForSealMigration

* begin working on new seal migration test

* create shamir seal migration test

* refactor testhelpers

* add VerifyRaftConfiguration to testhelpers

* stub out TestTransit

* Revert "refactor testhelpers"

This reverts commit 39593defd0d4c6fd79aedfd37df6298391abb9db.

* get shamir test working again

* stub out transit join

* work on transit join

* Revert "move resuable storage test to avoid creating import cycle"

This reverts commit b3ff2317381a5af12a53117f87d1c6fbb093af6b.

* remove debug code

* initTransit now works with raft join

* runTransit works with inmem

* work on runTransit with raft

* runTransit works with raft

* get rid of dis-used test

* cleanup tests

* TestSealMigration_TransitToShamir_Pre14

* TestSealMigration_ShamirToTransit_Pre14

* split for pre-1.4 testing

* add simple tests for transit and shamir

* fix typo in test suite

* debug wrapper type

* test debug

* test-debug

* refactor core migration

* Revert "refactor core migration"

This reverts commit a776452d32a9dca7a51e3df4a76b9234d8c0c7ce.

* begin refactor of adjustForSealMigration

* fix bug in adjustForSealMigration

* clean up tests

* clean up core refactoring

* fix bug in shamir->transit migration

* remove unnecessary lock from setSealsForMigration()

* rename sealmigration test package

* use ephemeral ports below 30000

* simplify use of numTestCores
2020-06-11 15:07:59 -04:00
ncabatoff 2ec9049ef7
Add ssh signing algorithm as a role option. (#9096) 2020-06-11 08:10:13 -04:00
ncabatoff fdba917b66
Fix feature flag persistence: we shouldn't have excluded dr primaries, they too must write feature flags. DR secondaries might not need depend on feature flags being there, but a DR primary could also be (or become) a perf primary. (#9148) 2020-06-04 13:00:33 -04:00
Mark Gritter 475fe0eede
Token creation counters (#9052)
* Add token creation counters.
* Created a utility to change TTL to bucket name.
* Add counter covering token creation for response wrapping.
* Fix namespace label, with a new utility function.
2020-06-02 13:40:54 -05:00
ncabatoff 8870b2e51c
Add mongodbatlas static roles support (#8987)
* Refactor PG container creation.
* Rework rotation tests to use shorter sleeps.
* Refactor rotation tests.
* Add a static role rotation test for MongoDB Atlas.
2020-05-29 14:21:23 -04:00
Scott Miller 12d704d97f
Provide token ttl and issue time in the audit log. (#9091)
* Populate a token_ttl and token_issue_time field on the Auth struct of audit log entries, and in the Auth portion of a response for login methods

* Revert go fmt, better zero checking

* Update unit tests

* changelog++
2020-05-29 12:30:47 -05:00
Michael Golowka b52950f884
Add user configurable password policies available to secret engines (#8637)
* Add random string generator with rules engine

This adds a random string generation library that validates random
strings against a set of rules. The library is designed for use as generating
passwords, but can be used to generate any random strings.
2020-05-27 12:28:00 -06:00