Commit Graph

2174 Commits

Author SHA1 Message Date
Hridoy Roy 22cab6185d
[VAULT-1441] Fix race that allowed remounting on path used by another mount (#11453)
* remount concurrent test fix

* changelog

* Update changelog/11453.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-04-24 09:05:41 -07:00
Angel Garbarino 2e35e9578c
UI/obscure secret on input (#11284)
* new font and add as font-family to be used in masked-input

* clean up logic

* refactor for displayOnly

* start cert masking

* work on certificates

* upload cert work

* fix global styling

* fix styling for class no longer used

* make mask by default and remove option

* glimmerize start and certificate on LDAP a file field

* glimmerize actions

* first part of glimmerizing text-file still need to do some clean up

* not doing awesome over here

* getting ready to un-glimmer

* unglimmerize

* remove placeholder based on conversations with design

* clean up text-file

* cleanup

* fix class bindings

* handle class binding

* set up for test

* fix elementId

* track down index

* update masked-input test

* add more to the masked-input test

* test-file test

* fix broken test

* clear old style

* clean up

* remove pgp key masked font, this really needs to be refactored to text-file component

* changelog

* cover other certificate view

* add allowCopy

* address some pr styling comments

* improve test coverage

* fix some issues

* add attr.options.masked
2021-04-22 08:58:37 -06:00
Josh Black 06809930a3
Add HTTP response headers for hostname and raft node ID (if applicable) (#11289) 2021-04-20 15:25:04 -07:00
Nick Cabatoff b7ed30afed
Poll until token disappears from storage. (#11424) 2021-04-20 14:55:10 -04:00
Nick Cabatoff 75ec35421d
Refactor TestTokenStore_CubbyholeTidy so that it can be tested on ent with a non-root namespace. (#11422) 2021-04-20 14:49:25 -04:00
Brian Kassouf c400dfdd3d
core/token: fix panic looking up invalid batch tokens (#11415) 2021-04-19 18:26:51 -07:00
Nick Cabatoff 474c4e8134
Make cubbyhole revocation/tidying compatible with cubbys in namespaces. (#11408) 2021-04-19 17:28:04 -04:00
Nick Cabatoff a8023e0fdb
Add support for unauthenticated pprof access on a per-listener basis,… (#11324)
* Add support for unauthenticated pprof access on a per-listener basis, as we do for metrics.

* Add missing pprof sub-targets like 'allocs' and 'block'.  Capture the goroutine subtarget a second time in text form.  This is mostly a convenience, but also I think the pprof format might be a bit lossy?
2021-04-19 14:30:59 -04:00
Nick Cabatoff 541ae8636c
On lease deletion, also delete non-orphan batch token parent index (#11377) 2021-04-16 17:03:22 -04:00
Nick Cabatoff b07a10331f
Add metrics for requests forwarded by standbys. (#11366) 2021-04-16 14:02:20 -04:00
Nick Cabatoff 242d258e94
Fix goroutine leak caused by updating rate quotas (#11371)
Make sure that when we modify a rate quota, we stop the existing goroutine before starting the new one.
2021-04-16 14:00:01 -04:00
Nick Cabatoff 33dd025278
When a standby does a ForwardRequest, it's not using the request context, and thus not getting timed out properly when it takes too long. (#11322)
The rpcClientConnContext is still used to terminate gRPC internal/dialer-related goroutines, but the actual RPC is now timed out when the request times out, e.g. due to the default max request duration.  This mirrors what we do with the parallel forwarding code in ENT.
2021-04-15 10:23:26 -04:00
Hridoy Roy 996f114ad7
TLS Diagnose Formatting Fixes (#11342)
* diagnose formatting fixes

* diagnose formatting fixes
2021-04-12 10:55:33 -07:00
Hridoy Roy fde9f2f71d
Add More TLS Tests and Verification of TLS Root Certificate (#11300)
* tls tests and root verification

* make the certificate verification check correct for non root CA case

* add expiry test

* addressed comments but struggling with the bug in parsing Cas and inters from single file:

* final checks on tls and listener

* cleanup
2021-04-12 08:39:40 -07:00
Vishal Nayak 4666f40925
Support autopilot when raft is for HA only (#11260) 2021-04-12 09:33:21 -04:00
Brian Kassouf 303c2aee7c
Run a more strict formatter over the code (#11312)
* Update tooling

* Run gofumpt

* go mod vendor
2021-04-08 09:43:39 -07:00
Brian Kassouf de0253056c
Fix a few static analysis findings (#11307) 2021-04-07 16:48:40 -07:00
Nick Cabatoff c2673ee86a
Move SanitizedConfig back to a shared-ent file. (#11291) 2021-04-07 10:25:05 -04:00
Hridoy Roy 049f2513e6
Initial Diagnose Command for TLS and Listener [VAULT-1896, VAULT-1899] (#11249)
* sanity checks for tls config in diagnose

* backup

* backup

* backup

* added necessary tests

* remove comment

* remove parallels causing test flakiness

* comments

* small fix

* separate out config hcl test case into new hcl file

* newline

* addressed comments

* addressed comments

* addressed comments

* addressed comments

* addressed comments

* reload funcs should be allowed to be nil
2021-04-06 16:40:43 -07:00
Nick Cabatoff 44c00cd54f
Fix: leader_tls_servername raft option only worked when used with mTLS and/or an explicit CA cert. (#11252) 2021-04-06 09:16:54 -04:00
Nick Cabatoff df7404e67e
Fix deadlock in inmemlayer (#11225)
Also tweak autopilot test timings to adapt to things running faster.
2021-03-31 13:40:26 -04:00
swayne275 532093d065
fix end date calculation (#11232) 2021-03-30 09:58:45 -06:00
Scott Miller c44f18e3e4
Use pointers to 64-bit atomics to enforce alignment in AES-GCM Barrier (#11211)
* Align atomics in barrier

* Use the pointer solution
2021-03-26 14:51:56 -05:00
Nick Cabatoff 41d9030fbb
Disable autopilot in raft-ha mode. (#11181)
* Disable autopilot in raft-ha mode.

* Also don't run autopilot on DR secondaries.
2021-03-23 14:13:44 -07:00
Brian Kassouf d01a068929
Remove retry from new raft test (#11158) 2021-03-19 12:41:57 -07:00
Nick Cabatoff b3af58d758
Expose snapshot_interval tunable instead of setting it in prod code for the sake of a test. (#11160) 2021-03-19 15:41:42 -04:00
Brian Kassouf 28aba513f2
storage/raft: Ensure peers are informed of their correct suffrage when added with AutoPilot (#11155)
* storage/raft: Ensure peers are informed of their correct suffrage when added with AutoPilot

* Add test ensuring peer sets are equivalent
2021-03-19 11:53:50 -07:00
Scott Miller 535bcf289e
Fix handling of minimum operations, and forward rotate/config requests to Primary (#11116)
* Boost max_operations to the greater of that specified or absoluteMinOperations

* Forward rotation config requests to the primary

* Reject rotation configs outside the min/max range

* Minor wording fix
2021-03-18 15:08:47 -05:00
Nick Cabatoff 411495514c
Add a test for server stabilization (#11128) 2021-03-17 17:23:13 -04:00
Vishal Nayak 9839e76192
Remove unneeded fields from state output (#11073) 2021-03-10 12:08:12 -05:00
Brian Kassouf aa00b53ba1
Make sure we sanitize the rotation config on each clone (#11050)
* Make sure we sanitize the rotation config on each clone

* Add regression test for missing rotation config

* use Equals

* simplify

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
2021-03-08 10:59:21 -06:00
Vishal Nayak 3e55e79a3f
Autopilot: Server Stabilization, State and Dead Server Cleanup (#10856)
* k8s doc: update for 0.9.1 and 0.8.0 releases (#10825)

* k8s doc: update for 0.9.1 and 0.8.0 releases

* Update website/content/docs/platform/k8s/helm/configuration.mdx

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>

* Autopilot initial commit

* Move autopilot related backend implementations to its own file

* Abstract promoter creation

* Add nil check for health

* Add server state oss no-ops

* Config ext stub for oss

* Make way for non-voters

* s/health/state

* s/ReadReplica/NonVoter

* Add synopsis and description

* Remove struct tags from AutopilotConfig

* Use var for config storage path

* Handle nin-config when reading

* Enable testing autopilot by using inmem cluster

* First passing test

* Only report the server as known if it is present in raft config

* Autopilot defaults to on for all existing and new clusters

* Add locking to some functions

* Persist initial config

* Clarify the command usage doc

* Add health metric for each node

* Fix audit logging issue

* Don't set DisablePerformanceStandby to true in test

* Use node id label for health metric

* Log updates to autopilot config

* Less aggressively consume config loading failures

* Return a mutable config

* Return early from known servers if raft config is unable to be pulled

* Update metrics name

* Reduce log level for potentially noisy log

* Add knob to disable autopilot

* Don't persist if default config is in use

* Autopilot: Dead server cleanup (#10857)

* Dead server cleanup

* Initialize channel in any case

* Fix a bunch of tests

* Fix panic

* Add follower locking in heartbeat tracker

* Add LastContactFailureThreshold to config

* Add log when marking node as dead

* Update follower state locking in heartbeat tracker

* Avoid follower states being nil

* Pull test to its own file

* Add execution status to state response

* Optionally enable autopilot in some tests

* Updates

* Added API function to fetch autopilot configuration

* Add test for default autopilot configuration

* Configuration tests

* Add State API test

* Update test

* Added TestClusterOptions.PhysicalFactoryConfig

* Update locking

* Adjust locking in heartbeat tracker

* s/last_contact_failure_threshold/left_server_last_contact_threshold

* Add disabling autopilot as a core config option

* Disable autopilot in some tests

* s/left_server_last_contact_threshold/dead_server_last_contact_threshold

* Set the lastheartbeat of followers to now when setting up active node

* Don't use config defaults from CLI command

* Remove config file support

* Remove HCL test as well

* Persist only supplied config; merge supplied config with default to operate

* Use pointer to structs for storing follower information

* Test update

* Retrieve non voter status from configbucket and set it up when a node comes up

* Manage desired suffrage

* Consider bucket being created already

* Move desired suffrage to its own entry

* s/DesiredSuffrageKey/LocalNodeConfigKey

* s/witnessSuffrage/recordSuffrage

* Fix test compilation

* Handle local node config post a snapshot install

* Commit to storage first; then record suffrage in fsm

* No need of local node config being nili case, post snapshot restore

* Reconcile autopilot config when a new leader takes over duty

* Grab fsm lock when recording suffrage

* s/Suffrage/DesiredSuffrage in FollowerState

* Instantiate autopilot only in leader

* Default to old ways in more scenarios

* Make API gracefully handle 404

* Address some feedback

* Make IsDead an atomic.Value

* Simplify follower hearbeat tracking

* Use uber.atomic

* Don't have multiple causes for having autopilot disabled

* Don't remove node from follower states if we fail to remove the dead server

* Autopilot server removals map (#11019)

* Don't remove node from follower states if we fail to remove the dead server

* Use map to track dead server removals

* Use lock and map

* Use delegate lock

* Adjust when to remove entry from map

* Only hold the lock while accessing map

* Fix race

* Don't set default min_quorum

* Fix test

* Ensure follower states is not nil before starting autopilot

* Fix race

Co-authored-by: Jason O'Donnell <2160810+jasonodonnell@users.noreply.github.com>
Co-authored-by: Theron Voran <tvoran@users.noreply.github.com>
2021-03-03 13:59:50 -05:00
swayne275 d74f82346b
Add Partial Month Client Count API for Activity Log (#11022)
* sketch out partial month activity log client API

* unit test partialMonthClientCount

* cleanup api

* add api doc, fix test, update api nomenclature to match existing

* cleanup

* add PR changelog file

* integration test for API

* report entities and tokens separately
2021-03-01 16:15:59 -07:00
Scott Miller 08d8f65e01
Take the state lock in checkBarrierRotate, and don't save on seal (#11028)
* Use the state lock, and don't bother a last minute check on seal

* defer
2021-03-01 16:32:17 -06:00
Brian Kassouf cbb8b21520 Fix test build 2021-03-01 12:29:12 -08:00
Brian Kassouf 1bc410783d OSS/ENT Drift 2021-03-01 10:51:04 -08:00
Brian Kassouf a112161f60
expiration: Add a few metrics to measure revoke queue lengths (#10955)
* expiration: Add a few metrics to measure revoke queue lengths

* Update the metric names

* Add appropriate cluster labels

* Add metrics to docs

* Update jobmanager.go
2021-02-26 16:00:39 -08:00
Scott Miller a7b372b447
Two minor changes not reflected OSS side (#11020) 2021-02-26 14:23:56 -06:00
Scott Miller b13b27f37e
OSS side barrier encryption tracking and automatic rotation (#11007)
* Automatic barrier key rotation, OSS portion

* Fix build issues

* Vendored version

* Add missing encs field, not sure where this got lost.
2021-02-25 14:27:25 -06:00
Nick Cabatoff c1ddfbb538
OSS parts of the new client controlled consistency feature (#10974) 2021-02-24 06:58:10 -05:00
swayne275 38a647c6e5
remove noisy log, simplify job interface (#10975) 2021-02-22 15:00:24 -07:00
Brian Kassouf 34a7fc0286
replication: Don't write request coutners on DR Secondary nodes (#10936) 2021-02-22 09:04:41 -08:00
Brian Kassouf 0ad63e5a20
core/expiration: Add backoff jitter to the expiration retries (#10937) 2021-02-18 20:20:01 -08:00
Hridoy Roy 4a96126d5a
Revert "Vault Dependency Upgrades [VAULT-871] (#10903)" (#10939)
This reverts commit eb74ca61fc4dcb7038f39defb127d5d639ba0ca1.
2021-02-18 15:40:18 -05:00
Hridoy Roy a26d1300e8
Vault Dependency Upgrades [VAULT-871] (#10903)
* upgrade vault dependency set

* etcd and grpc issues:

* better for tests

* testing

* all upgrades for hashicorp deps

* kubernetes plugin upgrade seems to work

* kubernetes plugin upgrade seems to work

* etcd and a bunch of other stuff

* all vulnerable packages upgraded

* k8s is broken in linux env but not locally

* test fixes

* fix testing

* fix etcd and grpc

* fix etcd and grpc

* use master branch of go-testing-interface

* roll back etcd upgrade

* have to fix grpc since other vendors pull in grpc 1.35.0 but we cant due to etcd

* rolling back in the replace directives

* a few more testing dependencies to clean up

* fix go mod vendor
2021-02-18 12:31:57 -08:00
swayne275 e4119a6a8a
Vault-1403 Switch Expiration Manager to use Fairsharing Backpressure (#1709) (#10932)
* basic pool and start testing

* refactor a bit for testing

* workFunc, start/stop safety, testing

* cleanup function for worker quit, more tests

* redo public/private members

* improve tests, export types, switch uuid package

* fix loop capture bug, cleanup

* cleanup tests

* update worker pool file name, other improvements

* add job manager prototype

* remove remnants

* add functions to wait for job manager and worker pool to stop, other fixes

* test job manager functionality, fix bugs

* encapsulate how jobs are distributed to workers

* make worker job channel read only

* add job interface, more testing, fixes

* set name for dispatcher

* fix test races

* wire up expiration manager most of the way

* dispatcher and job manager constructors don't return errors

* logger now dependency injected

* make some members private, test fcn to get worker pool size

* make GetNumWorkers public

* Update helper/fairshare/jobmanager_test.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* update fairsharing usage, add tests

* make workerpool private

* remove custom worker names

* concurrency improvements

* remove worker pool cleanup function

* remove cleanup func from job manager, remove non blocking stop from fairshare

* update job manager for new constructor

* stop job manager when expiration manager stopped

* unset env var after test

* stop fairshare when started in tests

* stop leaking job manager goroutine

* prototype channel for waking up to assign work

* fix typo/bug and add tests

* improve job manager wake up, fix test typo

* put channel drain back

* better start/pause test for job manager

* comment cleanup

* degrade possible noisy log

* remove closure, clean up context

* improve revocation context timer

* test: reduce number of revocation workers during many tests

* Update vault/expiration.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* feedback tweaks

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-02-17 14:30:27 -08:00
swayne275 6e1b183f79
Shutdown Test Cores when Tests Complete (#10912)
* Shutdown Test Cores when Tests Complete

* go mod vendor
2021-02-12 13:04:48 -07:00
Jim Kalafut 42bae71806
Improve error messages (#10843)
- Fix: "bytes" should be less than %!s(int=131072) message
- Also add a missing openapi type that was throwing warnings
2021-02-11 19:51:12 -08:00
Michael Golowka 108d4c6a68
MySQL - Add username customization (#10834) 2021-02-11 14:08:32 -07:00
Vishal Nayak 53cb1deb38
Revert "Read-replica instead of non-voter (#10875)" (#10890)
This reverts commit fc745670cf34821f5834357d9caebc3351dbc1e7.
2021-02-10 16:41:58 -05:00