Commit graph

2213 commits

Author SHA1 Message Date
swayne275 35aad1df4a
Vault 2256: fix lease count quotas causing panics on dr secondaries (#11742)
* lift relevant changes from ent to oss

* fix silent error bug in quotas
2021-06-02 10:12:05 -06:00
swayne275 9724f59180
Vault 1979: Query API for Irrevocable Leases (#11607)
* build out lease count (not fully working), start lease list

* build out irrevocable lease list

* bookkeeping

* test irrevocable lease counts for API/CLI

* fix listIrrevocableLeases, test listIrrevocableLeases, cleanup

* test expiration API limit

* namespace tweaks, test force flag on lease list

* integration test leases/count API, plenty of fixes and improvements

* test lease list API, fixes and improvements

* test force flag for irrevocable lease list API

* i guess this wasn't saved on the last refactor...

* fixes and improvements found during my review

* better test error msg

* Update vault/logical_system_paths.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* Update vault/logical_system_paths.go

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>

* return warning with data if more than default leases to list without force flag

* make api doc more generalized

* list leases in general, not by mount point

* change force flag to include_large_results

* sort leases by LeaseID for consistent API response

* switch from bool flag for API limit to string value

* sort first by leaseID, then stable sort by expiration

* move some utils to be in oss and ent

* improve sort efficiency for API response

Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
2021-06-02 10:11:30 -06:00
Scott Miller 59bea5f6fd
Add ulimit check, and tidy unit test cases (#11678)
* Add ulimit check, and tidy unit test cases to avoid needing to have all results and perfect ordering

* Make order independent check recursive

* Fix unit tests

* Try a 5s request timeout
2021-06-01 12:43:51 -05:00
Cory Bolar 6c726bde67
Only fail auth plugin backend mounting for custom plugins (#11690)
Checking if a plugin is a builtin by comparing it's type to those in the builtin
registry allows for a custom plugin loaded with the same name to be considered a
builtin during error handling of the mounting process.  This can cause the vault
installation to brick itself because it cannot be unsealed to register a new
sha256 or file path for a previously loaded custom plugin.  Improve this logic
by checking the plugin catalog rather than the builtin registry.

Fixes #11687
2021-05-27 13:37:45 -05:00
Scott Miller 46da8ebae6
Diagnose Disk usage checks (#11673)
* Disk usage checks

* Move disk free earlier

* Move logic to helpers

* Bring over test logic from the ulimit PR

* imports

* Report error

* Get unit tests working
2021-05-27 10:17:52 -07:00
Hridoy Roy b6e1d62d33
Modularize Run Command (#11573)
* initial refactoring of unseal step in run

* remove waitgroup

* remove waitgroup

* backup work

* backup

* backup

* completely modularize run and move into diagnose

* add diagnose errors for incorrect number of unseal keys

* comment tests back in

* backup

* first subspan

* finished subspanning but running into error with timeouts

* remove runtime checks

* meeting updates

* remove telemetry block

* roy comment

* subspans for seal finalization and wrapping diagnose latency checks

* fix storage latency test errors

* review comments

* use random uuid for latency checks instead of static id
2021-05-25 15:23:20 -07:00
Scott Miller e305f78e9c
Add a Success helper to set successful spans' messages (#11621) 2021-05-21 19:21:23 -07:00
Scott Miller 941d01eee3
Initial Diagnose CLI output (#11583)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* wip

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* wip

* wip

* wip

* Revendor otel

* Fix merge related problems

* imports

* Fix unit tests

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-21 19:21:11 -07:00
Nick Cabatoff 2adef1f878
OSS parts of #1891 (sys/health license addition) (#11676) 2021-05-20 13:32:15 -04:00
Nick Cabatoff 01f96f18ce
VAULT-2439: OSS parts of #1889 (raft licensing init) (#11665) 2021-05-19 16:07:58 -04:00
Brian Kassouf eabfda0c79
Make sure we are checking the final peerset (#11664) 2021-05-19 12:01:52 -07:00
Brian Kassouf f498d0d389
Reload raft TLS keys on active startup (#11660) 2021-05-19 10:03:32 -07:00
Scott Miller 6b8d7fe2e6
Patch expiration fix over from ENT (#11650)
* Patch expiration fix over from ENT

* Rename changelog
2021-05-18 14:55:38 -07:00
Nick Cabatoff e212ec5d8e
OSS parts of ent PR #1857: license autoloading init changes. (#11623) 2021-05-17 14:10:26 -04:00
Scott Miller 7b05fcf9bf
Don't backoff if a listener error was a timeout (#11594) 2021-05-13 17:02:25 -05:00
Jason O'Donnell 502cf3b212
core: updates to password policy generator (#11596)
* core: fix bug in password policies not using namespaces

* Add changelog
2021-05-13 09:55:46 -04:00
Josh Black 641a81b74b
Provide a new API endpoint for retrieving signed licenses (#11543) 2021-05-12 12:19:25 -07:00
Scott Miller 9dbf1a7dba
Add infrastructure and helpers for skipping diagnose checks (#11593)
* Add infrastructure for skipping tests

* Add infrastructure for skipping tests

* Set it

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Implement type alias for test functions

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-12 12:54:40 -05:00
Scott Miller f0c3192f20
Add an exponential backoff to TCP listeners to avoid fast loops in error scenarios (#11588)
* Add an exponential backoff to TCP listeners to avoid fast loops in error scenarios

* reset loop delay

* changelog
2021-05-12 10:47:38 -05:00
swayne275 0229787a26
Vault 2303: Count irrevocable leases in quotas/metrics and other improvements (#11542)
* shrink generic error message

* move zombie loading to updatePendingInternal from loadEntryInternal

* probably the right metric/lease behavior for irrevocable leases...

* comment improvements

* test total lease count with valid and irrevocable leases
2021-05-11 14:04:06 -06:00
Lars Lehtonen 53dd619d2f
vault: deprecate errwrap.Wrapf() (#11577) 2021-05-11 13:12:54 -04:00
swayne275 f467d86805
don't panic if no data, improve log (#11581) 2021-05-11 10:41:57 -06:00
Scott Miller 40e0806f29
Add timeout helper (#11554)
* Add timeout helper

* cleanup-imports

* didnt need mainsection

Co-authored-by: HridoyRoy <roy@hashicorp.com>
2021-05-10 17:55:43 -07:00
swayne275 ee53fc5afb
Fix activity log fragment race test (#11565)
* grab reference to done channel while locked

* add clarifying comment

* lock and grab channel reference again for test races
2021-05-07 17:02:11 -06:00
Nick Cabatoff 53c7d1de7d
config for autoloading license (oss parts) 2021-05-07 08:55:41 -04:00
Nick Cabatoff 0926e302c5
Fix some panics resulting from calling ExpirationManager.Stop directly. (#11547) 2021-05-06 12:39:01 -04:00
Nick Cabatoff 663ad150a7
Make TestActivityLog_MultipleFragmentsAndSegments timeout on its own (#11490)
* The main driver for this change was to make the read from a.newFragmentCh timeout quickly rather than waiting for the test timeout (much longer).  While testing the change I observed a panic during shutdown, but it was swallowed and moreover there was no stack trace so it wasn't obvious.  I'm hoping we can get rid of the recover, so I fixed the issue in the activitylog tests that needed it.
2021-05-06 10:19:53 -04:00
Scott Miller 4fc6e8b366
Fix barrier key autoration config edge cases (#11541)
* Add an Int64 type

* Use the new Int64 type so that even 32 bit builds can specify max_operations above 2^31

* Missed a spot

* go mod vendor

* fix cast

* changelog

* Update unit test to ensure this works on both 32 and 64-bit archs
2021-05-05 14:39:04 -05:00
swayne275 56c57ffddd
change zombie terminology to irrevocable (#11525) 2021-05-04 15:48:02 -06:00
swayne275 335e4c3711
Introduce Logical Unrecoverable Error, Use it in Expiration Manager (#11477)
* build out zombie lease system

* add typo for CI

* undo test CI commit

* time equality test isn't working on CI, so let's see what this does...

* add unrecoverable proto error, make proto, go mod vendor

* zombify leases if unrecoverable error, tests

* test fix: somehow pointer in pointer rx is null after pointer rx called

* tweaks based on roy feedback

* improve zombie errors

* update which errors are unrecoverable

* combine zombie logic

* keep subset of zombie lease in memory
2021-05-03 17:56:06 -06:00
Vishal Nayak 1e61f799ca
Use correct mount accessor when refreshing external group memberships (#11506)
* Use correct mount accessor when refreshing external group memberships

* Add CL

* Handle the renew case properly
2021-05-03 08:23:59 -04:00
Hridoy Roy 28aa9b9175
Diagnoses Consul Direct Access Check (#11505)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* run direct access checks in diagnose command

* review comments

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 16:21:06 -07:00
Hridoy Roy 5b63066c2c
Diagnose V0: Storage End to End Checks (#11468)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* consul tls checks

* draft for storage end to end check

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

* cleaned up chan logic. Tests next.

* fix tests

* remove a comment

* tests

* remove a comment

* cosmetic changes

Co-authored-by: Scott G. Miller <smiller@hashicorp.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
2021-05-02 13:33:13 -07:00
Scott Miller fcb9bab51b
Fix rotate/config unit test for 32 bit yet again (#11491) 2021-04-30 10:31:11 -05:00
Scott Miller 85fbd45e1c
Create helpers which integrate with OpenTelemetry for diagnose collection (#11454)
* Create helpers which integrate with OpenTelemetry for diagnose collection

* Go mod vendor

* Comments

* Update vault/diagnose/helpers.go

Co-authored-by: swayne275 <swayne275@gmail.com>

* Add unit test/example

* tweak output

* More comments

* add spot check concept

* Get unit tests working on Result structs

* Fix unit test

* Get unit tests working, and make diagnose sessions local rather than global

* Comments

* Last comments

* No need for init

* :|

* Fix helpers_test

Co-authored-by: swayne275 <swayne275@gmail.com>
2021-04-29 13:32:41 -05:00
swayne275 8db70c9e58
Expiration Manager: Handle Presumed Irrevocable Leases Separately (#11452)
* build out zombie lease system

* add typo for CI

* undo test CI commit

* time equality test isn't working on CI, so let's see what this does...

* protect against nil pointer receiver calls
2021-04-29 09:12:02 -06:00
Josh Black ec105f288f
Switch to shared raft-boltdb library and add metrics (#11269) 2021-04-26 16:01:26 -07:00
Nick Cabatoff e3a88c59e9
Add metrics for perf standby and replication node type. (#11472) 2021-04-26 13:54:19 -04:00
Nick Cabatoff 5a4151873c
Cast constant to int64 so it doesn't overflow. (#11466) 2021-04-26 10:04:39 -04:00
Hridoy Roy 22cab6185d
[VAULT-1441] Fix race that allowed remounting on path used by another mount (#11453)
* remount concurrent test fix

* changelog

* Update changelog/11453.txt

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>

Co-authored-by: Nick Cabatoff <ncabatoff@hashicorp.com>
2021-04-24 09:05:41 -07:00
Angel Garbarino 2e35e9578c
UI/obscure secret on input (#11284)
* new font and add as font-family to be used in masked-input

* clean up logic

* refactor for displayOnly

* start cert masking

* work on certificates

* upload cert work

* fix global styling

* fix styling for class no longer used

* make mask by default and remove option

* glimmerize start and certificate on LDAP a file field

* glimmerize actions

* first part of glimmerizing text-file still need to do some clean up

* not doing awesome over here

* getting ready to un-glimmer

* unglimmerize

* remove placeholder based on conversations with design

* clean up text-file

* cleanup

* fix class bindings

* handle class binding

* set up for test

* fix elementId

* track down index

* update masked-input test

* add more to the masked-input test

* test-file test

* fix broken test

* clear old style

* clean up

* remove pgp key masked font, this really needs to be refactored to text-file component

* changelog

* cover other certificate view

* add allowCopy

* address some pr styling comments

* improve test coverage

* fix some issues

* add attr.options.masked
2021-04-22 08:58:37 -06:00
Josh Black 06809930a3
Add HTTP response headers for hostname and raft node ID (if applicable) (#11289) 2021-04-20 15:25:04 -07:00
Nick Cabatoff b7ed30afed
Poll until token disappears from storage. (#11424) 2021-04-20 14:55:10 -04:00
Nick Cabatoff 75ec35421d
Refactor TestTokenStore_CubbyholeTidy so that it can be tested on ent with a non-root namespace. (#11422) 2021-04-20 14:49:25 -04:00
Brian Kassouf c400dfdd3d
core/token: fix panic looking up invalid batch tokens (#11415) 2021-04-19 18:26:51 -07:00
Nick Cabatoff 474c4e8134
Make cubbyhole revocation/tidying compatible with cubbys in namespaces. (#11408) 2021-04-19 17:28:04 -04:00
Nick Cabatoff a8023e0fdb
Add support for unauthenticated pprof access on a per-listener basis,… (#11324)
* Add support for unauthenticated pprof access on a per-listener basis, as we do for metrics.

* Add missing pprof sub-targets like 'allocs' and 'block'.  Capture the goroutine subtarget a second time in text form.  This is mostly a convenience, but also I think the pprof format might be a bit lossy?
2021-04-19 14:30:59 -04:00
Nick Cabatoff 541ae8636c
On lease deletion, also delete non-orphan batch token parent index (#11377) 2021-04-16 17:03:22 -04:00
Nick Cabatoff b07a10331f
Add metrics for requests forwarded by standbys. (#11366) 2021-04-16 14:02:20 -04:00
Nick Cabatoff 242d258e94
Fix goroutine leak caused by updating rate quotas (#11371)
Make sure that when we modify a rate quota, we stop the existing goroutine before starting the new one.
2021-04-16 14:00:01 -04:00