* add core state lockd eadlock detection config option v2
* add changelog
* split out NewTestCluster function to maintain build flag
* replace long func with constant
* remove line
* rename file, and move where detect deadlock flag is set
Create global quotas of each type in every NewTestCluster. Also switch some key locks to use DeadlockMutex to make it easier to discover deadlocks in testing.
NewTestCluster also now starts the cluster, and the Start method becomes a no-op. Unless SkipInit is provided, we also wait for a node to become active, eliminating the need for WaitForActiveNode. This was needed because otherwise we can't safely make the quota api call. We can't do it in Start because Start doesn't return an error, and I didn't want to begin storing the testing object T instead TestCluster just so we could call t.Fatal inside Start.
The last change here was to address the problem of how to skip setting up quotas when creating a cluster with a nonstandard handler that might not even implement the quotas endpoint. The challenge is that because we were taking a func pointer to generate the real handler func, we didn't have any way to compare that func pointer to the standard handler-generating func http.Handler without creating a circular dependency between packages vault and http. The solution was to pass a method instead of an anonymous func pointer so that we can do reflection on it.
* VAULT-7065 move lease-quota updating to process restore
* VAULT-7065 Avoid off-by-one issue that can happen during Restore
* VAULT-7065 Add comment to explain why we don't count creates in restore mode
* VAULT-6613 add DetermineRoleFromLoginRequest function to Core
* Fix body handling
* Role resolution for rate limit quotas
* VAULT-6613 update precedence test
* Add changelog
* VAULT-6614 start of changes for roles in LCQs
* Expiration changes for leases
* Add role information to RequestAuth
* VAULT-6614 Test updates
* VAULT-6614 Add expiration test with roles
* VAULT-6614 fix comment
* VAULT-6614 Protobuf on OSS
* VAULT-6614 Add rlock to determine role code
* VAULT-6614 Try lock instead of rlock
* VAULT-6614 back to rlock while I think about this more
* VAULT-6614 Additional safety for nil dereference
* VAULT-6614 Use %q over %s
* VAULT-6614 Add overloading to plugin backends
* VAULT-6614 RLocks instead
* VAULT-6614 Fix return for backend factory
* add func to set level for specific logger
* add endpoints to modify log level
* initialize base logger with IndependentLevels
* test to ensure other loggers remain unchanged
* add DELETE loggers endpoints to revert back to config
* add API docs page
* add changelog entry
* remove extraneous line
* add log level field to Core struct
* add godoc for getLogLevel
* add some loggers to c.allLoggers
* Various changes to try to ensure that fewer goroutines survive after a test completes:
* add Core.ShutdownWait that doesn't return until shutdown is done
* create the usedCodes cache on seal and nil it out on pre-seal so that the finalizer kills the janitor goroutine
* stop seal health checks on seal rather than wait for them to discover the active context is done
* make sure all lease-loading goroutines are done before returning from restore
* make uniquePoliciesGc discover closed quitCh immediately instead of only when the ticker fires
* make sure all loading goroutines are done before returning from loadEntities, loadCachedEntitiesOfLocalAliases
* port SSCT OSS
* port header hmac key to ent and generate token proto without make command
* remove extra nil check in request handling
* add changelog
* add comment to router.go
* change test var to use length constants
* remove local index is 0 check and extra defer which can be removed after use of ExternalID
* prototype function to retry irrevocable lease revocation, clean up comment
* setup irrevocable lease revoke retry on expiration manager creation
* reduce expiration lag to 1 hour, use core context and timeout on retry attempts
* add lease namespace to revocation call
* start irrevocable revocation attempts in setupExpiration
* grab reference to channel to avoid race test failures
* Acquire a per-lock lease to make renew and revoke atomic wrt each other.
This means we don't have to hold pendingLock during I/O.
* Attempted fix for deadlock in token revocation.
* Comment fix.
* Fix error checking in loadEntry.
* Add benchmark
* Add a few additional locking locations
* Improve benchmark slightly
* Update vault/expiration.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Update vault/expiration.go
Co-authored-by: swayne275 <swayne275@gmail.com>
* Add a lease lock into tidy
Co-authored-by: Scott Miller <smiller@hashicorp.com>
Co-authored-by: Brian Kassouf <bkassouf@hashicorp.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Co-authored-by: swayne275 <swayne275@gmail.com>
* build out lease count (not fully working), start lease list
* build out irrevocable lease list
* bookkeeping
* test irrevocable lease counts for API/CLI
* fix listIrrevocableLeases, test listIrrevocableLeases, cleanup
* test expiration API limit
* namespace tweaks, test force flag on lease list
* integration test leases/count API, plenty of fixes and improvements
* test lease list API, fixes and improvements
* test force flag for irrevocable lease list API
* i guess this wasn't saved on the last refactor...
* fixes and improvements found during my review
* better test error msg
* Update vault/logical_system_paths.go
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* Update vault/logical_system_paths.go
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* return warning with data if more than default leases to list without force flag
* make api doc more generalized
* list leases in general, not by mount point
* change force flag to include_large_results
* sort leases by LeaseID for consistent API response
* switch from bool flag for API limit to string value
* sort first by leaseID, then stable sort by expiration
* move some utils to be in oss and ent
* improve sort efficiency for API response
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* shrink generic error message
* move zombie loading to updatePendingInternal from loadEntryInternal
* probably the right metric/lease behavior for irrevocable leases...
* comment improvements
* test total lease count with valid and irrevocable leases
* build out zombie lease system
* add typo for CI
* undo test CI commit
* time equality test isn't working on CI, so let's see what this does...
* add unrecoverable proto error, make proto, go mod vendor
* zombify leases if unrecoverable error, tests
* test fix: somehow pointer in pointer rx is null after pointer rx called
* tweaks based on roy feedback
* improve zombie errors
* update which errors are unrecoverable
* combine zombie logic
* keep subset of zombie lease in memory
* build out zombie lease system
* add typo for CI
* undo test CI commit
* time equality test isn't working on CI, so let's see what this does...
* protect against nil pointer receiver calls
* basic pool and start testing
* refactor a bit for testing
* workFunc, start/stop safety, testing
* cleanup function for worker quit, more tests
* redo public/private members
* improve tests, export types, switch uuid package
* fix loop capture bug, cleanup
* cleanup tests
* update worker pool file name, other improvements
* add job manager prototype
* remove remnants
* add functions to wait for job manager and worker pool to stop, other fixes
* test job manager functionality, fix bugs
* encapsulate how jobs are distributed to workers
* make worker job channel read only
* add job interface, more testing, fixes
* set name for dispatcher
* fix test races
* wire up expiration manager most of the way
* dispatcher and job manager constructors don't return errors
* logger now dependency injected
* make some members private, test fcn to get worker pool size
* make GetNumWorkers public
* Update helper/fairshare/jobmanager_test.go
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* update fairsharing usage, add tests
* make workerpool private
* remove custom worker names
* concurrency improvements
* remove worker pool cleanup function
* remove cleanup func from job manager, remove non blocking stop from fairshare
* update job manager for new constructor
* stop job manager when expiration manager stopped
* unset env var after test
* stop fairshare when started in tests
* stop leaking job manager goroutine
* prototype channel for waking up to assign work
* fix typo/bug and add tests
* improve job manager wake up, fix test typo
* put channel drain back
* better start/pause test for job manager
* comment cleanup
* degrade possible noisy log
* remove closure, clean up context
* improve revocation context timer
* test: reduce number of revocation workers during many tests
* Update vault/expiration.go
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* feedback tweaks
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
Co-authored-by: Brian Kassouf <briankassouf@users.noreply.github.com>
* Add a flag to enable a permit pool to gate lease expiration
* Use the env var to get the size
* Add logs and metris to help debug this
Co-authored-by: Hridoy Roy <roy@hashicorp.com>
* Changes to expiration manager to walk tokens (including non-expiring ones.)
* Count by namespace in token manager.
* Keep a dictionary of policy lists and deduplicate based on it.
* core: revoke the proper token on partial failures from token-related requests
* move test to vault package, move test trigger to expiration manager
* update logging messages for clarity
* docstring fix
Hamid Ghaf