luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
open-nomad/nomad
History
Michael Schurter 80f521cce5 vault: expired tokens count toward batch limit 
As of 0.11.3 Vault token revocation and purging was done in batches.
However the batch size was only limited by the number of *non-expired*
tokens being revoked.

Due to bugs prior to 0.11.3, *expired* tokens were not properly purged.
Long-lived clusters could have thousands to *millions* of very old
expired tokens that never got purged from the state store.

Since these expired tokens did not count against the batch limit, very
large batches could be created and overwhelm servers.

This commit ensures expired tokens count toward the batch limit with
this one line change:

```
- if len(revoking) >= toRevoke {
+ if len(revoking)+len(ttlExpired) >= toRevoke {
```

However, this code was difficult to test due to being in a periodically
executing loop. Most of the changes are to make this one line change
testable and test it.
2020-07-28 15:42:47 -07:00
..
deploymentwatcher refactor: make it clear where we're accessing dstate 2020-07-20 11:25:53 -04:00
drainer avoid logging in draining job watcher 2020-03-30 07:06:53 -04:00
mock MRD: all regions should start pending (#8433) 2020-07-14 10:57:37 -04:00
state refactor: make it clear where we're accessing dstate 2020-07-20 11:25:53 -04:00
structs Merge pull request #8453 from hashicorp/oss-multi-vault-ns 2020-07-27 08:45:22 -04:00
types
volumewatcher csi: add -force flag to volume deregister (#8295) 2020-07-01 12:17:51 -04:00
acl.go Audit config, seams for enterprise audit features 2020-03-23 13:47:42 -04:00
acl_endpoint.go address feedback review 2019-11-26 08:39:04 -05:00
acl_endpoint_test.go Simplify Bootstrap logic in tests 2020-03-02 13:47:43 -05:00
acl_test.go Audit config, seams for enterprise audit features 2020-03-23 13:47:42 -04:00
alloc_endpoint.go rpc: allow querying allocs across namespaces 2020-06-17 16:31:06 -04:00
alloc_endpoint_test.go testS: add all namespaces test for allocations 2020-06-22 10:26:08 -04:00
autopilot.go implement MinQuorum 2020-02-16 16:04:59 -06:00
autopilot_test.go tests: deflake TestAutopilot_RollingUpdate 2020-04-03 17:15:41 -04:00
blocked_evals.go blocked_evals reset system evals on Flush 2019-07-18 10:32:13 -04:00
blocked_evals_system.go blocked_evals system evals indexed by job and node 2019-07-18 10:32:12 -04:00
blocked_evals_test.go blocked_evals_test disable calls Flush 2019-07-18 10:32:13 -04:00
client_agent_endpoint.go api: nomad debug new /agent/host (#8325) 2020-07-02 09:51:25 -04:00
client_agent_endpoint_test.go minor tweaks from Ent 2020-07-20 09:25:09 -04:00
client_alloc_endpoint.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
client_alloc_endpoint_test.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
client_csi_endpoint.go fix a trace logline 2020-05-26 10:18:09 -04:00
client_csi_endpoint_test.go tests: wait until clients are in the state store 2020-05-26 18:53:24 -04:00
client_fs_endpoint.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
client_fs_endpoint_test.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
client_rpc.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
client_rpc_test.go Simplify Bootstrap logic in tests 2020-03-02 13:47:43 -05:00
client_stats_endpoint.go server 2018-09-15 16:23:13 -07:00
client_stats_endpoint_test.go Simplify Bootstrap logic in tests 2020-03-02 13:47:43 -05:00
config.go Set AgentShutdown 2020-07-17 11:04:57 -04:00
consul.go always set purgeFunc 2020-05-21 21:05:53 -04:00
consul_policy.go nomad: fix leftover missed refactoring in consul policy checking 2020-01-31 19:05:06 -06:00
consul_policy_test.go nomad: fix leftover missed refactoring in consul policy checking 2020-01-31 19:05:06 -06:00
consul_test.go apply the same change to consul revocation 2020-05-21 08:30:31 -04:00
core_sched.go Periodic GC for volume claims (#7881) 2020-05-11 08:20:50 -04:00
core_sched_test.go Periodic GC for volume claims (#7881) 2020-05-11 08:20:50 -04:00
csi_batch.go csi: don't pass volume claim releases thru GC eval (#8021) 2020-05-20 15:22:51 -04:00
csi_batch_test.go csi: don't pass volume claim releases thru GC eval (#8021) 2020-05-20 15:22:51 -04:00
csi_endpoint.go csi: check for empty arguments on CSI endpoint (#8027) 2020-05-20 10:22:24 -04:00
csi_endpoint_test.go nomad: fix dropped test error (#8356) 2020-07-06 08:46:54 -04:00
deployment_endpoint.go multiregion: use pending instead of paused 2020-06-17 11:06:14 -04:00
deployment_endpoint_test.go remove test for ent-only behavior 2020-06-17 11:27:29 -04:00
deployment_watcher_shims.go Fix typos 2018-05-07 14:50:01 -05:00
drainer_int_test.go tests: use GreaterOrEqual and apply change to other tests 2020-05-27 11:22:48 -04:00
drainer_shims.go set node.StatusUpdatedAt in raft 2019-05-21 16:13:32 -04:00
endpoints_oss.go include pro tag in serveral oss.go files 2020-02-10 15:56:14 -05:00
eval_broker.go nomad: refactor waitForIndex into SnapshotAfter 2019-05-17 13:30:23 -07:00
eval_broker_test.go nomad: TestEvalBroker_Dequeue_Empty_Timeout() proper goroutine error handling (#6657) 2019-11-08 14:35:06 -05:00
eval_endpoint.go acl: check ACL against object namespace 2019-10-08 12:59:22 -04:00
eval_endpoint_test.go tests: swap lib/freeport for tweaked helper/freeport 2019-12-09 08:37:32 -06:00
fsm.go comment compat concern in fsm.go 2020-07-15 11:23:49 -04:00
fsm_not_ent.go sync 2017-10-13 14:36:02 -07:00
fsm_registry_oss.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
fsm_test.go Remove unused state.TestInitState 2020-07-20 09:55:55 -04:00
heartbeat.go goimports 2019-01-22 15:44:31 -08:00
heartbeat_test.go Simplify Bootstrap logic in tests 2020-03-02 13:47:43 -05:00
job_endpoint.go Merge pull request #8453 from hashicorp/oss-multi-vault-ns 2020-07-27 08:45:22 -04:00
job_endpoint_hook_connect.go nomad: recanonicalize network after connect hook (#8407) 2020-07-10 10:59:51 -04:00
job_endpoint_hook_connect_test.go nomad: recanonicalize network after connect hook (#8407) 2020-07-10 10:59:51 -04:00
job_endpoint_hook_expose_check.go nomad: build dynamic port for exposed checks if not specified (#7800) 2020-04-28 00:07:41 -04:00
job_endpoint_hook_expose_check_test.go nomad: build dynamic port for exposed checks if not specified (#7800) 2020-04-28 00:07:41 -04:00
job_endpoint_hooks.go core: add semver constraint 2019-11-19 08:40:19 -08:00
job_endpoint_oss.go oss compoments for multi-vault namespaces 2020-07-24 10:14:59 -04:00
job_endpoint_test.go Merge pull request #8453 from hashicorp/oss-multi-vault-ns 2020-07-27 08:45:22 -04:00
leader.go Atomic eval insertion with job (de-)registration 2020-07-14 11:59:29 -04:00
leader_oss.go include pro tag in serveral oss.go files 2020-02-10 15:56:14 -05:00
leader_test.go test for paused workers upon leadership revocation 2020-06-01 10:48:42 -04:00
merge.go
node_endpoint.go consul/connect: add support for running connect native tasks 2020-06-22 14:07:44 -05:00
node_endpoint_test.go tests: use t.Fatalf when it's clearer 2020-05-27 10:09:56 -04:00
operator_endpoint.go minor tweaks from Ent 2020-07-20 09:25:09 -04:00
operator_endpoint_test.go loosen raft timeout 2020-06-07 16:38:11 -04:00
periodic.go Atomic eval insertion with job (de-)registration 2020-07-14 11:59:29 -04:00
periodic_endpoint.go goimports 2019-01-22 15:44:31 -08:00
periodic_endpoint_test.go csi: fix index maintenance for CSIVolume and CSIPlugin tables (#7049) 2020-03-23 13:58:29 -04:00
periodic_test.go stop periodic dispatch at end of tests (#8111) 2020-06-04 09:15:00 -04:00
plan_apply.go scheduler/reconcile: set FollowupEvalID on lost stop_after_client_disconnect (#8105) (#8138) 2020-06-09 17:13:53 -04:00
plan_apply_not_ent.go sync 2017-10-13 14:36:02 -07:00
plan_apply_pool.go Log reason a plan gets rejected per node. 2017-07-13 17:14:02 -07:00
plan_apply_pool_test.go Enable more linters 2017-09-26 15:26:33 -07:00
plan_apply_test.go tests: swap lib/freeport for tweaked helper/freeport 2019-12-09 08:37:32 -06:00
plan_endpoint.go goimports 2019-01-22 15:44:31 -08:00
plan_endpoint_test.go tests: swap lib/freeport for tweaked helper/freeport 2019-12-09 08:37:32 -06:00
plan_normalization_test.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
plan_queue.go nomad: refactor waitForIndex into SnapshotAfter 2019-05-17 13:30:23 -07:00
plan_queue_test.go nomad: fix test goroutine (#6593) 2019-10-31 08:23:32 -04:00
raft_rpc.go Refactor 2018-02-15 13:59:00 -08:00
regions_endpoint.go server 2018-09-15 16:23:13 -07:00
regions_endpoint_test.go tests: swap lib/freeport for tweaked helper/freeport 2019-12-09 08:37:32 -06:00
rpc.go If leadership fails, consider it handled 2020-05-31 22:06:17 -04:00
rpc_test.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
scaling_endpoint.go add acl validation to Scaling.ListPolicies and Scaling.GetPolicy 2020-03-24 14:39:05 +00:00
scaling_endpoint_test.go update RPC scaling endpoint tests to use renamed 'scale' policy disposition 2020-03-24 20:18:12 +00:00
search_endpoint.go csi: CLI for volume status, registration/deregistration and plugin status (#7193) 2020-03-23 13:58:30 -04:00
search_endpoint_oss.go csi: implement volume ACLs (#7339) 2020-03-23 13:59:25 -04:00
search_endpoint_test.go csi: plugin deregistration on plugin job GC (#7502) 2020-03-26 17:07:18 -04:00
serf.go Simplify Bootstrap logic in tests 2020-03-02 13:47:43 -05:00
serf_test.go Simplify Bootstrap logic in tests 2020-03-02 13:47:43 -05:00
server.go oss compoments for multi-vault namespaces 2020-07-24 10:14:59 -04:00
server_setup_oss.go oss compoments for multi-vault namespaces 2020-07-24 10:14:59 -04:00
server_test.go Simplify Bootstrap logic in tests 2020-03-02 13:47:43 -05:00
stats_fetcher.go server 2018-09-15 16:23:13 -07:00
stats_fetcher_test.go Simplify Bootstrap logic in tests 2020-03-02 13:47:43 -05:00
status_endpoint.go server 2018-09-15 16:23:13 -07:00
status_endpoint_test.go tests: swap lib/freeport for tweaked helper/freeport 2019-12-09 08:37:32 -06:00
system_endpoint.go server 2018-09-15 16:23:13 -07:00
system_endpoint_test.go tests: swap lib/freeport for tweaked helper/freeport 2019-12-09 08:37:32 -06:00
testing.go tests: prefix agent logs to identify agent sources 2020-06-07 16:38:11 -04:00
timetable.go vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:45:21 -04:00
timetable_test.go fixup! vendor: explicit use of hashicorp/go-msgpack 2020-03-31 09:48:07 -04:00
util.go remove unused dropButLastChannel 2020-02-13 18:56:53 -05:00
util_test.go remove unused dropButLastChannel 2020-02-13 18:56:53 -05:00
vault.go vault: expired tokens count toward batch limit 2020-07-28 15:42:47 -07:00
vault_test.go vault: expired tokens count toward batch limit 2020-07-28 15:42:47 -07:00
vault_testing.go on leadership establishment, revoke Vault tokens in background 2020-05-21 07:38:27 -04:00
volumewatcher_shim.go csi: move volume claim release into volumewatcher (#7794) 2020-04-30 09:13:00 -04:00
worker.go add create and modify timestamps to evaluations (#5881) 2019-08-07 09:50:35 -07:00
worker_test.go scheduler/reconcile: set FollowupEvalID on lost stop_after_client_disconnect (#8105) (#8138) 2020-06-09 17:13:53 -04:00