open-nomad/client
Tim Gross 0a19fe3b60 fix multiple overflow errors in exponential backoff (#18200)
We use capped exponential backoff in several places in the code when handling
failures. The code we've copy-and-pasted all over has a check to see if the
backoff is greater than the limit, but this check happens after the bitshift and
we always increment the number of attempts. This causes an overflow with a
fairly small number of failures (ex. at one place I tested it occurs after only
24 iterations), resulting in a negative backoff which then never recovers. The
backoff becomes a tight loop consuming resources and/or DoS'ing a Nomad RPC
handler or an external API such as Vault. Note this doesn't occur in places
where we cap the number of iterations so the loop breaks (usually to return an
error), so long as the number of iterations is reasonable.

Introduce a helper with a check on the cap before the bitshift to avoid overflow in all 
places this can occur.

Fixes: #18199
Co-authored-by: stswidwinski <stan.swidwinski@gmail.com>
2023-08-15 14:39:09 -04:00
..
allocdir Add `disable_file` parameter to job's `vault` stanza (#13343) 2023-06-23 15:15:04 -04:00
allochealth [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
allocrunner fix multiple overflow errors in exponential backoff (#18200) 2023-08-15 14:39:09 -04:00
allocwatcher backport of commit a3a637ee8efe5e1251f60f781369bd9052c4d4a2 (#18132) 2023-08-02 08:47:19 -05:00
config compliance: add headers with fixed copywrite tool (#17353) 2023-05-30 09:20:32 -05:00
consul [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
devicemanager fix multiple overflow errors in exponential backoff (#18200) 2023-08-15 14:39:09 -04:00
dynamicplugins Merge pull request #16836 from hashicorp/compliance/add-headers 2023-04-10 16:32:03 -07:00
fingerprint fingerprint: fix 'default' alias not added to interface specified by `network_interface` (#18096) (#18116) 2023-08-01 08:38:03 -04:00
interfaces [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
lib backport of commit 615e76ef3c23497f768ebd175f0c624d32aeece8 (#17993) 2023-07-19 13:31:14 -05:00
logmon [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
pluginmanager fix multiple overflow errors in exponential backoff (#18200) 2023-08-15 14:39:09 -04:00
servers [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
serviceregistration chore(lint): use Go stdlib variables for HTTP methods and status codes (#17968) (#18074) 2023-07-26 16:38:39 +01:00
state CSI: persist previous mounts on client to restore during restart (#17840) 2023-07-10 13:20:15 -04:00
stats backport of commit 615e76ef3c23497f768ebd175f0c624d32aeece8 (#17993) 2023-07-19 13:31:14 -05:00
structs client: send node secret with every client-to-server RPC (#16799) 2023-06-22 11:06:49 -04:00
taskenv fix host port handling for ipv6 (#16723) 2023-04-20 19:53:20 -07:00
testutil Revert "hashicorp/go-msgpack v2 (#16810)" (#17047) 2023-05-01 17:18:34 -04:00
vaultclient [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
acl.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
acl_test.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
agent_endpoint.go Revert "hashicorp/go-msgpack v2 (#16810)" (#17047) 2023-05-01 17:18:34 -04:00
agent_endpoint_test.go Revert "hashicorp/go-msgpack v2 (#16810)" (#17047) 2023-05-01 17:18:34 -04:00
alloc_endpoint.go Revert "hashicorp/go-msgpack v2 (#16810)" (#17047) 2023-05-01 17:18:34 -04:00
alloc_endpoint_test.go Revert "hashicorp/go-msgpack v2 (#16810)" (#17047) 2023-05-01 17:18:34 -04:00
alloc_watcher_e2e_test.go cli: tls certs not created with correct SANs (#16959) 2023-05-22 09:31:56 -04:00
client.go update go1.21 (#18184) 2023-08-15 14:40:33 +02:00
client_interface_test.go prioritized client updates (#17354) 2023-05-31 15:34:16 -04:00
client_stats_endpoint.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
client_stats_endpoint_test.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
client_test.go client: send node secret with every client-to-server RPC (#16799) 2023-06-22 11:06:49 -04:00
csi_endpoint.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
csi_endpoint_test.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
drain.go compliance: add headers with fixed copywrite tool (#17353) 2023-05-30 09:20:32 -05:00
drain_test.go compliance: add headers with fixed copywrite tool (#17353) 2023-05-30 09:20:32 -05:00
driver_manager_test.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
enterprise_client_oss.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
fingerprint_manager.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
fingerprint_manager_test.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
fs_endpoint.go Revert "hashicorp/go-msgpack v2 (#16810)" (#17047) 2023-05-01 17:18:34 -04:00
fs_endpoint_test.go Revert "hashicorp/go-msgpack v2 (#16810)" (#17047) 2023-05-01 17:18:34 -04:00
gc.go allocrunner: provide factory function so we can build mock ARs (#17161) 2023-05-12 13:29:44 -04:00
gc_test.go allocrunner: provide factory function so we can build mock ARs (#17161) 2023-05-12 13:29:44 -04:00
heartbeatstop.go allocrunner: provide factory function so we can build mock ARs (#17161) 2023-05-12 13:29:44 -04:00
heartbeatstop_test.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
meta_endpoint.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
meta_endpoint_test.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
node_updater.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
rpc.go client: send node secret with every client-to-server RPC (#16799) 2023-06-22 11:06:49 -04:00
rpc_test.go cli: tls certs not created with correct SANs (#16959) 2023-05-22 09:31:56 -04:00
testing.go [COMPLIANCE] Add Copyright and License Headers 2023-04-10 15:36:59 +00:00
util.go client: de-duplicate alloc updates and gate during restore (#17074) 2023-05-11 09:05:24 -04:00