luxolus/open-consul
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity
open-consul/agent/consul
History
R.B. Boyer d860384731
server: partly fix config entry replication issue that prevents replication in some circumstances (#12307) 
There are some cross-config-entry relationships that are enforced during
"graph validation" at persistence time that are required to be
maintained. This means that config entries may form a digraph at times.

Config entry replication procedes in a particular sorted order by kind
and name.

Occasionally there are some fixups to these digraphs that end up
replicating in the wrong order and replicating the leaves
(ingress-gateway) before the roots (service-defaults) leading to
replication halting due to a graph validation error related to things
like mismatched service protocol requirements.

This PR changes replication to give each computed change (upsert/delete)
a fair shot at being applied before deciding to terminate that round of
replication in error. In the case where we've simply tried to do the
operations in the wrong order at least ONE of the outstanding requests
will complete in the right order, leading the subsequent round to have
fewer operations to do, with a smaller likelihood of graph validation
errors.

This does not address all scenarios, but for scenarios where the edits
are being applied in the wrong order this should avoid replication
halting.

Fixes #9319

The scenario that is NOT ADDRESSED by this PR is as follows:

1. create: service-defaults: name=new-web, protocol=http
2. create: service-defaults: name=old-web, protocol=http
3. create: service-resolver: name=old-web, redirect-to=new-web
4. delete: service-resolver: name=old-web
5. update: service-defaults: name=old-web, protocol=grpc
6. update: service-defaults: name=new-web, protocol=grpc
7. create: service-resolver: name=old-web, redirect-to=new-web

If you shutdown dc2 just before (4) and turn it back on after (7)
replication is impossible as there is no single edit you can make to
make forward progress.
2022-02-23 17:27:48 -06:00
..
authmethod Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
discoverychain configentry: make a new package to hold shared config entry structs that aren't used for RPC or the FSM (#12384) 2022-02-22 10:36:36 -06:00
fsm Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
prepared_query various partition related todos (#11822) 2021-12-13 11:43:33 -06:00
state configentry: make a new package to hold shared config entry structs that aren't used for RPC or the FSM (#12384) 2022-02-22 10:36:36 -06:00
stream streaming: split event buffer by key (#12080) 2022-01-28 12:27:00 +00:00
testdata ca: examine the full chain in newCARoot 2022-02-17 18:21:30 -05:00
usagemetrics Rename partition-exports to exported-services 2021-12-03 17:47:31 -07:00
wanfed grpc: ensure that streaming gRPC requests work over mesh gateway based wan federation (#10838) 2021-08-24 16:28:44 -05:00
acl.go acl: un-embed ACLIdentity 2022-02-02 12:07:31 -05:00
acl_authmethod.go
acl_authmethod_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
acl_authmethod_test.go
acl_client.go Merge pull request #12165 from hashicorp/dnephin/acl-resolve-token 2022-01-31 13:27:49 -05:00
acl_endpoint.go rpc: add errNotFound to all Get queries 2022-02-15 18:24:34 -05:00
acl_endpoint_legacy.go acl: remove most of the rest of structs/acl_legacy.go 2021-10-25 17:20:06 -04:00
acl_endpoint_oss.go Cross port of ent #1383 (#11726) 2021-12-03 10:20:25 -08:00
acl_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
acl_oss.go various partition related todos (#11822) 2021-12-13 11:43:33 -06:00
acl_oss_test.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
acl_replication.go acl: remove legacy ACL replication 2021-09-03 12:42:06 -04:00
acl_replication_test.go Rename ACLMasterToken => ACLInitialManagementToken (#11746) 2021-12-07 12:39:28 +00:00
acl_replication_types.go Handle FSM.Apply errors in raftApply 2021-04-20 13:29:29 -04:00
acl_server.go Merge pull request #12167 from hashicorp/dnephin/acl-resolve-token-3 2022-01-31 19:21:06 -05:00
acl_server_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
acl_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
acl_token_exp.go acl: embed ACLResolver in Client and Server 2022-01-22 14:07:26 -05:00
acl_token_exp_test.go [OSS] Remove remaining references to master (#11827) 2022-01-20 12:47:50 +00:00
auto_config_backend.go auto-config: ensure the feature works properly with partitions (#11699) 2021-12-01 13:32:34 -06:00
auto_config_backend_test.go [OSS] Remove remaining references to master (#11827) 2022-01-20 12:47:50 +00:00
auto_config_endpoint.go auto-config: ensure the feature works properly with partitions (#11699) 2021-12-01 13:32:34 -06:00
auto_config_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
auto_encrypt_endpoint.go rpc: remove unnecessary arg to ForwardRPC 2021-05-06 13:30:07 -04:00
auto_encrypt_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
autopilot.go agent: refactor the agent delegate interface to be partition friendly (#11429) 2021-10-26 15:08:55 -05:00
autopilot_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
autopilot_test.go bulk rewrite using this script 2022-01-20 10:46:23 -06:00
catalog_endpoint.go Clarify service and check error messages (use ID) 2022-01-04 11:42:37 -08:00
catalog_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
client.go Merge pull request #12165 from hashicorp/dnephin/acl-resolve-token 2022-01-31 13:27:49 -05:00
client_serf.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
client_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
cluster_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
config.go Rename ACLMasterToken => ACLInitialManagementToken (#11746) 2021-12-07 12:39:28 +00:00
config_endpoint.go rpc: set response to nil when not found 2022-02-18 12:26:06 -05:00
config_endpoint_test.go Make blockingQuery efficient with 'not found' results. 2022-02-15 18:24:33 -05:00
config_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
config_replication.go server: partly fix config entry replication issue that prevents replication in some circumstances (#12307) 2022-02-23 17:27:48 -06:00
config_replication_test.go server: partly fix config entry replication issue that prevents replication in some circumstances (#12307) 2022-02-23 17:27:48 -06:00
config_test.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
connect_ca_endpoint.go Use stored entmeta to fill authzContext 2021-10-14 08:57:40 -06:00
connect_ca_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
coordinate_endpoint.go rpc: add errNotFound to all Get queries 2022-02-15 18:24:34 -05:00
coordinate_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
discovery_chain_endpoint.go Remove useInDatacenter from disco chain requests 2021-10-26 23:35:21 -06:00
discovery_chain_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
enterprise_client_oss.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
enterprise_config_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
enterprise_server_oss.go light refactors to support making partitions and serf-based wan federation are mutually exclusive (#11755) 2021-12-06 13:18:02 -06:00
enterprise_server_oss_test.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
federation_state_endpoint.go rpc: set response to nil when not found 2022-02-18 12:26:06 -05:00
federation_state_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
federation_state_replication.go Handle FSM.Apply errors in raftApply 2021-04-20 13:29:29 -04:00
federation_state_replication_test.go testing: skip slow tests with -short 2020-12-07 13:42:55 -05:00
filter.go acl: some acl authz refactors for nodes (#10909) 2021-08-25 13:43:11 -05:00
filter_test.go acl: remove id and revision from Policy constructors 2021-11-05 15:45:08 -04:00
flood.go
gateway_locator.go rpc: improve docs for blockingQuery 2022-02-15 14:20:14 -05:00
gateway_locator_test.go rpc: improve docs for blockingQuery 2022-02-15 14:20:14 -05:00
health_endpoint.go health: support ResultsFilteredByACLs flag/header (#11602) 2021-12-03 17:31:32 +00:00
health_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
helper_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
intention_endpoint.go acl: rename ResolveTokenToIdentityAndAuthorizer to ResolveToken 2022-01-31 18:04:19 -05:00
intention_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
internal_endpoint.go acl: un-embed ACLIdentity 2022-02-02 12:07:31 -05:00
internal_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
issue_test.go stream: Use a no-op event publisher if streaming is disabled 2020-10-28 13:54:19 -04:00
kvs_endpoint.go rpc: add errNotFound to all Get queries 2022-02-15 18:24:34 -05:00
kvs_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
leader.go acl: embed ACLResolver in Client and Server 2022-01-22 14:07:26 -05:00
leader_connect.go Add virtual IP generation for term gateway backed services 2022-01-12 12:08:49 -08:00
leader_connect_ca.go Update TODOs to reference an issue with more details 2022-02-17 18:21:30 -05:00
leader_connect_ca_test.go ca: test that original certs from secondary still verify 2022-02-17 18:45:16 -05:00
leader_connect_test.go ca: examine the full chain in newCARoot 2022-02-17 18:21:30 -05:00
leader_federation_state_ae.go agent/structs: add a bunch more EnterpriseMeta helper functions to help with partitioning (#10669) 2021-07-22 13:20:45 -05:00
leader_federation_state_ae_test.go Rename ACLMasterToken => ACLInitialManagementToken (#11746) 2021-12-07 12:39:28 +00:00
leader_intentions.go Sync enterprise changes to oss (#10994) 2021-09-08 11:59:30 -04:00
leader_intentions_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
leader_intentions_oss_test.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
leader_intentions_test.go configentry: make a new package to hold shared config entry structs that aren't used for RPC or the FSM (#12384) 2022-02-22 10:36:36 -06:00
leader_metrics.go ca: use the new leaf signing lookup func in leader metrics 2022-01-06 16:55:49 -05:00
leader_oss_test.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
leader_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
logging.go
logging_test.go bulk rewrite using this script 2022-01-20 10:46:23 -06:00
merge.go light refactors to support making partitions and serf-based wan federation are mutually exclusive (#11755) 2021-12-06 13:18:02 -06:00
merge_oss.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
merge_oss_test.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
merge_test.go light refactors to support making partitions and serf-based wan federation are mutually exclusive (#11755) 2021-12-06 13:18:02 -06:00
operator_autopilot_endpoint.go Refactor to make ACL errors more structured. (#12308) 2022-02-11 12:53:23 -08:00
operator_autopilot_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
operator_endpoint.go
operator_raft_endpoint.go acl: un-embed ACLIdentity 2022-02-02 12:07:31 -05:00
operator_raft_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
options.go grpc: ensure that streaming gRPC requests work over mesh gateway based wan federation (#10838) 2021-08-24 16:28:44 -05:00
options_oss.go re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
prepared_query_endpoint.go query: support ResultsFilteredByACLs in query list endpoint (#11620) 2021-12-03 23:04:09 +00:00
prepared_query_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
raft_rpc.go rpc: authorize raft requests (#10925) 2021-08-26 15:04:32 -07:00
replication.go Apply suggestions from code review 2022-01-26 12:24:13 -05:00
replication_test.go Move some things around to allow for license updating via config reload 2021-05-25 09:57:50 -04:00
rpc.go rpc: set response to nil when not found 2022-02-18 12:26:06 -05:00
rpc_test.go Make blockingQuery efficient with 'not found' results. 2022-02-15 18:24:33 -05:00
rtt.go agent: ensure that most agent behavior correctly respects partition configuration (#10880) 2021-08-19 15:09:42 -05:00
rtt_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
segment_oss.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
serf_filter.go agent: refactor the agent delegate interface to be partition friendly (#11429) 2021-10-26 15:08:55 -05:00
serf_test.go
server.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
server_connect.go ca: accept only the cluster ID to SpiffeIDSigningForCluster 2021-11-16 16:57:21 -05:00
server_lookup.go
server_lookup_test.go
server_oss.go light refactors to support making partitions and serf-based wan federation are mutually exclusive (#11755) 2021-12-06 13:18:02 -06:00
server_register.go connect: update centralized upstreams representation in service-defaults (#10015) 2021-04-15 14:21:44 -05:00
server_serf.go acl: embed ACLResolver in Client and Server 2022-01-22 14:07:26 -05:00
server_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
session_endpoint.go rpc: add errNotFound to all Get queries 2022-02-15 18:24:34 -05:00
session_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
session_timers.go
session_timers_test.go testing: skip slow tests with -short 2020-12-07 13:42:55 -05:00
session_ttl.go port oss changes (#11736) 2021-12-03 17:23:55 -05:00
session_ttl_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
snapshot_endpoint.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
snapshot_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
stats_fetcher.go Switch to using the external autopilot module 2020-11-09 09:22:11 -05:00
stats_fetcher_test.go Maybe fix another data race in a test 2020-12-22 18:53:54 -05:00
status_endpoint.go Switch to using the external autopilot module 2020-11-09 09:22:11 -05:00
status_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
subscribe_backend.go grpc: strip local ACL tokens from RPCs during forwarding if crossing datacenters (#11099) 2021-09-22 13:14:26 -05:00
subscribe_backend_test.go subscribe: attempt to fix a flaky test 2021-10-27 15:09:09 -04:00
system_metadata.go Handle FSM.Apply errors in raftApply 2021-04-20 13:29:29 -04:00
system_metadata_test.go testing: Revert assertion for virtual IP flag (#11932) 2022-01-04 11:24:56 -05:00
txn_endpoint.go txn: support ResultsFilteredByACLs flag in Read endpoint (#11632) 2021-12-03 20:41:03 +00:00
txn_endpoint_test.go Vendor in rpc mono repo for net/rpc fork, go-msgpack, msgpackrpc. (#12311) 2022-02-14 09:45:45 -08:00
util.go partitions: various refactors to support partitioning the serf LAN pool (#11568) 2021-11-15 09:51:14 -06:00
util_test.go acl: remove legacy ACL upgrades from Server 2021-09-29 15:19:23 -04:00