Go to file
R.B. Boyer b089f93292
proxycfg: ensure that an irrecoverable error in proxycfg closes the xds session and triggers a replacement proxycfg watcher (#16497)
Receiving an "acl not found" error from an RPC in the agent cache and the
streaming/event components will cause any request loops to cease under the
assumption that they will never work again if the token was destroyed. This
prevents log spam (#14144, #9738).

Unfortunately due to things like:

- authz requests going to stale servers that may not have witnessed the token
  creation yet

- authz requests in a secondary datacenter happening before the tokens get
  replicated to that datacenter

- authz requests from a primary TO a secondary datacenter happening before the
  tokens get replicated to that datacenter

The caller will get an "acl not found" *before* the token exists, rather than
just after. The machinery added above in the linked PRs will kick in and
prevent the request loop from looping around again once the tokens actually
exist.

For `consul-dataplane` usages, where xDS is served by the Consul servers
rather than the clients ultimately this is not a problem because in that
scenario the `agent/proxycfg` machinery is on-demand and launched by a new xDS
stream needing data for a specific service in the catalog. If the watching
goroutines are terminated it ripples down and terminates the xDS stream, which
CDP will eventually re-establish and restart everything.

For Consul client usages, the `agent/proxycfg` machinery is ahead-of-time
launched at service registration time (called "local" in some of the proxycfg
machinery) so when the xDS stream comes in the data is already ready to go. If
the watching goroutines terminate it should terminate the xDS stream, but
there's no mechanism to re-spawn the watching goroutines. If the xDS stream
reconnects it will see no `ConfigSnapshot` and will not get one again until
the client agent is restarted, or the service is re-registered with something
changed in it.

This PR fixes a few things in the machinery:

- there was an inadvertent deadlock in fetching snapshot from the proxycfg
  machinery by xDS, such that when the watching goroutine terminated the
  snapshots would never be fetched. This caused some of the xDS machinery to
  get indefinitely paused and not finish the teardown properly.

- Every 30s we now attempt to re-insert all locally registered services into
  the proxycfg machinery.

- When services are re-inserted into the proxycfg machinery we special case
  "dead" ones such that we unilaterally replace them rather that doing that
  conditionally.
2023-03-03 14:27:53 -06:00
.changelog proxycfg: ensure that an irrecoverable error in proxycfg closes the xds session and triggers a replacement proxycfg watcher (#16497) 2023-03-03 14:27:53 -06:00
.circleci [OSS] security: update go to 1.20.1 (#16263) 2023-02-17 15:04:12 -05:00
.github chore: remove stable-website (#16386) 2023-02-23 10:39:15 -05:00
.release add back staging bits (#16411) 2023-02-23 19:39:40 -06:00
acl Synthesize anonymous token pre-bootstrap when needed (#16200) 2023-02-09 20:34:02 +00:00
agent proxycfg: ensure that an irrecoverable error in proxycfg closes the xds session and triggers a replacement proxycfg watcher (#16497) 2023-03-03 14:27:53 -06:00
api Add support for failover policies (#16505) 2023-03-03 11:12:38 -05:00
bench Gets benchmarks running again and does a rough pass for 0.7.1. 2016-11-29 13:02:26 -08:00
build-support Protobuf Refactoring for Multi-Module Cleanliness (#16302) 2023-02-17 16:14:46 -05:00
command modified unsupported envoy version error (#16518) 2023-03-03 10:29:34 -08:00
connect Protobuf Refactoring for Multi-Module Cleanliness (#16302) 2023-02-17 16:14:46 -05:00
contributing Move contributing to docs 2021-08-30 16:17:09 -04:00
docs Adding experimental support for a more efficient LogStore implementation (#16176) 2023-02-08 16:50:22 +00:00
envoyextensions fix: revert go mod compat for sdk,api to 1.19 (#16323) 2023-02-18 14:58:39 -05:00
grafana add readme outlining how to edit and publish 2021-01-12 14:47:11 -08:00
internal Remove private prefix from proto-gen-rpc-glue e2e test (#16433) 2023-03-03 14:05:14 -05:00
ipaddr Ensure Consul is IPv6 compliant (#5468) 2019-06-04 10:02:38 -04:00
lib Suppress AlreadyRegisteredError to fix test retries (#16501) 2023-03-02 12:08:03 -05:00
logging Run config entry controller routines on leader (#16054) 2023-01-25 12:21:46 -06:00
proto Add support for failover policies (#16505) 2023-03-03 11:12:38 -05:00
proto-public Protobuf Refactoring for Multi-Module Cleanliness (#16302) 2023-02-17 16:14:46 -05:00
sdk chore: document and unit test sdk/testutil/retry (#16049) 2023-02-21 10:48:25 -08:00
sentinel re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
service_os re-run gofmt on 1.17 (#11579) 2021-11-16 12:04:01 -06:00
snapshot removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
test Fix resolution of service resolvers with subsets for external upstreams (#16499) 2023-03-03 14:17:11 -05:00
testrpc Pass remote addr of incoming HTTP requests through to RPC(..) calls (#15700) 2022-12-14 09:24:22 -06:00
tlsutil Protobuf Refactoring for Multi-Module Cleanliness (#16302) 2023-02-17 16:14:46 -05:00
tools/internal-grpc-proxy grpc: rename public/private directories to external/internal (#13721) 2022-07-13 16:33:48 +01:00
troubleshoot fix: revert go mod compat for sdk,api to 1.19 (#16323) 2023-02-18 14:58:39 -05:00
types agent: convert listener config to TLS types (#12522) 2022-03-24 15:32:25 -04:00
ui UI: Fix rendering issue in search and lists (#16444) 2023-02-27 16:31:47 -07:00
version cli: remove stray whitespace when loading the consul version from the VERSION file (#16467) 2023-02-28 14:37:52 -06:00
website update connect/ca's vault AuthMethod conf section (#16346) 2023-03-03 19:32:21 +00:00
.dockerignore Update the scripting 2018-06-14 21:42:47 -04:00
.gitattributes Initial commit 2013-11-04 14:15:27 -08:00
.gitignore grpc: `protoc` plugin for generating gRPC rate limit specifications (#15564) 2023-01-04 16:07:02 +00:00
.golangci.yml removes ioutil usage everywhere which was deprecated in go1.16 (#15297) 2022-11-10 10:26:01 -06:00
CHANGELOG.md update changelog (#16426) 2023-02-24 15:29:07 -08:00
Dockerfile Upgrade Alpine image to 3.17 (#16358) 2023-02-22 10:09:41 -06:00
GNUmakefile Protobuf Refactoring for Multi-Module Cleanliness (#16302) 2023-02-17 16:14:46 -05:00
LICENSE [COMPLIANCE] Update MPL-2.0 LICENSE (#14964) 2022-11-09 12:24:14 -06:00
NOTICE.md add copyright notice file 2018-07-09 10:58:26 -07:00
README.md Fixed broken links referring to tutorials running as local agent (#14954) 2022-10-11 13:01:29 -07:00
buf.work.yaml Protobuf Refactoring for Multi-Module Cleanliness (#16302) 2023-02-17 16:14:46 -05:00
fixup_acl_move.sh Fixup script 2 2022-04-05 14:52:43 -07:00
go.mod [OSS] security: update go to 1.20.1 (#16263) 2023-02-17 15:04:12 -05:00
go.sum Bump x/time to 0.3.0 and fix related breakage linked to RPCRateLimit (#16241) 2023-02-13 11:11:51 -06:00
main.go [OSS] security: update go to 1.20.1 (#16263) 2023-02-17 15:04:12 -05:00

README.md

Consul logo Consul

Docker Pulls Go Report Card

Consul is a distributed, highly available, and data center aware solution to connect and configure applications across dynamic, distributed infrastructure.

Consul provides several key features:

  • Multi-Datacenter - Consul is built to be datacenter aware, and can support any number of regions without complex configuration.

  • Service Mesh - Consul Service Mesh enables secure service-to-service communication with automatic TLS encryption and identity-based authorization. Applications can use sidecar proxies in a service mesh configuration to establish TLS connections for inbound and outbound connections with Transparent Proxy.

  • Service Discovery - Consul makes it simple for services to register themselves and to discover other services via a DNS or HTTP interface. External services such as SaaS providers can be registered as well.

  • Health Checking - Health Checking enables Consul to quickly alert operators about any issues in a cluster. The integration with service discovery prevents routing traffic to unhealthy hosts and enables service level circuit breakers.

  • Key/Value Storage - A flexible key/value store enables storing dynamic configuration, feature flagging, coordination, leader election and more. The simple HTTP API makes it easy to use anywhere.

Consul runs on Linux, macOS, FreeBSD, Solaris, and Windows and includes an optional browser based UI. A commercial version called Consul Enterprise is also available.

Please note: We take Consul's security and our users' trust very seriously. If you believe you have found a security issue in Consul, please responsibly disclose by contacting us at security@hashicorp.com.

Quick Start

A few quick start guides are available on the Consul website:

Documentation

Full, comprehensive documentation is available on the Consul website: https://consul.io/docs

Contributing

Thank you for your interest in contributing! Please refer to CONTRIBUTING.md for guidance. For contributions specifically to the browser based UI, please refer to the UI's README.md for guidance.