From ac747888fd7008827afc1dbcbba9768eceafebd4 Mon Sep 17 00:00:00 2001 From: Mike Morris Date: Wed, 1 Jun 2022 13:56:22 -0400 Subject: [PATCH 001/149] docs(consul-api-gateway): add GatewayClassConfig deployment.defaultInstances --- website/content/docs/api-gateway/consul-api-gateway-install.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 087c0e70e..2f8b1ac33 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -168,6 +168,7 @@ The following table describes the allowed parameters for the `spec` array: | `consul.ports.http` | Specifies the port for Consul's HTTP server. | Integer | `8500` | | `consul.scheme` | Specifies the scheme to use for connecting to Consul. The supported values are `"http"` and `"https"`. | String | `"http"` | | `copyAnnotations.service` | List of annotations to copy to the gateway service. | Array | `["external-dns.alpha.kubernetes.io/hostname"]` | +| `deployment.defaultInstances` | Specifies the number of gateway instances that should be deployed by default. | Integer | 1 | | `image.consulAPIGateway` | The image to use for consul-api-gateway. View available image tags on [DockerHub](https://hub.docker.com/r/hashicorp/consul-api-gateway/tags). | String | `"hashicorp/consul-api-gateway:RELEASE_VERSION"` | | `image.envoy` | Specifies the container image to use for Envoy. View available image tags on [DockerHub](https://hub.docker.com/r/envoyproxy/envoy/tags). | String | `"envoyproxy/envoy:RELEASE_VERSION"` | | `logLevel` | Specifies the error reporting level for logs. You can specify the following values: `error`, `warning`, `info`, `debug`, `trace`. | String | `"info"` | From df1666765a466fde750f11561432e42414102ad1 Mon Sep 17 00:00:00 2001 From: Mike Morris Date: Wed, 1 Jun 2022 13:59:50 -0400 Subject: [PATCH 002/149] docs(consul-api-gateway): add GatewayClassConfig deployment.maxInstances and deployment.minInstances --- website/content/docs/api-gateway/consul-api-gateway-install.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 2f8b1ac33..405c917ee 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -169,6 +169,8 @@ The following table describes the allowed parameters for the `spec` array: | `consul.scheme` | Specifies the scheme to use for connecting to Consul. The supported values are `"http"` and `"https"`. | String | `"http"` | | `copyAnnotations.service` | List of annotations to copy to the gateway service. | Array | `["external-dns.alpha.kubernetes.io/hostname"]` | | `deployment.defaultInstances` | Specifies the number of gateway instances that should be deployed by default. | Integer | 1 | +| `deployment.maxInstances` | Specifies the maximum allowed number of gateway instances. | Integer | 8 | +| `deployment.minInstances` | Specifies the minimum allowed number of gateway instances. | Integer | 1 | | `image.consulAPIGateway` | The image to use for consul-api-gateway. View available image tags on [DockerHub](https://hub.docker.com/r/hashicorp/consul-api-gateway/tags). | String | `"hashicorp/consul-api-gateway:RELEASE_VERSION"` | | `image.envoy` | Specifies the container image to use for Envoy. View available image tags on [DockerHub](https://hub.docker.com/r/envoyproxy/envoy/tags). | String | `"envoyproxy/envoy:RELEASE_VERSION"` | | `logLevel` | Specifies the error reporting level for logs. You can specify the following values: `error`, `warning`, `info`, `debug`, `trace`. | String | `"info"` | From 601c13eefec8ca37fefab4a1c33fe8abf31f4425 Mon Sep 17 00:00:00 2001 From: Mike Morris Date: Wed, 1 Jun 2022 14:06:18 -0400 Subject: [PATCH 003/149] docs(consul-api-gateway): gateway instances -> instances per gateway --- .../content/docs/api-gateway/consul-api-gateway-install.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 405c917ee..c231083ab 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -168,9 +168,9 @@ The following table describes the allowed parameters for the `spec` array: | `consul.ports.http` | Specifies the port for Consul's HTTP server. | Integer | `8500` | | `consul.scheme` | Specifies the scheme to use for connecting to Consul. The supported values are `"http"` and `"https"`. | String | `"http"` | | `copyAnnotations.service` | List of annotations to copy to the gateway service. | Array | `["external-dns.alpha.kubernetes.io/hostname"]` | -| `deployment.defaultInstances` | Specifies the number of gateway instances that should be deployed by default. | Integer | 1 | -| `deployment.maxInstances` | Specifies the maximum allowed number of gateway instances. | Integer | 8 | -| `deployment.minInstances` | Specifies the minimum allowed number of gateway instances. | Integer | 1 | +| `deployment.defaultInstances` | Specifies the number of instances per gateway that should be deployed by default. | Integer | 1 | +| `deployment.maxInstances` | Specifies the maximum allowed number of instances per gateway. | Integer | 8 | +| `deployment.minInstances` | Specifies the minimum allowed number of instances per gateway. | Integer | 1 | | `image.consulAPIGateway` | The image to use for consul-api-gateway. View available image tags on [DockerHub](https://hub.docker.com/r/hashicorp/consul-api-gateway/tags). | String | `"hashicorp/consul-api-gateway:RELEASE_VERSION"` | | `image.envoy` | Specifies the container image to use for Envoy. View available image tags on [DockerHub](https://hub.docker.com/r/envoyproxy/envoy/tags). | String | `"envoyproxy/envoy:RELEASE_VERSION"` | | `logLevel` | Specifies the error reporting level for logs. You can specify the following values: `error`, `warning`, `info`, `debug`, `trace`. | String | `"info"` | From 2601a7d5633eaf8033bd74e1bef872f5f1cb1535 Mon Sep 17 00:00:00 2001 From: Mike Morris Date: Wed, 1 Jun 2022 14:42:08 -0400 Subject: [PATCH 004/149] docs(consul-api-gateway): add Gateway scaling section --- .../consul-api-gateway-install.mdx | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index c231083ab..3e2e6be97 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -266,6 +266,25 @@ Add the `listener` object to the `gateway` configuration and specify the followi Refer to the [Kubernetes Gateway API documentation](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.Listener) for details about configuring listeners. +#### Scaling + +A logical gateway object can be scaled to multiple instances within the bounds set in its associated GatewayClassConfig by using the [`kubectl scale`](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment) command. + +``` +> kubectl get deployment --selector api-gateway.consul.hashicorp.com/name=example-gateway +NAME READY UP-TO-DATE AVAILABLE +example-gateway 1/1 1 1 +``` +``` +> kubectl scale deployment/example-gateway --replicas=3 +deployment.apps/api-gateway scaled +``` +``` +> k get deployment -n consul --selector api-gateway.consul.hashicorp.com/name=api-gateway +NAME READY UP-TO-DATE AVAILABLE +api-gateway 3/3 3 3 +``` + ### Route Routes are independent configuration objects that are associated with specific listeners. From 75166b6fbd88502ffb7bcf1b3c4f121b7b529b30 Mon Sep 17 00:00:00 2001 From: Mike Morris Date: Wed, 1 Jun 2022 15:00:06 -0400 Subject: [PATCH 005/149] docs(consul-api-gateway): fixup CLI prompt to match convention --- .../content/docs/api-gateway/consul-api-gateway-install.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 3e2e6be97..99b1ea676 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -271,16 +271,16 @@ Refer to the [Kubernetes Gateway API documentation](https://gateway-api.sigs.k8s A logical gateway object can be scaled to multiple instances within the bounds set in its associated GatewayClassConfig by using the [`kubectl scale`](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment) command. ``` -> kubectl get deployment --selector api-gateway.consul.hashicorp.com/name=example-gateway +$ kubectl get deployment --selector api-gateway.consul.hashicorp.com/name=example-gateway NAME READY UP-TO-DATE AVAILABLE example-gateway 1/1 1 1 ``` ``` -> kubectl scale deployment/example-gateway --replicas=3 +$ kubectl scale deployment/example-gateway --replicas=3 deployment.apps/api-gateway scaled ``` ``` -> k get deployment -n consul --selector api-gateway.consul.hashicorp.com/name=api-gateway +$ k get deployment -n consul --selector api-gateway.consul.hashicorp.com/name=api-gateway NAME READY UP-TO-DATE AVAILABLE api-gateway 3/3 3 3 ``` From 39a450bb75f2fbdc5f083c658afa234872db44bf Mon Sep 17 00:00:00 2001 From: Mike Morris Date: Wed, 1 Jun 2022 15:01:17 -0400 Subject: [PATCH 006/149] docs(consul-api-gateway): fixup code snippets in gateway scaling section --- .../docs/api-gateway/consul-api-gateway-install.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 99b1ea676..6aed5da1d 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -277,12 +277,12 @@ example-gateway 1/1 1 1 ``` ``` $ kubectl scale deployment/example-gateway --replicas=3 -deployment.apps/api-gateway scaled +deployment.apps/example-gateway scaled ``` ``` -$ k get deployment -n consul --selector api-gateway.consul.hashicorp.com/name=api-gateway -NAME READY UP-TO-DATE AVAILABLE -api-gateway 3/3 3 3 +$ kubectl get deployment --selector api-gateway.consul.hashicorp.com/name=example-gateway +NAME READY UP-TO-DATE AVAILABLE +example-gateway 3/3 3 3 ``` ### Route From 00cdae18d513ec16b1f1cd0d47d7884faf5aa5d6 Mon Sep 17 00:00:00 2001 From: Mike Morris Date: Tue, 7 Jun 2022 16:17:45 -0400 Subject: [PATCH 007/149] Update website/content/docs/api-gateway/consul-api-gateway-install.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --- website/content/docs/api-gateway/consul-api-gateway-install.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 6aed5da1d..00938dff2 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -268,7 +268,7 @@ Refer to the [Kubernetes Gateway API documentation](https://gateway-api.sigs.k8s #### Scaling -A logical gateway object can be scaled to multiple instances within the bounds set in its associated GatewayClassConfig by using the [`kubectl scale`](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment) command. +You can scale a logical gateway object to multiple instances with the [`kubectl scale`](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment) command. The object scales according to the bounds set in GatewayClassConfig. ``` $ kubectl get deployment --selector api-gateway.consul.hashicorp.com/name=example-gateway From 24ca62ef1c0bdc7795904aefb98e7772b7a559d9 Mon Sep 17 00:00:00 2001 From: Mike Morris Date: Tue, 7 Jun 2022 16:24:35 -0400 Subject: [PATCH 008/149] Update website/content/docs/api-gateway/consul-api-gateway-install.mdx Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --- website/content/docs/api-gateway/consul-api-gateway-install.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 00938dff2..74fd062b0 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -168,7 +168,7 @@ The following table describes the allowed parameters for the `spec` array: | `consul.ports.http` | Specifies the port for Consul's HTTP server. | Integer | `8500` | | `consul.scheme` | Specifies the scheme to use for connecting to Consul. The supported values are `"http"` and `"https"`. | String | `"http"` | | `copyAnnotations.service` | List of annotations to copy to the gateway service. | Array | `["external-dns.alpha.kubernetes.io/hostname"]` | -| `deployment.defaultInstances` | Specifies the number of instances per gateway that should be deployed by default. | Integer | 1 | +| `deployment.defaultInstances` | Specifies the number of instances to deploy by default for each gateway. | Integer | 1 | | `deployment.maxInstances` | Specifies the maximum allowed number of instances per gateway. | Integer | 8 | | `deployment.minInstances` | Specifies the minimum allowed number of instances per gateway. | Integer | 1 | | `image.consulAPIGateway` | The image to use for consul-api-gateway. View available image tags on [DockerHub](https://hub.docker.com/r/hashicorp/consul-api-gateway/tags). | String | `"hashicorp/consul-api-gateway:RELEASE_VERSION"` | From b8b9be4d5620c87b855f7ef2540cc61d8147ba89 Mon Sep 17 00:00:00 2001 From: boruszak Date: Mon, 13 Jun 2022 12:58:16 -0500 Subject: [PATCH 009/149] Initial page creation --- .../connect/cluster-peering/create-manage-peering.mdx | 8 ++++++++ website/content/docs/connect/cluster-peering/index.mdx | 8 ++++++++ website/content/docs/connect/cluster-peering/k8s.mdx | 8 ++++++++ 3 files changed, 24 insertions(+) create mode 100644 website/content/docs/connect/cluster-peering/create-manage-peering.mdx create mode 100644 website/content/docs/connect/cluster-peering/index.mdx create mode 100644 website/content/docs/connect/cluster-peering/k8s.mdx diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx new file mode 100644 index 000000000..4dcad77a6 --- /dev/null +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -0,0 +1,8 @@ +--- +layout: docs +page_title: Create and Manage Peering Connections +description: |- + +--- + +# Create and Manage Peering Connections diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx new file mode 100644 index 000000000..ecc7a9f90 --- /dev/null +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -0,0 +1,8 @@ +--- +layout: docs +page_title: What is Cluster Peering? +description: |- + +--- + +# What is Cluster Peering? diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx new file mode 100644 index 000000000..91cfc9ce5 --- /dev/null +++ b/website/content/docs/connect/cluster-peering/k8s.mdx @@ -0,0 +1,8 @@ +--- +layout: docs +page_title: Cluster Peering on Kubernetes +description: |- + +--- + +# Cluster Peering on Kubernetes From d3fd58ad8ecc3e160ff44d65a0af46ec8951e731 Mon Sep 17 00:00:00 2001 From: boruszak Date: Mon, 13 Jun 2022 13:31:13 -0500 Subject: [PATCH 010/149] What is Cluster Peering? page --- .../docs/connect/cluster-peering/index.mdx | 41 ++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index ecc7a9f90..992cdb9fc 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -2,7 +2,46 @@ layout: docs page_title: What is Cluster Peering? description: |- - +This page details the cluster peering process for connecting Consul clusters across datacenters, including differences between cluster peering and the similar concept of WAN federation. --- # What is Cluster Peering? + +~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. + +Cluster peering is a Consul feature to allow service connectivity between two independent clusters. Similar to WAN Federation, you can use cluster peering to enable service connectivity between partitions in different datacenters. + +## Overview + +Cluster peering allows Consul clusters in different datacenters to communicate with each other. The cluster peering process consists of the following steps: +1. Create a peering token to share with other clusters +1. Establish a connection between clusters +1. Make services available to other clusters + +For detailed instructions on setting up cluster peering with the Consul CLI, refer to [Create and Manage Peering Connections](/docs/connect/cluster-peering/create-manage-peering). If you prefer to use Kubernetes, refer to [Cluster Peering on Kubernetes](/docs/connect/cluster-peering/k8s). + +### Differences between WAN Federation and cluster peering + +WAN Federation and cluster peering are different ways to connect clusters across datacenters. The most important distinction is that WAN Federation assumes clusters are owned by the same operators, so it uses the gossip protocol to replicate global states like ACLs. As a result, WAN Federation requires a “primary” datacenter to serve as an authority for replicated data. + +Regardless of whether you connect your clusters through WAN Federation or cluster peering, human and machine users can use either peering method to issue cross-datacenter queries for data including service endpoints and key/value stores. + +| | WAN Federation | Cluster Peering | +| ------------------------------------------------- | -------------- | --------------- | +| Connects clusters across datacenters | ✓ | ✓ | +| Shares queries, service endpoints, and key/values | ✓ | ✓ | +| Connects clusters owned by different operators | ✕ | ✓ | +| Functions without declaring “primary datacenter” | ✕ | ✓ | +| Uses gossip protocol | ✓ | ✕ | + +## Mesh gateways + +Mesh gateways for service to service traffic between clusters are available. Currently, mesh gateways for server to server traffic are not available. + +## Technical preview limitations +Not all features and functionality are available in the technical preview release. In particular, be aware of the following limitations: + +- Services exported to peered clusters must not be configured as HTTP. +- Support for dynamic routing such as splits, custom routes, or redirects is not available at this time. +- The ``consul intention` CLI command is not supported. +- L7 permissions are not supported. From 4b306efd94e08ad27dac414885a132f9b7986d54 Mon Sep 17 00:00:00 2001 From: boruszak Date: Mon, 13 Jun 2022 13:41:57 -0500 Subject: [PATCH 011/149] What is Cluster Peering? additional fixes --- website/content/docs/connect/cluster-peering/index.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 992cdb9fc..043a6cede 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -1,8 +1,8 @@ --- layout: docs page_title: What is Cluster Peering? -description: |- -This page details the cluster peering process for connecting Consul clusters across datacenters, including differences between cluster peering and the similar concept of WAN federation. +description: >- + This page details the cluster peering process for connecting Consul clusters across datacenters, including differences between cluster peering and the similar concept of WAN federation. --- # What is Cluster Peering? @@ -20,14 +20,14 @@ Cluster peering allows Consul clusters in different datacenters to communicate w For detailed instructions on setting up cluster peering with the Consul CLI, refer to [Create and Manage Peering Connections](/docs/connect/cluster-peering/create-manage-peering). If you prefer to use Kubernetes, refer to [Cluster Peering on Kubernetes](/docs/connect/cluster-peering/k8s). -### Differences between WAN Federation and cluster peering +### Differences between WAN federation and cluster peering -WAN Federation and cluster peering are different ways to connect clusters across datacenters. The most important distinction is that WAN Federation assumes clusters are owned by the same operators, so it uses the gossip protocol to replicate global states like ACLs. As a result, WAN Federation requires a “primary” datacenter to serve as an authority for replicated data. +WAN Federation and cluster peering are different ways to connect clusters across datacenters. The most important distinction is that WAN Federation assumes clusters are owned by the same operators, so it uses the gossip protocol to replicate global states like ACLs. As a result, WAN Federation requires a “primary datacenter" to serve as an authority for replicated data. Regardless of whether you connect your clusters through WAN Federation or cluster peering, human and machine users can use either peering method to issue cross-datacenter queries for data including service endpoints and key/value stores. | | WAN Federation | Cluster Peering | -| ------------------------------------------------- | -------------- | --------------- | +| :------------------------------------------------ | :------------: | :-------------: | | Connects clusters across datacenters | ✓ | ✓ | | Shares queries, service endpoints, and key/values | ✓ | ✓ | | Connects clusters owned by different operators | ✕ | ✓ | From fb573f7801ee6d5d5cd9bda89af337418eadfbf3 Mon Sep 17 00:00:00 2001 From: boruszak Date: Mon, 13 Jun 2022 14:24:02 -0500 Subject: [PATCH 012/149] Create and Manage Peering Connections page --- .../cluster-peering/create-manage-peering.mdx | 140 +++++++++++++++++- 1 file changed, 138 insertions(+), 2 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 4dcad77a6..ed2130986 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -1,8 +1,144 @@ --- layout: docs page_title: Create and Manage Peering Connections -description: |- - +description: >- + This page describes how to use the Consul CLI to create, manage, and delete peering connections for cluster peering. --- # Create and Manage Peering Connections + +~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. + +A peering token enables cluster peering between different datacenters. Once you generate a peering token, you can use it to initiate a connection between clusters. Then you can export services and authorize other clusters to call those services. + +To peer clusters, you must complete the following steps in order: + +1. Create a peering token +1. Establish a connection between clusters +1. Connect service endpoints +1. Authorize connections between peers + +## Create a peering token + +You can generate peering tokens and initiate connections from either the server agents or the client agents in your clusters. For the current release, we recommend you initiate peering through the client agents in the partitions you want to connect. + +To begin the cluster peering process, generate a peering token in one of your clusters. The other cluster uses this token to establish the peering connection. + +In “cluster-01,” run the ``generate-token`` command. + +```shell-session +$ consul peering generate-token -peer="cluster-02" +``` + +The CLI outputs the peering token, which is a string of alphanumeric characters and symbols. + +For the peering token to function correctly, you must enter the second cluster’s exact name in the ``peer`` parameter. + +## Establish a connection between clusters + +Next, use the peering token to establish a secure connection between the clusters. In the client agents of “cluster-02,” run the ``peering initiate`` command. This command does not generate an output. + +```shell-session +$ consul peering initiate -peer="cluster-01" -token="eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6IlNvbHIifQ.5T7L_L1MPfQ_5FjKGa1fTPqrzwK4bNSM812nW6oyjb8" +``` + +In the peer parameter, specify the first cluster’s exact name. The ``token`` parameter should include the entire peering token created in the first cluster. + +When you connect server agents through cluster peering, they will peer through their default partitions. To connect other partitions through server agents, you must specify the partitions you want to peer using the ``partition`` parameter. + +## Connect service endpoints + +After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to make services, queries, and key/value stores available to peered clusters. + +First, create a configuration entry and specify the ``Kind`` as ``“exported-services”``. + + + +```shell-session +Kind = "exported-services" +Partition = "partition-name" + +Services = [ + { + ## The name and namespace of the service to export. + Name = "service-name" + Namespace = "default" + + ## The list of peer clusters to export the service to. + Consumers = [ + { + ## The peer name to reference in config is the one set + ## during the peering process. + Peer = "cluster-02" + } + ] +``` + + + +Then, add the configuration entry to your cluster. + +```shell-session +$ consul config write peering-config.hcl +``` + +Before you proceed, wait for the clusters to sync and make services available to their peers + +### Check peering connection status + +To find issues blocking the cluster peering, you can call the ``/health/service`` HTTP endpoint. + +```shell-session +$ curl \ + http://127.0.0.1:8500/v1/health/service/service-name?peer=cluster-01 +``` + +You can also check the status of the data replication to the peer cluster. +```shell-session +$ curl \ + http://127.0.0.1:8500/v1/peering/cluster-01 +``` + +If the cluster peering process is successful, the output returns both ``"status": "active"`` and ``"connected": true``. + +## Authorize connections from peers + +Before you can call services from peered clusters, you must set service intentions that authorize those clusters to use specific services. Consul prevents services from being exported to unauthorized clusters. + +First, create a configuration entry and specify the ``Kind`` as ``“service-intentions”``. + + + +```shell-session +Kind = "service-intentions" +Name = "service-name" +Partition = "partition-name" + +Sources = [ + { + Name = "orders" + Peer = "cluster-02" + Action = "allow" + } +] +``` + + + +**Tip:** If the peer’s name is not specified in ``Peer``, then Consul assumes that the service is in the local cluster. + +Then, add the configuration entry to your cluster. + +```shell-session +$ consul config write peering-intentions.hcl +``` + +## Remove peering connections + +After you create a peering connection between clusters in different datacenters, you can “unpeer” the connected clusters. Deleting a peering connection stops data replication to the peer and deletes imported data, including services and CA certificates. + +In “cluster-01,” run the ``peering delete`` command. + +```shell-session +$ consul peering delete -peer="cluster-02" +``` From 42b840b6841183ce213cee63cf3b803cc69c1ddc Mon Sep 17 00:00:00 2001 From: Sarah Alsmiller Date: Mon, 13 Jun 2022 16:05:21 -0500 Subject: [PATCH 013/149] updated referencepolicy to referencegrant, added v0.3.0 upgrade instructions --- .../api-gateway/upgrade-specific-versions.mdx | 221 ++++++++++++++++-- 1 file changed, 201 insertions(+), 20 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index fa38c15bd..86363f5c2 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -9,9 +9,185 @@ description: >- This topic describes how to upgrade Consul API Gateway. -## Breaking Changes -Consul API Gateway v0.2.0 introduces a breaking change for people upgrading from Consul API Gateway v0.1.0. Routes with a `backendRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. +## v0.3.0 + +Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with a `secret` defined in a different namespace now require a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. + +## Requirements + +Ensure that the following requirements are met prior to upgrading: + +- Consul API Gateway should be running version v0.2.1 or lower. +- You should have the ability to run `kubectl` CLI commands. +- `kubectl` should be configured to point to the cluster containing the installation you are upgrading. +- You should have the following permission rights on your Kubernetes cluster: + - `Gateway.read` + - `ReferenceGrant.create` +- (Optional) The [jq](https://stedolan.github.io/jq/download/) command line processor for JSON can be installed, which will ease route retrieval during the upgrade process. + +## Procedure + +-> **NOTE** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.2.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.2.0` + +1. Verify the current version of the `consul-api-gateway-controller` `Deployment`: + + ```shell-session + $ kubectl get deployment --namespace consul consul-api-gateway-controller --output=jsonpath= "{@.spec.template.spec.containers[?(@.name=='api-gateway-controller')].image}" + ``` + + You should receive the following response: + + ```log + "hashicorp/consul-api-gateway:0.1.0" + ``` + +1. Retrieve all gateways that have a secret in a different namespace. If you have installed the [`jq`](https://stedolan.github.io/jq/) utility, you can skip to [step 4](#jq-command-secrets). Otherwise, issue the following command to get all `Gateways` across all namespaces: + + ```shell-session + $ kubectl get Gateway --output json --all-namespaces + ``` + + + If you have any active `Gateways`, you will receive output similar to the following response. The output has been truncated to show only relevant fields: + + ```yaml + apiVersion: v1 + items: + - apiVersion: gateway.networking.k8s.io/v1alpha2 + kind: HTTPRoute + metadata: + name: example-http-route, + namespace: example-namespace, + ... + spec: + parentRefs: + - group: gateway.networking.k8s.io + kind: Gateway + name: gateway + namespace: gw-ns + rules: + - backendRefs: + - group: "" + kind: Service + name: web-backend + namespace: gateway-namespace + ... + ... + - apiVersion: gateway.networking.k8s.io/v1alpha2 + kind: TCPRoute + metadata: + name: example-tcp-route, + namespace: a-different-namespace, + ... + spec: + parentRefs: + - group: gateway.networking.k8s.io + kind: Gateway + name: gateway + namespace: gateway-namespace + rules: + - backendRefs: + - group: "" + kind: Service + name: web-backend + namespace: gateway-namespace + ... + ... + ``` + +1. Inspect the `secret` entries for each of the routes. + + If a `namespace` field is not defined in the `secret` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `secret`. Otherwise, note the `namespace` field values for `secret` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferenceGrant` that explicitly allows each cross-namespace secret-to-gateway pair to prevent the route from breaking (see [step 5](#create-secret-reference-grant)). + + After completing this step, you will have a list of all secrets similar to the following: + + + + ```yaml hideClipboard + example-secret: + - namespace: secret-namespace + parentNamespace: gateway-namespace + + ``` + + + Proceed with the [standard-upgrade](#standard-upgrade) if your list is empty. + + +1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `Gateways` and filter for secrets that require a `ReferenceGrant`. + + ```shell-session + $ kubectl get Gateway -o json -A | jq -r '.items[] | {gateway_name: .metadata.name, gateway_namespace: .metadata.namespace, kind: .kind, crossNamespaceSecrets: ( .metadata.namespace as $parentnamespace | .spec.listeners[] .tls.certificateRefs[] | select(.namespace != null and .namespace != $parentnamespace ) )} ' + ``` + + + The output will resemble the following response if gateways that require a new `ReferenceGrant` are returned: + + + + ```log hideClipboard + { + "gateway_name": "example-gateway", + "gateway_namespace": "gateway-namespace", + "kind": "Gateway", + "crossNamespaceSecrets": { + "group": "", + "kind": "Secret", + "name": "secret-name", + "namespace": "secret-namespace" + } + } + ``` + + + + If your output is empty, proceed with the [standard-upgrade](#standard-upgrade). + + +1. Using the list of secrets you created earlier as a guide, create a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) to allow cross namespace traffic for each route service pair. + The `ReferenceGrant` explicitly allows each cross-namespace gateway to secret pair. The `ReferenceGrant` must be created in the same `namespace` as the `Secret`. + + Skip to the next step if you've already created a `ReferenceGrant`. + + The following example `ReferenceGrant` enables `example-gateway` in `gateway-namespace` to utilize secrets in the `secret-namespace` namespace: + + + + ```yaml + apiVersion: gateway.networking.k8s.io/v1alpha2 + kind: ReferenceGrant + metadata: + name: reference-grant + namespace: secret-namespace + spec: + from: + - group: gateway.networking.k8s.io + kind: Gateway + namespace: gateway-namespace + to: + - group: "" + kind: Secret + ``` + + + +1. If you have already created a `ReferenceGrant`, modify it to allow your route and save it as `referencegrant.yaml`. Note that each `ReferenceGrant` only supports one `to` field and one `from` field (refer the [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencegrant/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferenceGrant`s. + +1. Issue the following command to apply it to your cluster: + + ```shell-session + $ kubectl apply --filename referencegrant.yaml + ``` + + Repeat this step as needed until each of your cross-namespace secrets have a corresponding `ReferenceGrant`. + + +## v0.2.0 + +Consul API Gateway v0.2.0 introduces a breaking change for people upgrading from Consul API Gateway v0.1.0. Routes with a `backendRef` defined in a different namespace now require a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. ## Requirements @@ -23,7 +199,7 @@ Ensure that the following requirements are met prior to upgrading: - You should have the following permission rights on your Kubernetes cluster: - `HTTPRoute.read` - `TCPRoute.read` - - `ReferencePolicy.create` + - `ReferenceGrant.create` - (Optional) The [jq](https://stedolan.github.io/jq/download/) command line processor for JSON can be installed, which will ease route retrieval during the upgrade process. ## Procedure @@ -99,7 +275,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Inspect the `backendRefs` entries for each of the routes. - If a `namespace` field is not defined in the `backendRef` or if the namespace matches the namespace of the route, then no additional action is required for the `backendRef`. Otherwise, note the `group`, `kind`, `name`, and `namespace` field values for `backendRef` configurations that have a `namespace` defined that do not match the namespace of the parent route. You must also note the `kind` and `namespace` of the parent route. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace route-to-service pair to prevent the route from breaking (see [step 5](#create-reference-policy)). + If a `namespace` field is not defined in the `backendRef` or if the namespace matches the namespace of the route, then no additional action is required for the `backendRef`. Otherwise, note the `group`, `kind`, `name`, and `namespace` field values for `backendRef` configurations that have a `namespace` defined that do not match the namespace of the parent route. You must also note the `kind` and `namespace` of the parent route. You will need these to create a `ReferenceGrant` that explicitly allows each cross-namespace route-to-service pair to prevent the route from breaking (see [step 5](#create-reference-grant)). After completing this step, you will have a list of all routes similar to the following: @@ -127,10 +303,10 @@ Ensure that the following requirements are met prior to upgrading: - Skip to [step 8](#step-8) if your list is empty. + Proceed with [standard-upgrade](#standard-upgrade) if your list is empty. -1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `HTTPRoutes` and `TCPRoutes` and filter for routes that require a `ReferencePolicy`. +1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `HTTPRoutes` and `TCPRoutes` and filter for routes that require a `ReferenceGrant`. ```shell-session $ kubectl get HTTPRoute,TCPRoute -o json -A | jq -r '.items[] | {name: .metadata.name, namespace: .metadata.namespace, kind: .kind, crossNamespaceBackendReferences: ( .metadata.namespace as $parentnamespace | .spec.rules[] .backendRefs[] | select(.namespace != null and .namespace != $parentnamespace ) )} ' @@ -138,7 +314,7 @@ Ensure that the following requirements are met prior to upgrading: Note that the command retrieves all `HTTPRoutes` and `TCPRoutes`. `TLSRoutes` and `UDPRoutes` are not supported in v0.1.0. - The output will resemble the following response if routes that require a new `ReferencePolicy` are returned: + The output will resemble the following response if routes that require a new `ReferenceGrant` are returned: @@ -173,25 +349,25 @@ Ensure that the following requirements are met prior to upgrading: - If your output is empty, skip to [step 8](#step-8). - + If your output is empty, proceed with the [standard-upgrade](#standard-upgrade). + -1. Using the list of routes you created earlier as a guide, create a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) to allow cross namespace traffic for each route service pair. - The `ReferencePolicy` explicitly allows each cross-namespace route to service pair to prevent the route from breaking. The `ReferencePolicy` must be created in the same `namespace` as the backend `Service`. +1. Using the list of routes you created earlier as a guide, create a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) to allow cross namespace traffic for each route service pair. + The `ReferenceGrant` explicitly allows each cross-namespace route to service pair to prevent the route from breaking. The `ReferenceGrant` must be created in the same `namespace` as the backend `Service`. - Skip to the next step if you've already created a `ReferencePolicy`. + Skip to the next step if you've already created a `ReferenceGrant`. - The following example `ReferencePolicy` enables `HTTPRoute` traffic from the `example-namespace` to Kubernetes Services in the `web-backend` namespace: + The following example `ReferenceGrant` enables `HTTPRoute` traffic from the `example-namespace` to Kubernetes Services in the `web-backend` namespace: - + ```yaml apiVersion: gateway.networking.k8s.io/v1alpha2 - kind: ReferencePolicy + kind: ReferenceGrant metadata: - name: reference-policy + name: reference-grant namespace: gateway-namespace spec: from: @@ -206,16 +382,21 @@ Ensure that the following requirements are met prior to upgrading: -1. If you have already created a `ReferencePolicy`, modify it to allow your route and save it as `referencepolicy.yaml`. Note that each `ReferencePolicy` only supports one `to` field and one `from` field (refer the [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencepolicy/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferencePolicy`s. +1. If you have already created a `ReferenceGrant`, modify it to allow your route and save it as `referencegrant.yaml`. Note that each `ReferenceGrant` only supports one `to` field and one `from` field (refer the [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencegrant/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferenceGrant`s. 1. Issue the following command to apply it to your cluster: ```shell-session - $ kubectl apply --filename referencepolicy.yaml + $ kubectl apply --filename referencegrant.yaml ``` - Repeat this step as needed until each of your cross-namespace routes have a corresponding `ReferencePolicy`. - + Repeat this step as needed until each of your cross-namespace routes have a corresponding `ReferenceGrant`. + +## Standard Upgrade + +This is the upgrade path to use when there are no version specific steps to take. + + 1. Issue the following command to install the new version of CRDs into your cluster: From ecdb4cda96ed102f5fc1ef6f91a29cd87170464b Mon Sep 17 00:00:00 2001 From: boruszak Date: Mon, 13 Jun 2022 16:06:29 -0500 Subject: [PATCH 014/149] What is Cluster Peering? additional fixes --- .../docs/connect/cluster-peering/index.mdx | 28 ++++++++++--------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 043a6cede..a7587c575 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -24,24 +24,26 @@ For detailed instructions on setting up cluster peering with the Consul CLI, ref WAN Federation and cluster peering are different ways to connect clusters across datacenters. The most important distinction is that WAN Federation assumes clusters are owned by the same operators, so it uses the gossip protocol to replicate global states like ACLs. As a result, WAN Federation requires a “primary datacenter" to serve as an authority for replicated data. -Regardless of whether you connect your clusters through WAN Federation or cluster peering, human and machine users can use either peering method to issue cross-datacenter queries for data including service endpoints and key/value stores. +Regardless of whether you connect your clusters through WAN Federation or cluster peering, human and machine users can use either peering method to issue cross-datacenter queries for data, including service endpoints. -| | WAN Federation | Cluster Peering | -| :------------------------------------------------ | :------------: | :-------------: | -| Connects clusters across datacenters | ✓ | ✓ | -| Shares queries, service endpoints, and key/values | ✓ | ✓ | -| Connects clusters owned by different operators | ✕ | ✓ | -| Functions without declaring “primary datacenter” | ✕ | ✓ | -| Uses gossip protocol | ✓ | ✕ | - -## Mesh gateways - -Mesh gateways for service to service traffic between clusters are available. Currently, mesh gateways for server to server traffic are not available. +| | WAN Federation | Cluster Peering | +| :----------------------------------------------- | :------------: | :-------------: | +| Connects clusters across datacenters | ✅ | ✅ | +| Shares support queries and service endpoints | ✅ | ✅ | +| Connects clusters owned by different operators | ❌ | ✅ | +| Functions without declaring “primary datacenter” | ❌ | ✅ | +| Shares key/value stores | ✅ | ❌ | +| Uses gossip protocol | ✅ | ❌ | ## Technical preview limitations Not all features and functionality are available in the technical preview release. In particular, be aware of the following limitations: +- Mesh gateways for _service to service traffic_ between clusters are available. However, mesh gateways for _server to server traffic_ are not available. - Services exported to peered clusters must not be configured as HTTP. - Support for dynamic routing such as splits, custom routes, or redirects is not available at this time. - The ``consul intention` CLI command is not supported. -- L7 permissions are not supported. +- [L7 permissions](/docs/connect/l7-traffic) are not supported. +- Configuring service failover across peers is not supported. +- Accessing key/value stores across peers is not supported. +- Consul datacenters that are already federated stay federated. +- Non-enterprise Consul instances cannot sync services with namespaces outside of the default namespace. From 0428a2ef9901660434659d5cc51422965e207e3c Mon Sep 17 00:00:00 2001 From: Sarah Alsmiller Date: Mon, 13 Jun 2022 16:16:45 -0500 Subject: [PATCH 015/149] light restructureing/fixed some copypasta --- .../api-gateway/upgrade-specific-versions.mdx | 28 +++++++++++++------ 1 file changed, 19 insertions(+), 9 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 86363f5c2..81901a1cf 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -12,9 +12,9 @@ This topic describes how to upgrade Consul API Gateway. ## v0.3.0 -Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with a `secret` defined in a different namespace now require a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. +Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with a `secret` defined in a different namespace now require a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) that explicitly allows `Gateways` from the gateway's namesapce to use `Secrets` in the secret's namespace. -## Requirements +### Requirements Ensure that the following requirements are met prior to upgrading: @@ -26,9 +26,8 @@ Ensure that the following requirements are met prior to upgrading: - `ReferenceGrant.create` - (Optional) The [jq](https://stedolan.github.io/jq/download/) command line processor for JSON can be installed, which will ease route retrieval during the upgrade process. -## Procedure +### Procedure --> **NOTE** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.2.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.2.0` 1. Verify the current version of the `consul-api-gateway-controller` `Deployment`: @@ -145,7 +144,7 @@ Ensure that the following requirements are met prior to upgrading: If your output is empty, proceed with the [standard-upgrade](#standard-upgrade). -1. Using the list of secrets you created earlier as a guide, create a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) to allow cross namespace traffic for each route service pair. +1. Using the list of secrets you created earlier as a guide, create a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) to allow each gateway cross namespace secret access. The `ReferenceGrant` explicitly allows each cross-namespace gateway to secret pair. The `ReferenceGrant` must be created in the same `namespace` as the `Secret`. Skip to the next step if you've already created a `ReferenceGrant`. @@ -189,7 +188,7 @@ Ensure that the following requirements are met prior to upgrading: Consul API Gateway v0.2.0 introduces a breaking change for people upgrading from Consul API Gateway v0.1.0. Routes with a `backendRef` defined in a different namespace now require a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. -## Requirements +### Requirements Ensure that the following requirements are met prior to upgrading: @@ -202,9 +201,8 @@ Ensure that the following requirements are met prior to upgrading: - `ReferenceGrant.create` - (Optional) The [jq](https://stedolan.github.io/jq/download/) command line processor for JSON can be installed, which will ease route retrieval during the upgrade process. -## Procedure +### Procedure --> **NOTE** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.2.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.2.0` 1. Verify the current version of the `consul-api-gateway-controller` `Deployment`: @@ -394,6 +392,18 @@ Ensure that the following requirements are met prior to upgrading: ## Standard Upgrade +-> **NOTE** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.2.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.2.0` + +### Requirements + +Ensure that the following requirements are met prior to upgrading: + +- You should have the ability to run `kubectl` CLI commands. +- `kubectl` should be configured to point to the cluster containing the installation you are upgrading. + + +### Procedure + This is the upgrade path to use when there are no version specific steps to take. @@ -442,6 +452,6 @@ This is the upgrade path to use when there are no version specific steps to take $ kubectl create --filename ``` -## Post-Upgrade Configuration Changes +### Post-Upgrade Configuration Changes No additional configuration changes are required for this upgrade. From 00e648ab891ffc1de205b5f70a36611747c911e6 Mon Sep 17 00:00:00 2001 From: boruszak Date: Mon, 13 Jun 2022 16:38:44 -0500 Subject: [PATCH 016/149] Create and Manage Peering Connections additional fixes --- .../cluster-peering/create-manage-peering.mdx | 42 ++++++++----------- 1 file changed, 18 insertions(+), 24 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index ed2130986..548b56218 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -30,16 +30,16 @@ In “cluster-01,” run the ``generate-token`` command. $ consul peering generate-token -peer="cluster-02" ``` -The CLI outputs the peering token, which is a string of alphanumeric characters and symbols. +The CLI outputs the peering token, which is an encoded string of alphanumeric characters and symbols. For the peering token to function correctly, you must enter the second cluster’s exact name in the ``peer`` parameter. ## Establish a connection between clusters -Next, use the peering token to establish a secure connection between the clusters. In the client agents of “cluster-02,” run the ``peering initiate`` command. This command does not generate an output. +Next, use the peering token to establish a secure connection between the clusters. In the client agents of “cluster-02,” run the ``peering establish`` command. This command does not generate an output. ```shell-session -$ consul peering initiate -peer="cluster-01" -token="eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6IlNvbHIifQ.5T7L_L1MPfQ_5FjKGa1fTPqrzwK4bNSM812nW6oyjb8" +$ consul peering establish -peer="cluster-01" -token="eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6IlNvbHIifQ.5T7L_L1MPfQ_5FjKGa1fTPqrzwK4bNSM812nW6oyjb8" ``` In the peer parameter, specify the first cluster’s exact name. The ``token`` parameter should include the entire peering token created in the first cluster. @@ -48,7 +48,7 @@ When you connect server agents through cluster peering, they will peer through t ## Connect service endpoints -After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to make services, queries, and key/value stores available to peered clusters. +After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to make services and support queries for service mesh available to peered clusters. First, create a configuration entry and specify the ``Kind`` as ``“exported-services”``. @@ -84,39 +84,22 @@ $ consul config write peering-config.hcl Before you proceed, wait for the clusters to sync and make services available to their peers -### Check peering connection status - -To find issues blocking the cluster peering, you can call the ``/health/service`` HTTP endpoint. - -```shell-session -$ curl \ - http://127.0.0.1:8500/v1/health/service/service-name?peer=cluster-01 -``` - -You can also check the status of the data replication to the peer cluster. -```shell-session -$ curl \ - http://127.0.0.1:8500/v1/peering/cluster-01 -``` - -If the cluster peering process is successful, the output returns both ``"status": "active"`` and ``"connected": true``. - ## Authorize connections from peers Before you can call services from peered clusters, you must set service intentions that authorize those clusters to use specific services. Consul prevents services from being exported to unauthorized clusters. -First, create a configuration entry and specify the ``Kind`` as ``“service-intentions”``. +First, create a configuration entry and specify the ``Kind`` as ``“service-intentions”``. Declare the service on "cluster-02" that can access the service in "cluster-01." The following example sets service intentions so that "backend-service" can access "frontend-service." ```shell-session Kind = "service-intentions" -Name = "service-name" +Name = "frontend-service" Partition = "partition-name" Sources = [ { - Name = "orders" + Name = "backend-service" Peer = "cluster-02" Action = "allow" } @@ -133,6 +116,17 @@ Then, add the configuration entry to your cluster. $ consul config write peering-intentions.hcl ``` +## Check peered cluster status + +To confirm that you peered your clusters, you can [query the ``/health/service`` endpoint](https://www.consul.io/api-docs/health) of one cluster from the other cluster. For example, in "cluster-02," query the endpoint and add the ``peer=cluster-01`` parameter to the end of the URL. + +```shell-session +$ curl \ + http://127.0.0.1:8500/v1/health/service/service-name?peer=cluster-01 +``` + +Successful connections include ``"Status": "passing"`` in the output. + ## Remove peering connections After you create a peering connection between clusters in different datacenters, you can “unpeer” the connected clusters. Deleting a peering connection stops data replication to the peer and deletes imported data, including services and CA certificates. From ad4712334d8beac82aff2694a8c3678da4a0395c Mon Sep 17 00:00:00 2001 From: boruszak Date: Mon, 13 Jun 2022 16:42:29 -0500 Subject: [PATCH 017/149] Typo fix --- website/content/docs/connect/cluster-peering/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index a7587c575..324c95712 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -41,7 +41,7 @@ Not all features and functionality are available in the technical preview releas - Mesh gateways for _service to service traffic_ between clusters are available. However, mesh gateways for _server to server traffic_ are not available. - Services exported to peered clusters must not be configured as HTTP. - Support for dynamic routing such as splits, custom routes, or redirects is not available at this time. -- The ``consul intention` CLI command is not supported. +- The ``consul intention`` CLI command is not supported. - [L7 permissions](/docs/connect/l7-traffic) are not supported. - Configuring service failover across peers is not supported. - Accessing key/value stores across peers is not supported. From 0c1b6d77d83b82228ee30d5238c317b58003cced Mon Sep 17 00:00:00 2001 From: boruszak Date: Mon, 13 Jun 2022 16:47:47 -0500 Subject: [PATCH 018/149] Removing Kubernetes page - will submit separate PR for timing reason --- website/content/docs/connect/cluster-peering/k8s.mdx | 8 -------- 1 file changed, 8 deletions(-) delete mode 100644 website/content/docs/connect/cluster-peering/k8s.mdx diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx deleted file mode 100644 index 91cfc9ce5..000000000 --- a/website/content/docs/connect/cluster-peering/k8s.mdx +++ /dev/null @@ -1,8 +0,0 @@ ---- -layout: docs -page_title: Cluster Peering on Kubernetes -description: |- - ---- - -# Cluster Peering on Kubernetes From 3097f4d00fcc249563014cb34996887b332b97aa Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Mon, 13 Jun 2022 17:27:11 -0500 Subject: [PATCH 019/149] Added nav data --- website/data/docs-nav-data.json | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index f009f5611..139508de9 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -321,6 +321,19 @@ } ] }, + { + "title": "Cluster Peering", + "routes": [ + { + "title": "Overview", + "path": "connect/cluster-peering" + }, + { + "title": "Create and Manage Peering Connections", + "path": "connect/cluster-peering/create-manage-peering" + } + ] + }, { "title": "Nomad", "path": "connect/nomad" From 2e5163dfdc839b574cff874613acc404fbb58554 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Tue, 14 Jun 2022 10:00:22 -0500 Subject: [PATCH 020/149] Fixing double-ticks ` --- .../cluster-peering/create-manage-peering.mdx | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 548b56218..ba4269709 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -24,7 +24,7 @@ You can generate peering tokens and initiate connections from either the server To begin the cluster peering process, generate a peering token in one of your clusters. The other cluster uses this token to establish the peering connection. -In “cluster-01,” run the ``generate-token`` command. +In “cluster-01,” run the `generate-token` command. ```shell-session $ consul peering generate-token -peer="cluster-02" @@ -32,7 +32,7 @@ $ consul peering generate-token -peer="cluster-02" The CLI outputs the peering token, which is an encoded string of alphanumeric characters and symbols. -For the peering token to function correctly, you must enter the second cluster’s exact name in the ``peer`` parameter. +For the peering token to function correctly, you must enter the second cluster’s exact name in the `peer` parameter. ## Establish a connection between clusters @@ -42,15 +42,15 @@ Next, use the peering token to establish a secure connection between the cluster $ consul peering establish -peer="cluster-01" -token="eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6IlNvbHIifQ.5T7L_L1MPfQ_5FjKGa1fTPqrzwK4bNSM812nW6oyjb8" ``` -In the peer parameter, specify the first cluster’s exact name. The ``token`` parameter should include the entire peering token created in the first cluster. +In the peer parameter, specify the first cluster’s exact name. The `token` parameter should include the entire peering token created in the first cluster. -When you connect server agents through cluster peering, they will peer through their default partitions. To connect other partitions through server agents, you must specify the partitions you want to peer using the ``partition`` parameter. +When you connect server agents through cluster peering, they will peer through their default partitions. To connect other partitions through server agents, you must specify the partitions you want to peer using the `partition` parameter. ## Connect service endpoints After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to make services and support queries for service mesh available to peered clusters. -First, create a configuration entry and specify the ``Kind`` as ``“exported-services”``. +First, create a configuration entry and specify the `Kind` as `“exported-services”`. @@ -88,7 +88,7 @@ Before you proceed, wait for the clusters to sync and make services available to Before you can call services from peered clusters, you must set service intentions that authorize those clusters to use specific services. Consul prevents services from being exported to unauthorized clusters. -First, create a configuration entry and specify the ``Kind`` as ``“service-intentions”``. Declare the service on "cluster-02" that can access the service in "cluster-01." The following example sets service intentions so that "backend-service" can access "frontend-service." +First, create a configuration entry and specify the `Kind` as `“service-intentions”`. Declare the service on "cluster-02" that can access the service in "cluster-01." The following example sets service intentions so that "backend-service" can access "frontend-service." @@ -108,7 +108,7 @@ Sources = [ -**Tip:** If the peer’s name is not specified in ``Peer``, then Consul assumes that the service is in the local cluster. +**Tip:** If the peer’s name is not specified in `Peer`, then Consul assumes that the service is in the local cluster. Then, add the configuration entry to your cluster. @@ -118,20 +118,20 @@ $ consul config write peering-intentions.hcl ## Check peered cluster status -To confirm that you peered your clusters, you can [query the ``/health/service`` endpoint](https://www.consul.io/api-docs/health) of one cluster from the other cluster. For example, in "cluster-02," query the endpoint and add the ``peer=cluster-01`` parameter to the end of the URL. +To confirm that you peered your clusters, you can [query the `/health/service` endpoint](https://www.consul.io/api-docs/health) of one cluster from the other cluster. For example, in "cluster-02," query the endpoint and add the `peer=cluster-01` parameter to the end of the URL. ```shell-session $ curl \ http://127.0.0.1:8500/v1/health/service/service-name?peer=cluster-01 ``` -Successful connections include ``"Status": "passing"`` in the output. +Successful connections include `"Status": "passing"` in the output. ## Remove peering connections After you create a peering connection between clusters in different datacenters, you can “unpeer” the connected clusters. Deleting a peering connection stops data replication to the peer and deletes imported data, including services and CA certificates. -In “cluster-01,” run the ``peering delete`` command. +In “cluster-01,” run the `peering delete` command. ```shell-session $ consul peering delete -peer="cluster-02" From ac0be8644a32cd1bd82418e1381dcd526a1fc8ce Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 14 Jun 2022 15:11:28 -0400 Subject: [PATCH 021/149] Add docs for ReferencePolicy as it applies to Gateways --- .../consul-api-gateway-install.mdx | 51 ++++++++++++++++++- website/content/docs/api-gateway/index.mdx | 2 +- 2 files changed, 51 insertions(+), 2 deletions(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 74fd062b0..fb6cbdd71 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -244,7 +244,56 @@ spec: -Refer to the [Kubernetes Gateway API documentation](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.Gateway) for details about configuring gateways: +For a listener's `certificateRef` to reference a secret in a different namespace, you must also create a [ReferencePolicy](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy). + +The following example creates a `Gateway` named `example-gateway` in namespace `gateway-namespace`. This `Gateway` has a `certificateRef` in namespace `secret-namespace`. +The reference is allowed because the `ReferencePolicy`, named `reference-policy` in namespace `secret-namespace`, allows `Gateways` in `gateway-namespace` to reference `Secrets` in `secret-namespace`. + + + +```yaml +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: Gateway +metadata: + name: example-gateway + namespace: gateway-namespace + annotations: + 'external-dns.alpha.kubernetes.io/hostname': DNS_HOSTNAME +spec: + gatewayClassName: test-gateway-class + listeners: + - protocol: HTTPS + hostname: DNS_HOSTNAME + port: 443 + name: https + allowedRoutes: + namespaces: + from: Same + tls: + certificateRefs: + - name: gateway-production-certificate + namespace: secret-namespace +--- + +apiVersion: gateway.networking.k8s.io/v1alpha2 +kind: ReferencePolicy +metadata: + name: reference-policy + namespace: secret-namespace +spec: + from: + - group: gateway.networking.k8s.io + kind: Gateway + namespace: gateway-namespace + to: + - group: "" + kind: Secret + name: gateway-production-certificate +``` + + + +Refer to the [Kubernetes Gateway API documentation](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.Gateway) for further details about configuring gateways. #### Listeners diff --git a/website/content/docs/api-gateway/index.mdx b/website/content/docs/api-gateway/index.mdx index 5234eac60..d279bc5e6 100644 --- a/website/content/docs/api-gateway/index.mdx +++ b/website/content/docs/api-gateway/index.mdx @@ -38,7 +38,7 @@ are used, see the [documentation in our GitHub repo](https://github.com/hashicor | [`Gateway`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.Gateway) |
  • Supported protocols: `HTTP`, `HTTPS`, `TCP`
  • Header-based hostname matching (no SNI support)
  • Supported filters: header addition, removal, and setting
  • TLS modes supported: `terminate`
  • Certificate types supported: `core/v1/Secret`
  • Extended options: TLS version and cipher constraints
| | [`HTTPRoute`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.HTTPRoute) |
  • Weight-based load balancing
  • Supported rules: path, header, query, and method-based matching
  • Supported filters: header addition, removal, and setting
  • Supported backend types:
    1. `core/v1/Service` (must map to a registered Consul service)
    2. `api-gateway.consul.hashicorp.com/v1alpha1/MeshService`
| | [`TCPRoute`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.TCPRoute) |
  • Supported backend types:
    1. `core/v1/Service` (must map to a registered Consul service)
    2. `api-gateway.consul.hashicorp.com/v1alpha1/MeshService`
| -| [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) |
  • Required to allow any reference from an `HTTPRoute` or `TCPRoute` to a Kubernetes `core/v1/Service` in a different namespace.
  • A route with an unpermitted BackendRef caused by the lack of a ReferencePolicy sets a `ResolvedRefs` status to `False` with the reason `RefNotPermitted`. The gateway listener rejects routes with an unpermitted BackendRef.
  • WARNING: If a route BackendRef becomes unpermitted, the entire route is removed from the gateway listener.
    • A BackendRef can become unpermitted when you delete a Reference Policy or add a new unpermitted BackendRef to an existing route.
| +| [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) |
  • Required to allow any reference from a `Gateway` to a Kubernetes `core/v1/Secret` in a different namespace.
  • Required to allow any reference from an `HTTPRoute` or `TCPRoute` to a Kubernetes `core/v1/Service` in a different namespace.
  • A route with an unpermitted BackendRef caused by the lack of a ReferencePolicy sets a `ResolvedRefs` status to `False` with the reason `RefNotPermitted`. The gateway listener rejects routes with an unpermitted BackendRef.
  • WARNING: If a route BackendRef becomes unpermitted, the entire route is removed from the gateway listener.
    • A BackendRef can become unpermitted when you delete a Reference Policy or add a new unpermitted BackendRef to an existing route.
| ## Additional Resources From 0bffbc429cd718e04f43f38b3f8b3ad3dd367793 Mon Sep 17 00:00:00 2001 From: boruszak Date: Tue, 14 Jun 2022 16:15:57 -0500 Subject: [PATCH 022/149] Cluster Peering on Kubernetes page creation --- website/content/docs/connect/cluster-peering/k8s.mdx | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 website/content/docs/connect/cluster-peering/k8s.mdx diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx new file mode 100644 index 000000000..e69de29bb From a768fb8e804d01440e0c2ca0357caa79a5960dab Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 14 Jun 2022 17:15:50 -0400 Subject: [PATCH 023/149] Indent points specific to xRoute backend references --- website/content/docs/api-gateway/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/index.mdx b/website/content/docs/api-gateway/index.mdx index d279bc5e6..d00e54bf7 100644 --- a/website/content/docs/api-gateway/index.mdx +++ b/website/content/docs/api-gateway/index.mdx @@ -38,7 +38,7 @@ are used, see the [documentation in our GitHub repo](https://github.com/hashicor | [`Gateway`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.Gateway) |
  • Supported protocols: `HTTP`, `HTTPS`, `TCP`
  • Header-based hostname matching (no SNI support)
  • Supported filters: header addition, removal, and setting
  • TLS modes supported: `terminate`
  • Certificate types supported: `core/v1/Secret`
  • Extended options: TLS version and cipher constraints
| | [`HTTPRoute`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.HTTPRoute) |
  • Weight-based load balancing
  • Supported rules: path, header, query, and method-based matching
  • Supported filters: header addition, removal, and setting
  • Supported backend types:
    1. `core/v1/Service` (must map to a registered Consul service)
    2. `api-gateway.consul.hashicorp.com/v1alpha1/MeshService`
| | [`TCPRoute`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.TCPRoute) |
  • Supported backend types:
    1. `core/v1/Service` (must map to a registered Consul service)
    2. `api-gateway.consul.hashicorp.com/v1alpha1/MeshService`
| -| [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) |
  • Required to allow any reference from a `Gateway` to a Kubernetes `core/v1/Secret` in a different namespace.
  • Required to allow any reference from an `HTTPRoute` or `TCPRoute` to a Kubernetes `core/v1/Service` in a different namespace.
  • A route with an unpermitted BackendRef caused by the lack of a ReferencePolicy sets a `ResolvedRefs` status to `False` with the reason `RefNotPermitted`. The gateway listener rejects routes with an unpermitted BackendRef.
  • WARNING: If a route BackendRef becomes unpermitted, the entire route is removed from the gateway listener.
    • A BackendRef can become unpermitted when you delete a Reference Policy or add a new unpermitted BackendRef to an existing route.
| +| [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) |
  • Required to allow any reference from a `Gateway` to a Kubernetes `core/v1/Secret` in a different namespace.
  • Required to allow any reference from an `HTTPRoute` or `TCPRoute` to a Kubernetes `core/v1/Service` in a different namespace.
    • A route with an unpermitted BackendRef caused by the lack of a ReferencePolicy sets a `ResolvedRefs` status to `False` with the reason `RefNotPermitted`. The gateway listener rejects routes with an unpermitted BackendRef.
    • WARNING: If a route BackendRef becomes unpermitted, the entire route is removed from the gateway listener.
      • A BackendRef can become unpermitted when you delete a Reference Policy or add a new unpermitted BackendRef to an existing route.
| ## Additional Resources From c901667dd436ecb87a70e02aa84dbadc8f334e90 Mon Sep 17 00:00:00 2001 From: boruszak Date: Tue, 14 Jun 2022 16:33:29 -0500 Subject: [PATCH 024/149] Cluster Peering on Kubernetes initial draft --- .../docs/connect/cluster-peering/k8s.mdx | 187 ++++++++++++++++++ 1 file changed, 187 insertions(+) diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx index e69de29bb..95d4ad3e2 100644 --- a/website/content/docs/connect/cluster-peering/k8s.mdx +++ b/website/content/docs/connect/cluster-peering/k8s.mdx @@ -0,0 +1,187 @@ +--- +layout: docs +page_title: Cluster Peering on Kubernetes +description: >- + This page describes how to create peering connections, deploy services, export cluster services, and end peering connections for Consul cluster peering using Kubernetes (K8s). +--- + +# Cluster Peering on Kubernetes + +~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. + +To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart, and create custom resource definitions for each side of the peering. + +The following Custom Resource Definitions (CRDs) are used to create and manage a peering connection: + +- `PeeringAcceptor`: Generates a peering token and accepts an incoming peering connection. +- `PeeringDialer`: Uses a peering token to make an outbound peering connection with the cluster that generated the token. + +## Prerequisites + +To create and use cluster peering connections with Kubernetes, you need at least two Kubernetes clusters running in a flat network with Consul on Kubernetes v. 0.45 or later. + +### Helm chart configuration +To establish cluster peering through Kubernetes, deploy clusters with the following Helm values. + +```yaml +global: + peering: + enabled: true +connectInject: + enabled: true +meshGateway: + enabled: true + replicas: 1 +``` + +## Create a peering connection + +To peer Kubernetes clusters running Consul, you need to create a peering token and share it with the other cluster. + +1. In “cluster-01,” create the `PeeringAcceptor` custom resource. + + ```yaml + apiVersion: consul.hashicorp.com/v1alpha1 + kind: PeeringAcceptor + metadata: + name: cluster-02 ## The name of the peer you want to connect to + spec: + peer: + secret: + name: "peering-token" + key: "data" + backend: "kubernetes" + ``` + +1. Apply the `PeeringAcceptor` resource to the first cluster. + + ```shell-session + $ kubectl apply -f acceptor.yml + ```` + +1. Save your peering token so that you can export it to the second cluster. + + ```shell-session + $ kubectl get secret peering-token -o yaml > peering-token.yml + ``` + +1. Apply the peering token to the second cluster. + + ```shell-session + $ kubectl apply -f peering-token.yml + ``` + +1. In “cluster-02,” create the `PeeringDialer` custom resource. + + ```yaml + apiVersion: consul.hashicorp.com/v1alpha1 + kind: PeeringDialer + metadata: + name: cluster-01 ## The name of the peer you want to connect to + spec: + peer: + secret: + name: "peering-token" + key: "data" + backend: "kubernetes" + ``` + +1. Apply the `PeeringDialer` resource to the second cluster. + + ```shell-session + $ kubectl apply -f dialer.yml + ``` + +## Deploy and export cluster services + +1. For the service in “cluster-02” that you want to export, add the following [annotations to your service files](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams). + + ```yaml + … + annotations: + "consul.hashicorp.com/connect-inject": "true" + "consul.hashicorp.com/transparent-proxy": "false" + … + ``` + +1. In “cluster-02,” create an `ExportedServices` custom resource. + + ```yaml + apiVersion: consul.hashicorp.com/v1alpha1 + kind: ExportedServices + metadata: + name: default ## The name of the partition containing the service + spec: + services: + name: backend-service ## The name of the service you want to export + consumers: + peerName: cluster-01 ## The name of the peer that receives the service + ``` + +1. Create service intentions for the second cluster. + + ```yaml + apiVersion: consul.hashicorp.com/v1alpha1 + kind: ServiceIntentions + metadata: + name: backend-deny + spec: + destination: + name: backend-service + sources: + - name: "*" + action: deny + - name: frontend-service + action: allow + ``` + +1. Apply the service file, the `ExportedServices` resource, and the intentions in the second cluster. + + ```shell-session + $ kubectl apply -f backend-service.yml; kubectl apply -f exportedsvc.yml; kubectl apply -f intention.yml + ``` + +1. To confirm that you peered your clusters, in “cluster-01,” query the `/health` HTTP endpoint. + + ```shell-session + $ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02" + ``` + +1. For the services in “cluster-01” that you want to access the “backend-service,” add the following annotations to the service file. + + ```yaml + … + annotations: + "consul.hashicorp.com/connect-inject": "true" + "consul.hashicorp.com/transparent-proxy": "false" + "consul.hashicorp.com/connect-service-upstreams": "backend-service.svc.cluster-02.peer:1234" + … + ``` + +1. Apply the service file to the first cluster. + + ```shell-session + $ kubectl apply -f frontend-service.yml + ``` + +1. Run the following command and check the output to confirm that you peered your clusters successfully. + + ```shell-session + $ curl localhost:1234 + { + “name”: “backend-service”, + … + “body”: “Response from backend”, + “code”: 200 + } + ``` + +## End a peering connection + +To end a peering connection, delete both the `PeeringAcceptor` and `PeeringDialer` resources. + +To confirm that you deleted your peering connection, in “cluster-01,” query the `/health` HTTP endpoint. The peered services should no longer appear. + +```shell-session +$ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02" +``` From 0b60e6b53950954bfeed604a628c19daa8c7a771 Mon Sep 17 00:00:00 2001 From: boruszak Date: Tue, 14 Jun 2022 16:55:25 -0500 Subject: [PATCH 025/149] Code Block fixes --- .../docs/connect/cluster-peering/k8s.mdx | 27 +++++++++++++------ 1 file changed, 19 insertions(+), 8 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx index 95d4ad3e2..03563997e 100644 --- a/website/content/docs/connect/cluster-peering/k8s.mdx +++ b/website/content/docs/connect/cluster-peering/k8s.mdx @@ -9,7 +9,7 @@ description: >- ~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. -To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart, and create custom resource definitions for each side of the peering. +To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart and create custom resource definitions for each side of the peering. The following Custom Resource Definitions (CRDs) are used to create and manage a peering connection: @@ -40,6 +40,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. In “cluster-01,” create the `PeeringAcceptor` custom resource. + ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: PeeringAcceptor @@ -52,6 +53,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a key: "data" backend: "kubernetes" ``` + 1. Apply the `PeeringAcceptor` resource to the first cluster. @@ -59,7 +61,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a $ kubectl apply -f acceptor.yml ```` -1. Save your peering token so that you can export it to the second cluster. +1. Save your peering token so that you can export it to the other cluster. ```shell-session $ kubectl get secret peering-token -o yaml > peering-token.yml @@ -73,6 +75,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. In “cluster-02,” create the `PeeringDialer` custom resource. + ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: PeeringDialer @@ -85,6 +88,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a key: "data" backend: "kubernetes" ``` + 1. Apply the `PeeringDialer` resource to the second cluster. @@ -96,16 +100,19 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. For the service in “cluster-02” that you want to export, add the following [annotations to your service files](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams). + ```yaml - … + ##… annotations: "consul.hashicorp.com/connect-inject": "true" "consul.hashicorp.com/transparent-proxy": "false" - … + ##… ``` + 1. In “cluster-02,” create an `ExportedServices` custom resource. + ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: ExportedServices @@ -120,6 +127,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. Create service intentions for the second cluster. + ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceIntentions @@ -134,8 +142,9 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a - name: frontend-service action: allow ``` + -1. Apply the service file, the `ExportedServices` resource, and the intentions in the second cluster. +1. Apply the service file, the `ExportedServices` resource, and the intentions to the second cluster. ```shell-session $ kubectl apply -f backend-service.yml; kubectl apply -f exportedsvc.yml; kubectl apply -f intention.yml @@ -149,14 +158,16 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. For the services in “cluster-01” that you want to access the “backend-service,” add the following annotations to the service file. + ```yaml - … + ##… annotations: "consul.hashicorp.com/connect-inject": "true" "consul.hashicorp.com/transparent-proxy": "false" "consul.hashicorp.com/connect-service-upstreams": "backend-service.svc.cluster-02.peer:1234" - … + ##… ``` + 1. Apply the service file to the first cluster. @@ -170,7 +181,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a $ curl localhost:1234 { “name”: “backend-service”, - … + ##… “body”: “Response from backend”, “code”: 200 } From 8687f1511e7c5427bb25a9fbd7a8fb2a178eb748 Mon Sep 17 00:00:00 2001 From: boruszak Date: Tue, 14 Jun 2022 16:58:07 -0500 Subject: [PATCH 026/149] fixes --- .../content/docs/connect/cluster-peering/k8s.mdx | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx index 03563997e..2796cda8b 100644 --- a/website/content/docs/connect/cluster-peering/k8s.mdx +++ b/website/content/docs/connect/cluster-peering/k8s.mdx @@ -41,6 +41,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. In “cluster-01,” create the `PeeringAcceptor` custom resource. + ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: PeeringAcceptor @@ -53,6 +54,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a key: "data" backend: "kubernetes" ``` + 1. Apply the `PeeringAcceptor` resource to the first cluster. @@ -76,6 +78,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. In “cluster-02,” create the `PeeringDialer` custom resource. + ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: PeeringDialer @@ -88,6 +91,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a key: "data" backend: "kubernetes" ``` + 1. Apply the `PeeringDialer` resource to the second cluster. @@ -101,6 +105,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. For the service in “cluster-02” that you want to export, add the following [annotations to your service files](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams). + ```yaml ##… annotations: @@ -108,11 +113,13 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a "consul.hashicorp.com/transparent-proxy": "false" ##… ``` + 1. In “cluster-02,” create an `ExportedServices` custom resource. + ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: ExportedServices @@ -125,9 +132,12 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a peerName: cluster-01 ## The name of the peer that receives the service ``` + + 1. Create service intentions for the second cluster. + ```yaml apiVersion: consul.hashicorp.com/v1alpha1 kind: ServiceIntentions @@ -142,6 +152,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a - name: frontend-service action: allow ``` + 1. Apply the service file, the `ExportedServices` resource, and the intentions to the second cluster. @@ -159,6 +170,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. For the services in “cluster-01” that you want to access the “backend-service,” add the following annotations to the service file. + ```yaml ##… annotations: @@ -167,6 +179,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a "consul.hashicorp.com/connect-service-upstreams": "backend-service.svc.cluster-02.peer:1234" ##… ``` + 1. Apply the service file to the first cluster. From b9a4ff73ce5cb3111b621588548a221a88be122a Mon Sep 17 00:00:00 2001 From: boruszak Date: Tue, 14 Jun 2022 17:01:48 -0500 Subject: [PATCH 027/149] Updated nav.json --- website/data/docs-nav-data.json | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 139508de9..1fe87cf30 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -331,6 +331,10 @@ { "title": "Create and Manage Peering Connections", "path": "connect/cluster-peering/create-manage-peering" + }, + { + "title": "Cluster Peering on Kubernetes", + "path": "connect/cluster-peering/k8s" } ] }, From 703ce412c792f8b4c6022d2823a1f9dd7e9cefcb Mon Sep 17 00:00:00 2001 From: boruszak Date: Tue, 14 Jun 2022 17:12:45 -0500 Subject: [PATCH 028/149] Removing k8s updates on this branch --- .../docs/connect/cluster-peering/k8s.mdx | 211 ------------------ website/data/docs-nav-data.json | 4 - 2 files changed, 215 deletions(-) delete mode 100644 website/content/docs/connect/cluster-peering/k8s.mdx diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx deleted file mode 100644 index 2796cda8b..000000000 --- a/website/content/docs/connect/cluster-peering/k8s.mdx +++ /dev/null @@ -1,211 +0,0 @@ ---- -layout: docs -page_title: Cluster Peering on Kubernetes -description: >- - This page describes how to create peering connections, deploy services, export cluster services, and end peering connections for Consul cluster peering using Kubernetes (K8s). ---- - -# Cluster Peering on Kubernetes - -~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. - -To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart and create custom resource definitions for each side of the peering. - -The following Custom Resource Definitions (CRDs) are used to create and manage a peering connection: - -- `PeeringAcceptor`: Generates a peering token and accepts an incoming peering connection. -- `PeeringDialer`: Uses a peering token to make an outbound peering connection with the cluster that generated the token. - -## Prerequisites - -To create and use cluster peering connections with Kubernetes, you need at least two Kubernetes clusters running in a flat network with Consul on Kubernetes v. 0.45 or later. - -### Helm chart configuration -To establish cluster peering through Kubernetes, deploy clusters with the following Helm values. - -```yaml -global: - peering: - enabled: true -connectInject: - enabled: true -meshGateway: - enabled: true - replicas: 1 -``` - -## Create a peering connection - -To peer Kubernetes clusters running Consul, you need to create a peering token and share it with the other cluster. - -1. In “cluster-01,” create the `PeeringAcceptor` custom resource. - - - - ```yaml - apiVersion: consul.hashicorp.com/v1alpha1 - kind: PeeringAcceptor - metadata: - name: cluster-02 ## The name of the peer you want to connect to - spec: - peer: - secret: - name: "peering-token" - key: "data" - backend: "kubernetes" - ``` - - - -1. Apply the `PeeringAcceptor` resource to the first cluster. - - ```shell-session - $ kubectl apply -f acceptor.yml - ```` - -1. Save your peering token so that you can export it to the other cluster. - - ```shell-session - $ kubectl get secret peering-token -o yaml > peering-token.yml - ``` - -1. Apply the peering token to the second cluster. - - ```shell-session - $ kubectl apply -f peering-token.yml - ``` - -1. In “cluster-02,” create the `PeeringDialer` custom resource. - - - - ```yaml - apiVersion: consul.hashicorp.com/v1alpha1 - kind: PeeringDialer - metadata: - name: cluster-01 ## The name of the peer you want to connect to - spec: - peer: - secret: - name: "peering-token" - key: "data" - backend: "kubernetes" - ``` - - - -1. Apply the `PeeringDialer` resource to the second cluster. - - ```shell-session - $ kubectl apply -f dialer.yml - ``` - -## Deploy and export cluster services - -1. For the service in “cluster-02” that you want to export, add the following [annotations to your service files](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams). - - - - ```yaml - ##… - annotations: - "consul.hashicorp.com/connect-inject": "true" - "consul.hashicorp.com/transparent-proxy": "false" - ##… - ``` - - - -1. In “cluster-02,” create an `ExportedServices` custom resource. - - - - ```yaml - apiVersion: consul.hashicorp.com/v1alpha1 - kind: ExportedServices - metadata: - name: default ## The name of the partition containing the service - spec: - services: - name: backend-service ## The name of the service you want to export - consumers: - peerName: cluster-01 ## The name of the peer that receives the service - ``` - - - -1. Create service intentions for the second cluster. - - - - ```yaml - apiVersion: consul.hashicorp.com/v1alpha1 - kind: ServiceIntentions - metadata: - name: backend-deny - spec: - destination: - name: backend-service - sources: - - name: "*" - action: deny - - name: frontend-service - action: allow - ``` - - - -1. Apply the service file, the `ExportedServices` resource, and the intentions to the second cluster. - - ```shell-session - $ kubectl apply -f backend-service.yml; kubectl apply -f exportedsvc.yml; kubectl apply -f intention.yml - ``` - -1. To confirm that you peered your clusters, in “cluster-01,” query the `/health` HTTP endpoint. - - ```shell-session - $ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02" - ``` - -1. For the services in “cluster-01” that you want to access the “backend-service,” add the following annotations to the service file. - - - - ```yaml - ##… - annotations: - "consul.hashicorp.com/connect-inject": "true" - "consul.hashicorp.com/transparent-proxy": "false" - "consul.hashicorp.com/connect-service-upstreams": "backend-service.svc.cluster-02.peer:1234" - ##… - ``` - - - -1. Apply the service file to the first cluster. - - ```shell-session - $ kubectl apply -f frontend-service.yml - ``` - -1. Run the following command and check the output to confirm that you peered your clusters successfully. - - ```shell-session - $ curl localhost:1234 - { - “name”: “backend-service”, - ##… - “body”: “Response from backend”, - “code”: 200 - } - ``` - -## End a peering connection - -To end a peering connection, delete both the `PeeringAcceptor` and `PeeringDialer` resources. - -To confirm that you deleted your peering connection, in “cluster-01,” query the `/health` HTTP endpoint. The peered services should no longer appear. - -```shell-session -$ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02" -``` diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 1fe87cf30..139508de9 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -331,10 +331,6 @@ { "title": "Create and Manage Peering Connections", "path": "connect/cluster-peering/create-manage-peering" - }, - { - "title": "Cluster Peering on Kubernetes", - "path": "connect/cluster-peering/k8s" } ] }, From 72b91c348114208e97b44c6ace91c039445959bd Mon Sep 17 00:00:00 2001 From: boruszak Date: Tue, 14 Jun 2022 17:14:34 -0500 Subject: [PATCH 029/149] Nav.json updates --- website/data/docs-nav-data.json | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 139508de9..1fe87cf30 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -331,6 +331,10 @@ { "title": "Create and Manage Peering Connections", "path": "connect/cluster-peering/create-manage-peering" + }, + { + "title": "Cluster Peering on Kubernetes", + "path": "connect/cluster-peering/k8s" } ] }, From e22171cdbf0c4bbaf0eaf20870eede0ce29ff2af Mon Sep 17 00:00:00 2001 From: boruszak Date: Tue, 14 Jun 2022 17:15:14 -0500 Subject: [PATCH 030/149] Cluster Peering on Kubernetes page creation --- .../docs/connect/cluster-peering/k8s.mdx | 211 ++++++++++++++++++ 1 file changed, 211 insertions(+) create mode 100644 website/content/docs/connect/cluster-peering/k8s.mdx diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx new file mode 100644 index 000000000..2796cda8b --- /dev/null +++ b/website/content/docs/connect/cluster-peering/k8s.mdx @@ -0,0 +1,211 @@ +--- +layout: docs +page_title: Cluster Peering on Kubernetes +description: >- + This page describes how to create peering connections, deploy services, export cluster services, and end peering connections for Consul cluster peering using Kubernetes (K8s). +--- + +# Cluster Peering on Kubernetes + +~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. + +To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart and create custom resource definitions for each side of the peering. + +The following Custom Resource Definitions (CRDs) are used to create and manage a peering connection: + +- `PeeringAcceptor`: Generates a peering token and accepts an incoming peering connection. +- `PeeringDialer`: Uses a peering token to make an outbound peering connection with the cluster that generated the token. + +## Prerequisites + +To create and use cluster peering connections with Kubernetes, you need at least two Kubernetes clusters running in a flat network with Consul on Kubernetes v. 0.45 or later. + +### Helm chart configuration +To establish cluster peering through Kubernetes, deploy clusters with the following Helm values. + +```yaml +global: + peering: + enabled: true +connectInject: + enabled: true +meshGateway: + enabled: true + replicas: 1 +``` + +## Create a peering connection + +To peer Kubernetes clusters running Consul, you need to create a peering token and share it with the other cluster. + +1. In “cluster-01,” create the `PeeringAcceptor` custom resource. + + + + ```yaml + apiVersion: consul.hashicorp.com/v1alpha1 + kind: PeeringAcceptor + metadata: + name: cluster-02 ## The name of the peer you want to connect to + spec: + peer: + secret: + name: "peering-token" + key: "data" + backend: "kubernetes" + ``` + + + +1. Apply the `PeeringAcceptor` resource to the first cluster. + + ```shell-session + $ kubectl apply -f acceptor.yml + ```` + +1. Save your peering token so that you can export it to the other cluster. + + ```shell-session + $ kubectl get secret peering-token -o yaml > peering-token.yml + ``` + +1. Apply the peering token to the second cluster. + + ```shell-session + $ kubectl apply -f peering-token.yml + ``` + +1. In “cluster-02,” create the `PeeringDialer` custom resource. + + + + ```yaml + apiVersion: consul.hashicorp.com/v1alpha1 + kind: PeeringDialer + metadata: + name: cluster-01 ## The name of the peer you want to connect to + spec: + peer: + secret: + name: "peering-token" + key: "data" + backend: "kubernetes" + ``` + + + +1. Apply the `PeeringDialer` resource to the second cluster. + + ```shell-session + $ kubectl apply -f dialer.yml + ``` + +## Deploy and export cluster services + +1. For the service in “cluster-02” that you want to export, add the following [annotations to your service files](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams). + + + + ```yaml + ##… + annotations: + "consul.hashicorp.com/connect-inject": "true" + "consul.hashicorp.com/transparent-proxy": "false" + ##… + ``` + + + +1. In “cluster-02,” create an `ExportedServices` custom resource. + + + + ```yaml + apiVersion: consul.hashicorp.com/v1alpha1 + kind: ExportedServices + metadata: + name: default ## The name of the partition containing the service + spec: + services: + name: backend-service ## The name of the service you want to export + consumers: + peerName: cluster-01 ## The name of the peer that receives the service + ``` + + + +1. Create service intentions for the second cluster. + + + + ```yaml + apiVersion: consul.hashicorp.com/v1alpha1 + kind: ServiceIntentions + metadata: + name: backend-deny + spec: + destination: + name: backend-service + sources: + - name: "*" + action: deny + - name: frontend-service + action: allow + ``` + + + +1. Apply the service file, the `ExportedServices` resource, and the intentions to the second cluster. + + ```shell-session + $ kubectl apply -f backend-service.yml; kubectl apply -f exportedsvc.yml; kubectl apply -f intention.yml + ``` + +1. To confirm that you peered your clusters, in “cluster-01,” query the `/health` HTTP endpoint. + + ```shell-session + $ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02" + ``` + +1. For the services in “cluster-01” that you want to access the “backend-service,” add the following annotations to the service file. + + + + ```yaml + ##… + annotations: + "consul.hashicorp.com/connect-inject": "true" + "consul.hashicorp.com/transparent-proxy": "false" + "consul.hashicorp.com/connect-service-upstreams": "backend-service.svc.cluster-02.peer:1234" + ##… + ``` + + + +1. Apply the service file to the first cluster. + + ```shell-session + $ kubectl apply -f frontend-service.yml + ``` + +1. Run the following command and check the output to confirm that you peered your clusters successfully. + + ```shell-session + $ curl localhost:1234 + { + “name”: “backend-service”, + ##… + “body”: “Response from backend”, + “code”: 200 + } + ``` + +## End a peering connection + +To end a peering connection, delete both the `PeeringAcceptor` and `PeeringDialer` resources. + +To confirm that you deleted your peering connection, in “cluster-01,” query the `/health` HTTP endpoint. The peered services should no longer appear. + +```shell-session +$ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02" +``` From 24409fa40b76c232d33930637a1e96f43d951862 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Tue, 14 Jun 2022 17:23:07 -0500 Subject: [PATCH 031/149] Update website/content/docs/connect/cluster-peering/index.mdx Co-authored-by: Tu Nguyen --- website/content/docs/connect/cluster-peering/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 324c95712..51f1f916a 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -18,7 +18,7 @@ Cluster peering allows Consul clusters in different datacenters to communicate w 1. Establish a connection between clusters 1. Make services available to other clusters -For detailed instructions on setting up cluster peering with the Consul CLI, refer to [Create and Manage Peering Connections](/docs/connect/cluster-peering/create-manage-peering). If you prefer to use Kubernetes, refer to [Cluster Peering on Kubernetes](/docs/connect/cluster-peering/k8s). +For detailed instructions on setting up cluster peering with the Consul CLI, refer to [Create and Manage Peering Connections](/docs/connect/cluster-peering/create-manage-peering). ### Differences between WAN federation and cluster peering From fe0a5491d209bd4049243d52903e93b08282d492 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Tue, 14 Jun 2022 17:27:03 -0500 Subject: [PATCH 032/149] Update website/content/docs/connect/cluster-peering/create-manage-peering.mdx Co-authored-by: Tu Nguyen --- .../docs/connect/cluster-peering/create-manage-peering.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index ba4269709..84771616a 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -52,7 +52,7 @@ After you establish a connection between the clusters, you need to create a conf First, create a configuration entry and specify the `Kind` as `“exported-services”`. - + ```shell-session Kind = "exported-services" From 9a8235993acca9a0d5212c191240094ff7d4b954 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Tue, 14 Jun 2022 17:28:06 -0500 Subject: [PATCH 033/149] Update website/content/docs/connect/cluster-peering/create-manage-peering.mdx --- .../docs/connect/cluster-peering/create-manage-peering.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 84771616a..42b27e05b 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -90,7 +90,7 @@ Before you can call services from peered clusters, you must set service intentio First, create a configuration entry and specify the `Kind` as `“service-intentions”`. Declare the service on "cluster-02" that can access the service in "cluster-01." The following example sets service intentions so that "backend-service" can access "frontend-service." - + ```shell-session Kind = "service-intentions" From 687c16b9e089c6955ea0b8bf8065e8cf3ccd83bf Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Tue, 14 Jun 2022 17:29:30 -0500 Subject: [PATCH 034/149] Update website/content/docs/connect/cluster-peering/create-manage-peering.mdx Co-authored-by: Freddy --- .../docs/connect/cluster-peering/create-manage-peering.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 42b27e05b..f98fd8952 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -9,7 +9,7 @@ description: >- ~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. -A peering token enables cluster peering between different datacenters. Once you generate a peering token, you can use it to initiate a connection between clusters. Then you can export services and authorize other clusters to call those services. +A peering token enables cluster peering between different datacenters. Once you generate a peering token, you can use it to establish a connection between clusters. Then you can export services and authorize other clusters to call those services. To peer clusters, you must complete the following steps in order: From caa2dc5bfbfd4d192966a7bd13969fdf17dd53c2 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Tue, 14 Jun 2022 17:34:21 -0500 Subject: [PATCH 035/149] Apply suggestions from code review Co-authored-by: Freddy --- .../cluster-peering/create-manage-peering.mdx | 21 +++++++++---------- 1 file changed, 10 insertions(+), 11 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index f98fd8952..b09f3d221 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -15,7 +15,7 @@ To peer clusters, you must complete the following steps in order: 1. Create a peering token 1. Establish a connection between clusters -1. Connect service endpoints +1. Export service endpoints 1. Authorize connections between peers ## Create a peering token @@ -24,31 +24,30 @@ You can generate peering tokens and initiate connections from either the server To begin the cluster peering process, generate a peering token in one of your clusters. The other cluster uses this token to establish the peering connection. -In “cluster-01,” run the `generate-token` command. +In “cluster-01,” issue a request for a peering token using the HTTP API. ```shell-session -$ consul peering generate-token -peer="cluster-02" +$ curl -X POST --data '{"PeerName":"cluster-02"}' http://localhost:8500/v1/peering/token ``` The CLI outputs the peering token, which is an encoded string of alphanumeric characters and symbols. -For the peering token to function correctly, you must enter the second cluster’s exact name in the `peer` parameter. ## Establish a connection between clusters -Next, use the peering token to establish a secure connection between the clusters. In the client agents of “cluster-02,” run the ``peering establish`` command. This command does not generate an output. +Next, use the peering token to establish a secure connection between the clusters. In the client agents of “cluster-02,” establish the peering using the HTTP API. This endpoint does not generate an output unless there is an error. ```shell-session $ consul peering establish -peer="cluster-01" -token="eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6IlNvbHIifQ.5T7L_L1MPfQ_5FjKGa1fTPqrzwK4bNSM812nW6oyjb8" ``` -In the peer parameter, specify the first cluster’s exact name. The `token` parameter should include the entire peering token created in the first cluster. +In the peer parameter, specify a name for the first cluster. The `PeeringToken` parameter should include the entire peering token created in the first cluster. -When you connect server agents through cluster peering, they will peer through their default partitions. To connect other partitions through server agents, you must specify the partitions you want to peer using the `partition` parameter. +When you connect server agents through cluster peering, they will peer their default partitions. To establish peerings for other partitions through server agents, you must specify the partitions you want to peer using the `Partition` field of the request body. ## Connect service endpoints -After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to make services and support queries for service mesh available to peered clusters. +After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to replicate service definitions and support service mesh connections across clusters. First, create a configuration entry and specify the `Kind` as `“exported-services”`. @@ -125,14 +124,14 @@ $ curl \ http://127.0.0.1:8500/v1/health/service/service-name?peer=cluster-01 ``` -Successful connections include `"Status": "passing"` in the output. +Successful connections include service definitions in the output. ## Remove peering connections After you create a peering connection between clusters in different datacenters, you can “unpeer” the connected clusters. Deleting a peering connection stops data replication to the peer and deletes imported data, including services and CA certificates. -In “cluster-01,” run the `peering delete` command. +In “cluster-01,” request the deletion via the HTTP API. ```shell-session -$ consul peering delete -peer="cluster-02" +$ curl -X DELETE http://127.0.0.1:8500/v1/peering/cluster-02 ``` From 1c7d51f9d93546518729ce6f6fa45b95f86508b4 Mon Sep 17 00:00:00 2001 From: boruszak Date: Wed, 15 Jun 2022 13:55:53 -0500 Subject: [PATCH 036/149] peering_token.json addition --- .../cluster-peering/create-manage-peering.mdx | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index b09f3d221..18bfc8a9f 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -32,6 +32,18 @@ $ curl -X POST --data '{"PeerName":"cluster-02"}' http://localhost:8500/v1/peeri The CLI outputs the peering token, which is an encoded string of alphanumeric characters and symbols. +Create a JSON file that contains the first cluster's name and the peering token. + + + +```json +{ + "PeerName": "cluster-01", + "PeeringToken": "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6IlNvbHIifQ.5T7L_L1MPfQ_5FjKGa1fTPqrzwK4bNSM812nW6oyjb8" +} +``` + + ## Establish a connection between clusters @@ -53,7 +65,7 @@ First, create a configuration entry and specify the `Kind` as `“exported-servi -```shell-session +```hcl Kind = "exported-services" Partition = "partition-name" @@ -91,14 +103,14 @@ First, create a configuration entry and specify the `Kind` as `“service-intent -```shell-session +```hcl Kind = "service-intentions" Name = "frontend-service" Partition = "partition-name" Sources = [ { - Name = "backend-service" + Name = "backend-service" Peer = "cluster-02" Action = "allow" } From 199e9a900a5ac23fa822fbf86cc10ffa974e71ce Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 13:56:55 -0500 Subject: [PATCH 037/149] Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Co-authored-by: Blake Covarrubias --- .../connect/cluster-peering/create-manage-peering.mdx | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 18bfc8a9f..5b1e43b6d 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -7,7 +7,7 @@ description: >- # Create and Manage Peering Connections -~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. +~> **Cluster peering is currently in technical preview:** Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support. A peering token enables cluster peering between different datacenters. Once you generate a peering token, you can use it to establish a connection between clusters. Then you can export services and authorize other clusters to call those services. @@ -20,17 +20,17 @@ To peer clusters, you must complete the following steps in order: ## Create a peering token -You can generate peering tokens and initiate connections from either the server agents or the client agents in your clusters. For the current release, we recommend you initiate peering through the client agents in the partitions you want to connect. +You can generate peering tokens and initiate connections using the Consul API on any available agent. However, we recommend performing these operations through a client agent in the partition you want to connect. To begin the cluster peering process, generate a peering token in one of your clusters. The other cluster uses this token to establish the peering connection. In “cluster-01,” issue a request for a peering token using the HTTP API. ```shell-session -$ curl -X POST --data '{"PeerName":"cluster-02"}' http://localhost:8500/v1/peering/token +$ curl --request POST --data '{"PeerName":"cluster-02"}' --url http://localhost:8500/v1/peering/token ``` -The CLI outputs the peering token, which is an encoded string of alphanumeric characters and symbols. +The CLI outputs the peering token, which is a base64-encoded string containing the token details. Create a JSON file that contains the first cluster's name and the peering token. @@ -67,7 +67,6 @@ First, create a configuration entry and specify the `Kind` as `“exported-servi ```hcl Kind = "exported-services" -Partition = "partition-name" Services = [ { From c23ab4259edd13b848c3553974092d83f15f44f1 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 14:01:34 -0500 Subject: [PATCH 038/149] Apply suggestions from code review Co-authored-by: Blake Covarrubias Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- .../connect/cluster-peering/create-manage-peering.mdx | 11 +++++------ .../content/docs/connect/cluster-peering/index.mdx | 2 +- 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 5b1e43b6d..9d5e794f7 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -59,7 +59,7 @@ When you connect server agents through cluster peering, they will peer their def ## Connect service endpoints -After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to replicate service definitions and support service mesh connections across clusters. +After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to advertise service information and support service mesh connections across clusters. First, create a configuration entry and specify the `Kind` as `“exported-services”`. @@ -105,7 +105,6 @@ First, create a configuration entry and specify the `Kind` as `“service-intent ```hcl Kind = "service-intentions" Name = "frontend-service" -Partition = "partition-name" Sources = [ { @@ -128,14 +127,14 @@ $ consul config write peering-intentions.hcl ## Check peered cluster status -To confirm that you peered your clusters, you can [query the `/health/service` endpoint](https://www.consul.io/api-docs/health) of one cluster from the other cluster. For example, in "cluster-02," query the endpoint and add the `peer=cluster-01` parameter to the end of the URL. +To confirm that you peered your clusters, you can [query the `/health/service` endpoint](/api-docs/health) of one cluster from the other cluster. For example, in "cluster-02," query the endpoint and add the `peer=cluster-01` query parameter to the end of the URL. ```shell-session $ curl \ - http://127.0.0.1:8500/v1/health/service/service-name?peer=cluster-01 + "http://127.0.0.1:8500/v1/health/service/service-name?peer=cluster-01" ``` -Successful connections include service definitions in the output. +A successful query will include service information in the output. ## Remove peering connections @@ -144,5 +143,5 @@ After you create a peering connection between clusters in different datacenters, In “cluster-01,” request the deletion via the HTTP API. ```shell-session -$ curl -X DELETE http://127.0.0.1:8500/v1/peering/cluster-02 +$ curl --request DELETE http://127.0.0.1:8500/v1/peering/cluster-02 ``` diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 51f1f916a..af93e05d0 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -7,7 +7,7 @@ description: >- # What is Cluster Peering? -~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. +->**Cluster peering is currently in technical preview**: Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support. Cluster peering is a Consul feature to allow service connectivity between two independent clusters. Similar to WAN Federation, you can use cluster peering to enable service connectivity between partitions in different datacenters. From e8d34bab688e8c0578aeb2cb9bf3a51cb7b1fa62 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 14:04:52 -0500 Subject: [PATCH 039/149] Apply suggestions from code review --- .../docs/connect/cluster-peering/create-manage-peering.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 9d5e794f7..8dcb1c962 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -50,7 +50,7 @@ Create a JSON file that contains the first cluster's name and the peering token. Next, use the peering token to establish a secure connection between the clusters. In the client agents of “cluster-02,” establish the peering using the HTTP API. This endpoint does not generate an output unless there is an error. ```shell-session -$ consul peering establish -peer="cluster-01" -token="eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiJhZG1pbiIsImF1ZCI6IlNvbHIifQ.5T7L_L1MPfQ_5FjKGa1fTPqrzwK4bNSM812nW6oyjb8" +$ curl --request POST --data @peering_token.json http://127.0.0.1:8500/v1/peering/establish ``` In the peer parameter, specify a name for the first cluster. The `PeeringToken` parameter should include the entire peering token created in the first cluster. @@ -92,7 +92,7 @@ Then, add the configuration entry to your cluster. $ consul config write peering-config.hcl ``` -Before you proceed, wait for the clusters to sync and make services available to their peers +Before you proceed, wait for the clusters to sync and make services available to their peers. You can issue an endpoint query to [check the peered cluster status](#check-peered-cluster-status). ## Authorize connections from peers From 80f779a528b664558ecf2774040996ec66d2175e Mon Sep 17 00:00:00 2001 From: boruszak Date: Wed, 15 Jun 2022 14:07:56 -0500 Subject: [PATCH 040/149] Switch fronend-service and backend-service --- .../docs/connect/cluster-peering/create-manage-peering.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 8dcb1c962..2438fbf57 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -98,17 +98,17 @@ Before you proceed, wait for the clusters to sync and make services available to Before you can call services from peered clusters, you must set service intentions that authorize those clusters to use specific services. Consul prevents services from being exported to unauthorized clusters. -First, create a configuration entry and specify the `Kind` as `“service-intentions”`. Declare the service on "cluster-02" that can access the service in "cluster-01." The following example sets service intentions so that "backend-service" can access "frontend-service." +First, create a configuration entry and specify the `Kind` as `“service-intentions”`. Declare the service on "cluster-02" that can access the service in "cluster-01." The following example sets service intentions so that "frontend-service" can access "backend-service." ```hcl Kind = "service-intentions" -Name = "frontend-service" +Name = "backend"-service" Sources = [ { - Name = "backend-service" + Name = "frontend-service" Peer = "cluster-02" Action = "allow" } From 37acf49357076402ecaf673387340d486d57fe81 Mon Sep 17 00:00:00 2001 From: boruszak Date: Wed, 15 Jun 2022 14:08:34 -0500 Subject: [PATCH 041/149] typo fix --- .../docs/connect/cluster-peering/create-manage-peering.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index 2438fbf57..db0d1bc6b 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -104,7 +104,7 @@ First, create a configuration entry and specify the `Kind` as `“service-intent ```hcl Kind = "service-intentions" -Name = "backend"-service" +Name = "backend-service" Sources = [ { From 08716c527956eca4edd5b6e4fe5b74ac39cf8093 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 14:19:03 -0500 Subject: [PATCH 042/149] Apply suggestions from code review Co-authored-by: Freddy --- website/content/docs/connect/cluster-peering/index.mdx | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index af93e05d0..935f25a74 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -9,7 +9,7 @@ description: >- ->**Cluster peering is currently in technical preview**: Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support. -Cluster peering is a Consul feature to allow service connectivity between two independent clusters. Similar to WAN Federation, you can use cluster peering to enable service connectivity between partitions in different datacenters. +You can create peering connections between two or more independent clusters so that services deployed to different partitions or datacenters can to communicate. ## Overview @@ -22,9 +22,9 @@ For detailed instructions on setting up cluster peering with the Consul CLI, ref ### Differences between WAN federation and cluster peering -WAN Federation and cluster peering are different ways to connect clusters across datacenters. The most important distinction is that WAN Federation assumes clusters are owned by the same operators, so it uses the gossip protocol to replicate global states like ACLs. As a result, WAN Federation requires a “primary datacenter" to serve as an authority for replicated data. +WAN Federation and cluster peering are different ways to connect clusters. The most important distinction is that WAN Federation assumes clusters are owned by the same operators, so it maintains and replicates global states like ACLs and Configuration Entries. As a result, WAN Federation requires a “primary datacenter" to serve as an authority for replicated data. -Regardless of whether you connect your clusters through WAN Federation or cluster peering, human and machine users can use either peering method to issue cross-datacenter queries for data, including service endpoints. +Regardless of whether you connect your clusters through WAN Federation or cluster peering, human and machine users can use either method to discover services in other clusters or dial them through the service mesh. | | WAN Federation | Cluster Peering | | :----------------------------------------------- | :------------: | :-------------: | From b00262381bc84d1ae55303f7280e8aabcdd9eef3 Mon Sep 17 00:00:00 2001 From: boruszak Date: Wed, 15 Jun 2022 14:21:58 -0500 Subject: [PATCH 043/149] Limitations -> Constraints --- website/content/docs/connect/cluster-peering/index.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 935f25a74..dec57a339 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -9,7 +9,7 @@ description: >- ->**Cluster peering is currently in technical preview**: Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support. -You can create peering connections between two or more independent clusters so that services deployed to different partitions or datacenters can to communicate. +You can create peering connections between two or more independent clusters so that services deployed to different partitions or datacenters can to communicate. ## Overview @@ -35,13 +35,13 @@ Regardless of whether you connect your clusters through WAN Federation or cluste | Shares key/value stores | ✅ | ❌ | | Uses gossip protocol | ✅ | ❌ | -## Technical preview limitations -Not all features and functionality are available in the technical preview release. In particular, be aware of the following limitations: +## Technical preview constraints +Not all features and functionality are available in the technical preview release. In particular, consider the following technical constraints: - Mesh gateways for _service to service traffic_ between clusters are available. However, mesh gateways for _server to server traffic_ are not available. - Services exported to peered clusters must not be configured as HTTP. - Support for dynamic routing such as splits, custom routes, or redirects is not available at this time. -- The ``consul intention`` CLI command is not supported. +- The `consul intention CLI` command is not supported. - [L7 permissions](/docs/connect/l7-traffic) are not supported. - Configuring service failover across peers is not supported. - Accessing key/value stores across peers is not supported. From 1a6eea4fc3b06933422b8c8f95b43f3e3fe51589 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 14:23:18 -0500 Subject: [PATCH 044/149] Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- website/content/docs/connect/cluster-peering/index.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index dec57a339..75847bc9a 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -40,10 +40,10 @@ Not all features and functionality are available in the technical preview releas - Mesh gateways for _service to service traffic_ between clusters are available. However, mesh gateways for _server to server traffic_ are not available. - Services exported to peered clusters must not be configured as HTTP. -- Support for dynamic routing such as splits, custom routes, or redirects is not available at this time. +- Support for dynamic routing such as splits, custom routes, or redirects is not available. - The `consul intention CLI` command is not supported. - [L7 permissions](/docs/connect/l7-traffic) are not supported. - Configuring service failover across peers is not supported. - Accessing key/value stores across peers is not supported. - Consul datacenters that are already federated stay federated. -- Non-enterprise Consul instances cannot sync services with namespaces outside of the default namespace. +- Non-enterprise Consul instances cannot sync services with namespaces outside of the `default` namespace. From 627173110b808b4e7388e52f0fde6beebe0e027f Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 14:26:40 -0500 Subject: [PATCH 045/149] Update website/content/docs/connect/cluster-peering/index.mdx --- website/content/docs/connect/cluster-peering/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 75847bc9a..1f5fd0053 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -41,7 +41,7 @@ Not all features and functionality are available in the technical preview releas - Mesh gateways for _service to service traffic_ between clusters are available. However, mesh gateways for _server to server traffic_ are not available. - Services exported to peered clusters must not be configured as HTTP. - Support for dynamic routing such as splits, custom routes, or redirects is not available. -- The `consul intention CLI` command is not supported. +- The `consul intention CLI` command is not supported. To manage intentions that specify services in peered clusters, use [configuration entries](/docs/connect/config-entries/service-intentions). - [L7 permissions](/docs/connect/l7-traffic) are not supported. - Configuring service failover across peers is not supported. - Accessing key/value stores across peers is not supported. From 3b85a9dda20d8e245515aa185205d767d499d3fb Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 14:26:54 -0500 Subject: [PATCH 046/149] Update website/data/docs-nav-data.json Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- website/data/docs-nav-data.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 139508de9..b5d441515 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -325,7 +325,7 @@ "title": "Cluster Peering", "routes": [ { - "title": "Overview", + "title": "What is Cluster Peering", "path": "connect/cluster-peering" }, { From 521e0403561a5f0dcec60967e28143f960dc9650 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Wed, 15 Jun 2022 15:46:46 -0400 Subject: [PATCH 047/149] Add note about expected status for invalid CertificateRef --- website/content/docs/api-gateway/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/index.mdx b/website/content/docs/api-gateway/index.mdx index d00e54bf7..a5bfd9dc8 100644 --- a/website/content/docs/api-gateway/index.mdx +++ b/website/content/docs/api-gateway/index.mdx @@ -38,7 +38,7 @@ are used, see the [documentation in our GitHub repo](https://github.com/hashicor | [`Gateway`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.Gateway) |
  • Supported protocols: `HTTP`, `HTTPS`, `TCP`
  • Header-based hostname matching (no SNI support)
  • Supported filters: header addition, removal, and setting
  • TLS modes supported: `terminate`
  • Certificate types supported: `core/v1/Secret`
  • Extended options: TLS version and cipher constraints
| | [`HTTPRoute`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.HTTPRoute) |
  • Weight-based load balancing
  • Supported rules: path, header, query, and method-based matching
  • Supported filters: header addition, removal, and setting
  • Supported backend types:
    1. `core/v1/Service` (must map to a registered Consul service)
    2. `api-gateway.consul.hashicorp.com/v1alpha1/MeshService`
| | [`TCPRoute`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.TCPRoute) |
  • Supported backend types:
    1. `core/v1/Service` (must map to a registered Consul service)
    2. `api-gateway.consul.hashicorp.com/v1alpha1/MeshService`
| -| [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) |
  • Required to allow any reference from a `Gateway` to a Kubernetes `core/v1/Secret` in a different namespace.
  • Required to allow any reference from an `HTTPRoute` or `TCPRoute` to a Kubernetes `core/v1/Service` in a different namespace.
    • A route with an unpermitted BackendRef caused by the lack of a ReferencePolicy sets a `ResolvedRefs` status to `False` with the reason `RefNotPermitted`. The gateway listener rejects routes with an unpermitted BackendRef.
    • WARNING: If a route BackendRef becomes unpermitted, the entire route is removed from the gateway listener.
      • A BackendRef can become unpermitted when you delete a Reference Policy or add a new unpermitted BackendRef to an existing route.
| +| [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) |
  • Required to allow any reference from a `Gateway` to a Kubernetes `core/v1/Secret` in a different namespace.
    • A Gateway with an unpermitted CertificateRef caused by the lack of a ReferencePolicy sets a `ResolvedRefs` status to `False` with the reason `InvalidCertificateRef`. The Gateway will not become ready in this case.
  • Required to allow any reference from an `HTTPRoute` or `TCPRoute` to a Kubernetes `core/v1/Service` in a different namespace.
    • A route with an unpermitted BackendRef caused by the lack of a ReferencePolicy sets a `ResolvedRefs` status to `False` with the reason `RefNotPermitted`. The gateway listener rejects routes with an unpermitted BackendRef.
    • WARNING: If a route BackendRef becomes unpermitted, the entire route is removed from the gateway listener.
      • A BackendRef can become unpermitted when you delete a Reference Policy or add a new unpermitted BackendRef to an existing route.
| ## Additional Resources From 082bc3632b10fa33f23cb1e83599f432bcd95e1f Mon Sep 17 00:00:00 2001 From: Freddy Date: Wed, 15 Jun 2022 14:18:14 -0600 Subject: [PATCH 048/149] Add peering endpoint API docs (#13454) --- website/content/api-docs/peering.mdx | 322 +++++++++++++++++++++++++++ website/data/api-docs-nav-data.json | 4 + 2 files changed, 326 insertions(+) create mode 100644 website/content/api-docs/peering.mdx diff --git a/website/content/api-docs/peering.mdx b/website/content/api-docs/peering.mdx new file mode 100644 index 000000000..fa396015f --- /dev/null +++ b/website/content/api-docs/peering.mdx @@ -0,0 +1,322 @@ +--- +layout: api +page_title: Cluster Peering - HTTP API +description: The /peering endpoints allow for managing cluster peerings. +--- + +# Cluster Peering - HTTP API + +~> **Cluster peering is currently in technical preview:** Functionality associated +with cluster peering is subject to change. You should never use the technical +preview release in secure environments or production scenarios. Features in +technical preview may have performance issues, scaling issues, and limited support. + +The functionality described here is available only in +[Consul](https://www.hashicorp.com/products/consul/) version 1.13.0 and later. + +## Generate a Peering Token + +This endpoint generates a peering token. + +| Method | Path | Produces | +| ------- | -------------------- | ------------------ | +| `POST` | `/peering/token` | `application/json` | + +The table below shows this endpoint's support for +[blocking queries](/api-docs/features/blocking), +[consistency modes](/api-docs/features/consistency), +[agent caching](/api-docs/features/caching), and +[required ACLs](/api#authentication). + +| Blocking Queries | Consistency Modes | Agent Caching | ACL Required | +| ---------------- | ----------------- | ------------- | ---------------- | +| `NO` | `none` | `none` | `none` | + +### JSON Request Body Schema + +- `PeerName` `(string: )` - The name assigned to the peer cluster. + The `PeerName` is used to reference the peer cluster in service discovery queries + and configuration entries such as `service-intentions`. This field must be a + valid DNS hostname label. + +- `Partition` `(string: "")` - The admin partition that the + peering token is generated from. Uses `default` when not specified. + +- `Datacenter` `(string: "")` - Specifies the datacenter. Defaults to the + agent's datacenter when not specified. + +- `Token` `(string: "")` - Specifies the ACL token to use in the request. Takes precedence + over the token specified in the `token` query parameter, `X-Consul-Token` request header, + and `CONSUL_HTTP_TOKEN` environment variable. + +- `Meta` `(map: )` - Specifies KV metadata to associate with + the peering. This parameter is not required and does not directly impact the cluster + peering process. + +### Sample Payload + +```json +{ + "PeerName": "cluster-02", + "Meta": { + "env": "production" + } +} +``` + +### Sample Request + +```shell-session +$ curl --request POST \ + --header "X-Consul-Token: 5cdcae6c-0cce-4210-86fe-5dff3b984a6e" \ + --data @payload.json \ + http://127.0.0.1:8500/v1/peering/token +``` + +### Sample Response + +```json +{ + "PeeringToken": "eyJDQSI6bnVsbCwiU2V..." +} +``` + +## Establish a Peering + +This endpoint establishes a peering with a given peering token. + +| Method | Path | Produces | +| ------- | -------------------- | ------------------ | +| `POST` | `/peering/establish` | `application/json` | + +This endpoint returns no data. Success or failure is indicated by the status +code returned. + +The table below shows this endpoint's support for +[blocking queries](/api-docs/features/blocking), +[consistency modes](/api-docs/features/consistency), +[agent caching](/api-docs/features/caching), and +[required ACLs](/api#authentication). + +| Blocking Queries | Consistency Modes | Agent Caching | ACL Required | +| ---------------- | ----------------- | ------------- | ---------------- | +| `NO` | `none` | `none` | `none` | + +### JSON Request Body Schema + +- `PeerName` `(string: )` - The name assigned to the peer cluster. + The `PeerName` is used to reference the peer cluster in service discovery queries + and configuration entries such as `service-intentions`. This field must be a + valid DNS hostname label. + +- `Partition` `(string: "")` - The admin partition + that peers to the cluster that generated the peering token. Uses `default` + when not specified. + +- `PeeringToken` `(string: )` - The peering token fetched from the + peer cluster. + +- `Datacenter` `(string: "")` - Specifies the datacenter. Defaults to the + agent's datacenter when not specified. + +- `Token` `(string: "")` - Specifies the ACL token to use in the request. Takes precedence + over the token specified in the `token` query parameter, `X-Consul-Token` request header, + and `CONSUL_HTTP_TOKEN` environment variable. + +- `Meta` `(map: )` - Specifies KV metadata to associate with + the peering. This parameter is not required and does not directly impact the cluster + peering process. + +### Sample Payload + +```json +{ + "PeerName": "cluster-01", + "PeeringToken": "eyJDQSI6bnVsbCwiU2V...", + "Meta": { + "env": "production" + } +} +``` + +### Sample Request + +```shell-session +$ curl --request POST \ + --header "X-Consul-Token: 5cdcae6c-0cce-4210-86fe-5dff3b984a6e" \ + --data @payload.json \ + http://127.0.0.1:8500/v1/peering/establish +``` + +### Sample Response + +```json +{} +``` + +## Read a Peering + +This endpoint reads a peering with the given name. + +| Method | Path | Produces | +| ------ | ------------------ | ------------------ | +| `GET` | `/peering/:name` | `application/json` | + +The table below shows this endpoint's support for +[blocking queries](/api-docs/features/blocking), +[consistency modes](/api-docs/features/consistency), +[agent caching](/api-docs/features/caching), and +[required ACLs](/api#authentication). + +| Blocking Queries | Consistency Modes | Agent Caching | ACL Required | +| ---------------- | ----------------- | ------------- | ------------ | +| `NO` | `consistent` | `none` | `none` | + +### Path Parameters + +- `name` `(string: )` - Specifies the peering to read. + +### Query Parameters + +- `partition` `(string: "")` - Specifies the partition of the peering + to read. If not specified will default to `default`. + +### Sample Request + +```shell-session +$ curl --header "X-Consul-Token: b23b3cad-5ea1-4413-919e-c76884b9ad60" \ + http://127.0.0.1:8500/v1/peering/cluster-02 +``` + +### Sample Response + +```json +{ + "ID": "462c45e8-018e-f19d-85eb-1fc1bcc2ef12", + "Name": "cluster-02", + "State": "INITIAL", + "PeerID": "e83a315c-027e-bcb1-7c0c-a46650904a05", + "PeerServerName": "server.dc1.consul", + "PeerServerAddresses": [ + "10.0.0.1:8300" + ], + "CreateIndex": 89, + "ModifyIndex": 89 +} +``` + +## Delete a Peering + +This endpoint marks a peering for deletion. Once marked, Consul will +delete all the data imported from the peer in the background. Only once +all associated data has been deleted will the peering actually disappear. +Until then, further reads can be performed on the peering and a `DeletedAt` +field will be populated with the timestamp of when the peering was +marked for deletion. + +| Method | Path | Produces | +| -------- | ------------------ | -------- | +| `DELETE` | `/peering/:name` | N/A | + +This endpoint returns no data. Success or failure is indicated by the status +code returned. + +The table below shows this endpoint's support for +[blocking queries](/api-docs/features/blocking), +[consistency modes](/api-docs/features/consistency), +[agent caching](/api-docs/features/caching), and +[required ACLs](/api#authentication). + +| Blocking Queries | Consistency Modes | Agent Caching | ACL Required | +| ---------------- | ----------------- | ------------- | ------------- | +| `NO` | `none` | `none` | `none` | + +### Path Parameters + +- `name` `(string: )` - Specifies the name of the peering to delete. +- `partition` `(string: "")` - Specifies the partition of the peering + to delete. Uses `default` when not specified. + +### Sample Request + +```shell-session +$ curl --request DELETE \ + --header "X-Consul-Token: b23b3cad-5ea1-4413-919e-c76884b9ad60" \ + http://127.0.0.1:8500/v1/peering/cluster-02 +``` + +### Sample Read Output After Deletion Prior to Removal + +```json +{ + "ID": "462c45e8-018e-f19d-85eb-1fc1bcc2ef12", + "Name": "cluster-02", + "State": "TERMINATED", + "PeerID": "e83a315c-027e-bcb1-7c0c-a46650904a05", + "PeerServerName": "server.dc1.consul", + "PeerServerAddresses": [ + "10.0.0.1:8300" + ], + "DeletedAt": "2022-12-14T23:00:00Z", + "CreateIndex": 89, + "ModifyIndex": 89 +} +``` + +## List all Peerings + +This endpoint lists all the peerings. + +@include 'http_api_results_filtered_by_acls.mdx' + +| Method | Path | Produces | +| ------ | ------------- | ------------------ | +| `GET` | `/peerings` | `application/json` | + +The table below shows this endpoint's support for +[blocking queries](/api-docs/features/blocking), +[consistency modes](/api-docs/features/consistency), +[agent caching](/api-docs/features/caching), and +[required ACLs](/api#authentication). + +| Blocking Queries | Consistency Modes | Agent Caching | ACL Required | +| ---------------- | ----------------- | ------------- | ------------- | +| `NO` | `consistent` | `none` | `none` | + +### Sample Request + +```shell-session +$ curl --header "X-Consul-Token: 0137db51-5895-4c25-b6cd-d9ed992f4a52" \ + http://127.0.0.1:8500/v1/peerings +``` + +### Sample Response + +```json +[ + { + "ID": "462c45e8-018e-f19d-85eb-1fc1bcc2ef12", + "Name": "cluster-02", + "State": "ACTIVE", + "Partition": "default", + "PeerID": "e83a315c-027e-bcb1-7c0c-a46650904a05", + "PeerServerName": "server.dc1.consul", + "PeerServerAddresses": [ + "10.0.0.1:8300" + ], + "CreateIndex": 89, + "ModifyIndex": 89 + }, + { + "ID": "1460ada9-26d2-f30d-3359-2968aa7dc47d", + "Name": "cluster-03", + "State": "INITIAL", + "Partition": "default", + "Meta": { + "env": "production" + }, + "CreateIndex": 109, + "ModifyIndex": 119 + }, +] +``` diff --git a/website/data/api-docs-nav-data.json b/website/data/api-docs-nav-data.json index ee1df87c2..ab494f91b 100644 --- a/website/data/api-docs-nav-data.json +++ b/website/data/api-docs-nav-data.json @@ -89,6 +89,10 @@ "title": "Catalog", "path": "catalog" }, + { + "title": "Cluster Peering", + "path": "peering" + }, { "title": "Config", "path": "config" From 8bd57c75feb2e5bf7d457db08aa5d465335915dd Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 16:15:03 -0500 Subject: [PATCH 049/149] Apply suggestions from code review Co-authored-by: Nitya Dhanushkodi Co-authored-by: Blake Covarrubias --- .../content/docs/connect/cluster-peering/k8s.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx index 2796cda8b..8943ebb25 100644 --- a/website/content/docs/connect/cluster-peering/k8s.mdx +++ b/website/content/docs/connect/cluster-peering/k8s.mdx @@ -18,7 +18,7 @@ The following Custom Resource Definitions (CRDs) are used to create and manage a ## Prerequisites -To create and use cluster peering connections with Kubernetes, you need at least two Kubernetes clusters running in a flat network with Consul on Kubernetes v. 0.45 or later. +To create and use cluster peering connections with Kubernetes, you need at least two Kubernetes clusters running in a flat network with Consul on Kubernetes v.0.45 or later. ### Helm chart configuration To establish cluster peering through Kubernetes, deploy clusters with the following Helm values. @@ -60,19 +60,19 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. Apply the `PeeringAcceptor` resource to the first cluster. ```shell-session - $ kubectl apply -f acceptor.yml + $ kubectl apply --filename acceptor.yml ```` 1. Save your peering token so that you can export it to the other cluster. ```shell-session - $ kubectl get secret peering-token -o yaml > peering-token.yml + $ kubectl get secret peering-token --output yaml > peering-token.yml ``` 1. Apply the peering token to the second cluster. ```shell-session - $ kubectl apply -f peering-token.yml + $ kubectl apply --filename peering-token.yml ``` 1. In “cluster-02,” create the `PeeringDialer` custom resource. @@ -97,12 +97,12 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. Apply the `PeeringDialer` resource to the second cluster. ```shell-session - $ kubectl apply -f dialer.yml + $ kubectl apply --filename dialer.yml ``` ## Deploy and export cluster services -1. For the service in “cluster-02” that you want to export, add the following [annotations to your service files](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams). +1. For the service in “cluster-02” that you want to export, add the following [annotations](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams) to your service's pods. This service is referred to as "backend-service" in the following steps. @@ -158,7 +158,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. Apply the service file, the `ExportedServices` resource, and the intentions to the second cluster. ```shell-session - $ kubectl apply -f backend-service.yml; kubectl apply -f exportedsvc.yml; kubectl apply -f intention.yml + $ kubectl apply --filename backend-service.yml --filename exportedsvc.yml --filename intention.yml ``` 1. To confirm that you peered your clusters, in “cluster-01,” query the `/health` HTTP endpoint. @@ -185,7 +185,7 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a 1. Apply the service file to the first cluster. ```shell-session - $ kubectl apply -f frontend-service.yml + $ kubectl apply --filename frontend-service.yml ``` 1. Run the following command and check the output to confirm that you peered your clusters successfully. From 93a50d5b126c0fce3db5ac3a9403cb5bd4ad4e27 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Wed, 15 Jun 2022 16:25:57 -0500 Subject: [PATCH 050/149] Additional consistency edits --- website/content/docs/connect/cluster-peering/k8s.mdx | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/k8s.mdx b/website/content/docs/connect/cluster-peering/k8s.mdx index 8943ebb25..7b32d1743 100644 --- a/website/content/docs/connect/cluster-peering/k8s.mdx +++ b/website/content/docs/connect/cluster-peering/k8s.mdx @@ -7,7 +7,10 @@ description: >- # Cluster Peering on Kubernetes -~> This page covers features that are currently in _technical preview_. Features and functionality are subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may face performance and scaling issues, with limited support options available. +~> **Cluster peering is currently in technical preview:** Functionality associated +with cluster peering is subject to change. You should never use the technical +preview release in secure environments or production scenarios. Features in +technical preview may have performance issues, scaling issues, and limited support. To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart and create custom resource definitions for each side of the peering. @@ -21,6 +24,7 @@ The following Custom Resource Definitions (CRDs) are used to create and manage a To create and use cluster peering connections with Kubernetes, you need at least two Kubernetes clusters running in a flat network with Consul on Kubernetes v.0.45 or later. ### Helm chart configuration + To establish cluster peering through Kubernetes, deploy clusters with the following Helm values. ```yaml From 4b0ffb227a7e9a84559e30934ea2d5fff30ae9f0 Mon Sep 17 00:00:00 2001 From: "Chris S. Kim" Date: Thu, 16 Jun 2022 09:30:05 -0400 Subject: [PATCH 051/149] Update docs with Source.Peer field (#13463) --- .../config-entries/service-intentions.mdx | 30 ++++++++++++------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/website/content/docs/connect/config-entries/service-intentions.mdx b/website/content/docs/connect/config-entries/service-intentions.mdx index b73eb61fb..c6dd14de9 100644 --- a/website/content/docs/connect/config-entries/service-intentions.mdx +++ b/website/content/docs/connect/config-entries/service-intentions.mdx @@ -446,27 +446,37 @@ spec: }, }, { - name: 'Namespace', - enterprise: true, - type: 'string', + name: 'Peer', + type: 'string: ""', description: { hcl: - "The namespace of the source service. Defaults to the namespace of the destination service (i.e. the config entry's namespace)", + "Specifies the [peer](/docs/connect/cluster-peering/index.mdx) of the source service. `Peer` is mutually exclusive with `Partition`.", yaml: - 'The namespace of the source service. Defaults to the namespace of the destination service (i.e. `spec.destination.namespace`)', + "Specifies the [peer](/docs/connect/cluster-peering/index.mdx) of the source service. `peer` is mutually exclusive with `partition`.", + }, + }, + { + name: 'Namespace', + enterprise: true, + type: 'string: ""', + description: { + hcl: + "The namespace of the source service. If `Peer` is empty, `Namespace` defaults to the namespace of the destination service (i.e. the config entry's namespace).", + yaml: + 'The namespace of the source service. If `peer` is empty, `namespace` defaults to the namespace of the destination service (i.e. `spec.destination.namespace`).', }, }, { name: 'Partition', enterprise: true, - type: 'string', + type: 'string: ""', description: { hcl: - "Specifies the admin partition of the source service. Defaults to the destination service's partition, i.e., the configuration entry's partition", + "Specifies the admin partition of the source service. If `Peer` is empty, `Partition` defaults to the destination service's partition (i.e. the configuration entry's partition). `Partition` is mutually exclusive with `Peer`.", yaml: - "Specifies the admin partition of the source service. Defaults to the destination service's partition, i.e. `spec.destination.partition`", + "Specifies the admin partition of the source service. If `peer` is empty, `partition` defaults to the destination service's partition (i.e. `spec.destination.partition`). `partition` is mutually exclusive with `peer`.", }, - }, + }, { name: 'Action', type: 'string: ""', @@ -501,7 +511,7 @@ spec: intention behavior is defined by the default [ACL policy](/docs/agent/config/config-files#acl_default_policy).

This should be omitted for an L4 intention as it is mutually exclusive with the \`action\` field.

- Setting \`permissions\` is not valid if a wildcard is used for the \`spec.destination.name\` or \`spec.destination.namespace\` + Setting \`permissions\` is not valid if a wildcard is used for the \`spec.destination.name\` or \`spec.destination.namespace\` because they can only be applied to services with a compatible protocol.`, }, }, From 42f7d6a40326b3f5da9eadc68a610721734c59f2 Mon Sep 17 00:00:00 2001 From: "Chris S. Kim" Date: Thu, 16 Jun 2022 16:25:44 -0400 Subject: [PATCH 052/149] Update docs with peer query parameter (#13462) Co-authored-by: Tu Nguyen --- website/content/api-docs/health.mdx | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/website/content/api-docs/health.mdx b/website/content/api-docs/health.mdx index fcda3cbca..898c8ffe4 100644 --- a/website/content/api-docs/health.mdx +++ b/website/content/api-docs/health.mdx @@ -235,7 +235,7 @@ The table below shows this endpoint's support for - `near` `(string: "")` - Specifies a node name to sort the node list in ascending order based on the estimated round trip time from that node. Passing `?near=_agent` uses the agent's node for the sort. - **Note** that using `near` will ignore + ~> **Note:** Using `near` will ignore [`use_streaming_backend`](/docs/agent/config/config-files#use_streaming_backend) and always use blocking queries, because the data required to sort the results is not available to the streaming backend. @@ -259,6 +259,8 @@ The table below shows this endpoint's support for - `filter` `(string: "")` - Specifies the expression used to filter the queries results prior to returning the data. +- `peer` `(string: "")` - Specifies the imported service's peer. Applies only to imported services. + - `ns` `(string: "")` - Specifies the namespace of the service. You can also [specify the namespace through other methods](#methods-to-specify-namespace). @@ -431,8 +433,8 @@ gateway](/docs/connect/gateways/ingress-gateway) for a service in a given datace Parameters and response format are the same as [`/health/service/:service`](/api-docs/health#list-nodes-for-service). -**Note** that unlike `/health/connect/:service` and `/health/service/:service` this -endpoint does not support the [streaming backend](/api-docs/features/blocking#streaming-backend). +~> **Note:** Unlike `/health/connect/:service` and `/health/service/:service` this +endpoint does not support the `peer` query parameter and the [streaming backend](/api-docs/features/blocking#streaming-backend). ## List Checks in State From 1c610cb01d1d909f3e652a83af88e6e0f936f7af Mon Sep 17 00:00:00 2001 From: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Date: Thu, 16 Jun 2022 15:36:27 -0500 Subject: [PATCH 053/149] Update upgrade-specific-versions.mdx --- .../content/docs/api-gateway/upgrade-specific-versions.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 81901a1cf..a7312bde3 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -38,7 +38,7 @@ Ensure that the following requirements are met prior to upgrading: You should receive the following response: ```log - "hashicorp/consul-api-gateway:0.1.0" + "hashicorp/consul-api-gateway:0.2.1" ``` 1. Retrieve all gateways that have a secret in a different namespace. If you have installed the [`jq`](https://stedolan.github.io/jq/) utility, you can skip to [step 4](#jq-command-secrets). Otherwise, issue the following command to get all `Gateways` across all namespaces: @@ -430,7 +430,7 @@ This is the upgrade path to use when there are no version specific steps to take 1. Issue the following command to upgrade your Consul installation: ```shell-session - $ helm upgrade --values values.yaml --namespace consul --version hashicorp/consul + $ helm upgrade --values values.yaml --namespace consul --version hashicorp/consul ``` Note that the upgrade will cause the Consul API Gateway controller shut down and restart with the new version. From 7c5ef2aa3fd114853de16d9794da3b71816592a8 Mon Sep 17 00:00:00 2001 From: Kyle Havlovitz Date: Thu, 16 Jun 2022 17:18:37 -0700 Subject: [PATCH 054/149] command: Add TLS support for envoy prometheus endpoint --- command/connect/envoy/bootstrap_config.go | 65 +++- .../connect/envoy/bootstrap_config_test.go | 182 ++++++++++ command/connect/envoy/bootstrap_tpl.go | 15 + command/connect/envoy/envoy.go | 30 ++ command/connect/envoy/envoy_test.go | 66 ++++ .../prometheus-metrics-tls-ca-file.golden | 320 ++++++++++++++++++ .../prometheus-metrics-tls-ca-path.golden | 320 ++++++++++++++++++ 7 files changed, 997 insertions(+), 1 deletion(-) create mode 100644 command/connect/envoy/testdata/prometheus-metrics-tls-ca-file.golden create mode 100644 command/connect/envoy/testdata/prometheus-metrics-tls-ca-path.golden diff --git a/command/connect/envoy/bootstrap_config.go b/command/connect/envoy/bootstrap_config.go index 9c94eaac7..f2ea33b7f 100644 --- a/command/connect/envoy/bootstrap_config.go +++ b/command/connect/envoy/bootstrap_config.go @@ -637,6 +637,31 @@ func (c *BootstrapConfig) generateListenerConfig(args *BootstrapTplArgs, bindAdd ] } }` + + // Enable TLS on the prometheus listener if cert/private key are provided. + var tlsConfig string + if args.PrometheusCertFile != "" { + tlsConfig = `, + "transportSocket": { + "name": "tls", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext", + "commonTlsContext": { + "tlsCertificateSdsSecretConfigs": [ + { + "name": "prometheus_cert" + } + ], + "validationContextSdsSecretConfig": { + "trustedCa": { + "name": "prometheus_validation_context" + } + } + } + } + }` + } + listenerJSON := `{ "name": "` + name + `_listener", "address": { @@ -694,11 +719,43 @@ func (c *BootstrapConfig) generateListenerConfig(args *BootstrapTplArgs, bindAdd ] } } - ] + ]` + tlsConfig + ` } ] }` + secretsTemplate := `{ + "name": "prometheus_cert", + "tlsCertificate": { + "certificateChain": { + "filename": "%s" + }, + "privateKey": { + "filename": "%s" + } + } + }, + { + "name": "prometheus_validation_context", + "validationContext": { + %s + } + }` + var validationContext string + if args.PrometheusCAPath != "" { + validationContext = fmt.Sprintf(`"watchedDirectory": { + "path": "%s" + }`, args.PrometheusCAPath) + } else { + validationContext = fmt.Sprintf(`"trustedCa": { + "filename": "%s" + }`, args.PrometheusCAFile) + } + var secretsJSON string + if args.PrometheusCertFile != "" { + secretsJSON = fmt.Sprintf(secretsTemplate, args.PrometheusCertFile, args.PrometheusKeyFile, validationContext) + } + // Make sure we do not append the same cluster multiple times, as that will // cause envoy startup to fail. selfAdminClusterExists, err := containsSelfAdminCluster(args.StaticClustersJSON) @@ -716,6 +773,12 @@ func (c *BootstrapConfig) generateListenerConfig(args *BootstrapTplArgs, bindAdd listenerJSON = ",\n" + listenerJSON } args.StaticListenersJSON += listenerJSON + + if args.StaticSecretsJSON != "" { + secretsJSON = ",\n" + secretsJSON + } + args.StaticSecretsJSON += secretsJSON + return nil } diff --git a/command/connect/envoy/bootstrap_config_test.go b/command/connect/envoy/bootstrap_config_test.go index f02ae04c5..bb471566c 100644 --- a/command/connect/envoy/bootstrap_config_test.go +++ b/command/connect/envoy/bootstrap_config_test.go @@ -273,6 +273,126 @@ const ( } ] }` + expectedPromListenerWithBackendAndTLS = `{ + "name": "envoy_prometheus_metrics_listener", + "address": { + "socket_address": { + "address": "0.0.0.0", + "port_value": 9000 + } + }, + "filter_chains": [ + { + "filters": [ + { + "name": "envoy.filters.network.http_connection_manager", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager", + "stat_prefix": "envoy_prometheus_metrics", + "codec_type": "HTTP1", + "route_config": { + "name": "self_admin_route", + "virtual_hosts": [ + { + "name": "self_admin", + "domains": [ + "*" + ], + "routes": [ + { + "match": { + "path": "/metrics" + }, + "route": { + "cluster": "prometheus_backend", + "prefix_rewrite": "/stats/prometheus" + } + }, + { + "match": { + "prefix": "/" + }, + "direct_response": { + "status": 404 + } + } + ] + } + ] + }, + "http_filters": [ + { + "name": "envoy.filters.http.router", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router" + } + } + ] + } + } + ], + "transportSocket": { + "name": "tls", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext", + "commonTlsContext": { + "tlsCertificateSdsSecretConfigs": [ + { + "name": "prometheus_cert" + } + ], + "validationContextSdsSecretConfig": { + "trustedCa": { + "name": "prometheus_validation_context" + } + } + } + } + } + } + ] + }` + + expectedPromSecretsWithBackendAndTLS = `{ + "name": "prometheus_cert", + "tlsCertificate": { + "certificateChain": { + "filename": "test-cert-file" + }, + "privateKey": { + "filename": "test-key-file" + } + } + }, + { + "name": "prometheus_validation_context", + "validationContext": { + "trustedCa": { + "filename": "test-ca-file" + } + } + }` + + expectedPromSecretsWithBackendAndTLSCAPath = `{ + "name": "prometheus_cert", + "tlsCertificate": { + "certificateChain": { + "filename": "test-cert-file" + }, + "privateKey": { + "filename": "test-key-file" + } + } + }, + { + "name": "prometheus_validation_context", + "validationContext": { + "watchedDirectory": { + "path": "test-ca-directory" + } + } + }` + expectedStatsListener = `{ "name": "envoy_metrics_listener", "address": { @@ -760,6 +880,68 @@ func TestBootstrapConfig_ConfigureArgs(t *testing.T) { }, wantErr: false, }, + { + name: "prometheus-bind-addr-with-backend-and-tls", + input: BootstrapConfig{ + PrometheusBindAddr: "0.0.0.0:9000", + }, + baseArgs: BootstrapTplArgs{ + AdminBindAddress: "127.0.0.1", + AdminBindPort: "19000", + PrometheusBackendPort: "20100", + PrometheusScrapePath: "/metrics", + PrometheusCAFile: "test-ca-file", + PrometheusCertFile: "test-cert-file", + PrometheusKeyFile: "test-key-file", + }, + wantArgs: BootstrapTplArgs{ + AdminBindAddress: "127.0.0.1", + AdminBindPort: "19000", + // Should use the "prometheus_backend" cluster instead, which + // uses the PrometheusBackendPort rather than Envoy admin port + StaticClustersJSON: expectedPrometheusBackendCluster, + StaticListenersJSON: expectedPromListenerWithBackendAndTLS, + StaticSecretsJSON: expectedPromSecretsWithBackendAndTLS, + StatsConfigJSON: defaultStatsConfigJSON, + PrometheusBackendPort: "20100", + PrometheusScrapePath: "/metrics", + PrometheusCAFile: "test-ca-file", + PrometheusCertFile: "test-cert-file", + PrometheusKeyFile: "test-key-file", + }, + wantErr: false, + }, + { + name: "prometheus-bind-addr-with-backend-and-tls-ca-path", + input: BootstrapConfig{ + PrometheusBindAddr: "0.0.0.0:9000", + }, + baseArgs: BootstrapTplArgs{ + AdminBindAddress: "127.0.0.1", + AdminBindPort: "19000", + PrometheusBackendPort: "20100", + PrometheusScrapePath: "/metrics", + PrometheusCAPath: "test-ca-directory", + PrometheusCertFile: "test-cert-file", + PrometheusKeyFile: "test-key-file", + }, + wantArgs: BootstrapTplArgs{ + AdminBindAddress: "127.0.0.1", + AdminBindPort: "19000", + // Should use the "prometheus_backend" cluster instead, which + // uses the PrometheusBackendPort rather than Envoy admin port + StaticClustersJSON: expectedPrometheusBackendCluster, + StaticListenersJSON: expectedPromListenerWithBackendAndTLS, + StaticSecretsJSON: expectedPromSecretsWithBackendAndTLSCAPath, + StatsConfigJSON: defaultStatsConfigJSON, + PrometheusBackendPort: "20100", + PrometheusScrapePath: "/metrics", + PrometheusCAPath: "test-ca-directory", + PrometheusCertFile: "test-cert-file", + PrometheusKeyFile: "test-key-file", + }, + wantErr: false, + }, { name: "stats-bind-addr", input: BootstrapConfig{ diff --git a/command/connect/envoy/bootstrap_tpl.go b/command/connect/envoy/bootstrap_tpl.go index 6cb238d49..e6b71f548 100644 --- a/command/connect/envoy/bootstrap_tpl.go +++ b/command/connect/envoy/bootstrap_tpl.go @@ -76,6 +76,10 @@ type BootstrapTplArgs struct { // https://www.envoyproxy.io/docs/envoy/v1.9.0/api-v2/config/metrics/v2/stats.proto#envoy-api-msg-config-metrics-v2-statsconfig. StatsConfigJSON string + // StaticSecretsJSON is a JSON string containing zero or more Secret definitions. + // See https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/transport_sockets/tls/v3/secret.proto#envoy-v3-api-msg-extensions-transport-sockets-tls-v3-secret + StaticSecretsJSON string + // StatsFlushInterval is the time duration between Envoy stats flushes. It is // in proto3 "duration" string format for example "1.12s" See // https://developers.google.com/protocol-buffers/docs/proto3#json and @@ -106,6 +110,11 @@ type BootstrapTplArgs struct { // PrometheusScrapePath will configure the path where metrics are exposed on // the envoy_prometheus_bind_addr listener. PrometheusScrapePath string + + PrometheusCAFile string + PrometheusCAPath string + PrometheusCertFile string + PrometheusKeyFile string } // GRPC settings used in the bootstrap template. @@ -209,6 +218,12 @@ const bootstrapTemplate = `{ {{ .StaticListenersJSON }} ] {{- end }} + {{- if .StaticSecretsJSON -}} + , + "secrets": [ + {{ .StaticSecretsJSON }} + ] + {{- end }} }, {{- if .StatsSinksJSON }} "stats_sinks": {{ .StatsSinksJSON }}, diff --git a/command/connect/envoy/envoy.go b/command/connect/envoy/envoy.go index a78ae54d5..342357dfa 100644 --- a/command/connect/envoy/envoy.go +++ b/command/connect/envoy/envoy.go @@ -52,6 +52,10 @@ type cmd struct { envoyVersion string prometheusBackendPort string prometheusScrapePath string + prometheusCAFile string + prometheusCAPath string + prometheusCertFile string + prometheusKeyFile string // mesh gateway registration information register bool @@ -174,6 +178,19 @@ func (c *cmd) init() { "0.0.0.0:20200/scrape-metrics. "+ "Only applicable when envoy_prometheus_bind_addr is set in proxy config.") + c.flags.StringVar(&c.prometheusCAFile, "prometheus-ca-file", "", + "Path to a CA file for Envoy to use when serving TLS on the Prometheus metrics endpoint. "+ + "Only applicable when envoy_prometheus_bind_addr is set in proxy config.") + c.flags.StringVar(&c.prometheusCAPath, "prometheus-ca-path", "", + "Path to a directory of CA certificates for Envoy to use when serving the Prometheus metrics endpoint. "+ + "Only applicable when envoy_prometheus_bind_addr is set in proxy config.") + c.flags.StringVar(&c.prometheusCertFile, "prometheus-cert-file", "", + "Path to a certificate file for Envoy to use when serving TLS on the Prometheus metrics endpoint. "+ + "Only applicable when envoy_prometheus_bind_addr is set in proxy config.") + c.flags.StringVar(&c.prometheusKeyFile, "prometheus-key-file", "", + "Path to a private key file for Envoy to use when serving TLS on the Prometheus metrics endpoint. "+ + "Only applicable when envoy_prometheus_bind_addr is set in proxy config.") + c.http = &flags.HTTPFlags{} flags.Merge(c.flags, c.http.ClientFlags()) flags.Merge(c.flags, c.http.MultiTenancyFlags()) @@ -306,6 +323,15 @@ func (c *cmd) run(args []string) int { return 1 } + // If any of CA/Cert/Key are specified, make sure they are all present. + if c.prometheusKeyFile != "" || c.prometheusCertFile != "" || (c.prometheusCAFile != "" || c.prometheusCAPath != "") { + if c.prometheusKeyFile == "" || c.prometheusCertFile == "" || (c.prometheusCAFile == "" && c.prometheusCAPath == "") { + c.UI.Error("Must provide a CA (-prometheus-ca-file or -prometheus-ca-path) as well as " + + "-prometheus-cert-file and -prometheus-key-file to enable TLS for prometheus metrics") + return 1 + } + } + if c.register { if c.nodeName != "" { c.UI.Error("'-register' cannot be used with '-node-name'") @@ -505,6 +531,10 @@ func (c *cmd) templateArgs() (*BootstrapTplArgs, error) { Datacenter: httpCfg.Datacenter, PrometheusBackendPort: c.prometheusBackendPort, PrometheusScrapePath: c.prometheusScrapePath, + PrometheusCAFile: c.prometheusCAFile, + PrometheusCAPath: c.prometheusCAPath, + PrometheusCertFile: c.prometheusCertFile, + PrometheusKeyFile: c.prometheusKeyFile, }, nil } diff --git a/command/connect/envoy/envoy_test.go b/command/connect/envoy/envoy_test.go index 0e29a6934..4eedd16d4 100644 --- a/command/connect/envoy/envoy_test.go +++ b/command/connect/envoy/envoy_test.go @@ -211,6 +211,72 @@ func TestGenerateConfig(t *testing.T) { PrometheusScrapePath: "/scrape-path", }, }, + { + Name: "prometheus-metrics-tls-ca-file", + Flags: []string{"-proxy-id", "test-proxy", + "-prometheus-backend-port", "20100", "-prometheus-scrape-path", "/scrape-path", + "-prometheus-ca-file", "../../../test/key/ourdomain.cer", "-prometheus-cert-file", "../../../test/key/ourdomain_server.cer", + "-prometheus-key-file", "../../../test/key/ourdomain_server.key"}, + ProxyConfig: map[string]interface{}{ + // When envoy_prometheus_bind_addr is set, if + // PrometheusBackendPort is set, there will be a + // "prometheus_backend" cluster in the Envoy configuration. + "envoy_prometheus_bind_addr": "0.0.0.0:9000", + }, + WantArgs: BootstrapTplArgs{ + ProxyCluster: "test-proxy", + ProxyID: "test-proxy", + // We don't know this til after the lookup so it will be empty in the + // initial args call we are testing here. + ProxySourceService: "", + GRPC: GRPC{ + AgentAddress: "127.0.0.1", + AgentPort: "8502", // Note this is the gRPC port + }, + AdminAccessLogPath: "/dev/null", + AdminBindAddress: "127.0.0.1", + AdminBindPort: "19000", + LocalAgentClusterName: xds.LocalAgentClusterName, + PrometheusBackendPort: "20100", + PrometheusScrapePath: "/scrape-path", + PrometheusCAFile: "../../../test/key/ourdomain.cer", + PrometheusCertFile: "../../../test/key/ourdomain_server.cer", + PrometheusKeyFile: "../../../test/key/ourdomain_server.key", + }, + }, + { + Name: "prometheus-metrics-tls-ca-path", + Flags: []string{"-proxy-id", "test-proxy", + "-prometheus-backend-port", "20100", "-prometheus-scrape-path", "/scrape-path", + "-prometheus-ca-path", "../../../test/ca_path", "-prometheus-cert-file", "../../../test/key/ourdomain_server.cer", + "-prometheus-key-file", "../../../test/key/ourdomain_server.key"}, + ProxyConfig: map[string]interface{}{ + // When envoy_prometheus_bind_addr is set, if + // PrometheusBackendPort is set, there will be a + // "prometheus_backend" cluster in the Envoy configuration. + "envoy_prometheus_bind_addr": "0.0.0.0:9000", + }, + WantArgs: BootstrapTplArgs{ + ProxyCluster: "test-proxy", + ProxyID: "test-proxy", + // We don't know this til after the lookup so it will be empty in the + // initial args call we are testing here. + ProxySourceService: "", + GRPC: GRPC{ + AgentAddress: "127.0.0.1", + AgentPort: "8502", // Note this is the gRPC port + }, + AdminAccessLogPath: "/dev/null", + AdminBindAddress: "127.0.0.1", + AdminBindPort: "19000", + LocalAgentClusterName: xds.LocalAgentClusterName, + PrometheusBackendPort: "20100", + PrometheusScrapePath: "/scrape-path", + PrometheusCAPath: "../../../test/ca_path", + PrometheusCertFile: "../../../test/key/ourdomain_server.cer", + PrometheusKeyFile: "../../../test/key/ourdomain_server.key", + }, + }, { Name: "token-arg", Flags: []string{"-proxy-id", "test-proxy", diff --git a/command/connect/envoy/testdata/prometheus-metrics-tls-ca-file.golden b/command/connect/envoy/testdata/prometheus-metrics-tls-ca-file.golden new file mode 100644 index 000000000..239250c72 --- /dev/null +++ b/command/connect/envoy/testdata/prometheus-metrics-tls-ca-file.golden @@ -0,0 +1,320 @@ +{ + "admin": { + "access_log_path": "/dev/null", + "address": { + "socket_address": { + "address": "127.0.0.1", + "port_value": 19000 + } + } + }, + "node": { + "cluster": "test", + "id": "test-proxy", + "metadata": { + "namespace": "default", + "partition": "default" + } + }, + "static_resources": { + "clusters": [ + { + "name": "local_agent", + "ignore_health_on_host_removal": false, + "connect_timeout": "1s", + "type": "STATIC", + "http2_protocol_options": {}, + "loadAssignment": { + "clusterName": "local_agent", + "endpoints": [ + { + "lbEndpoints": [ + { + "endpoint": { + "address": { + "socket_address": { + "address": "127.0.0.1", + "port_value": 8502 + } + } + } + } + ] + } + ] + } + }, + { + "name": "prometheus_backend", + "ignore_health_on_host_removal": false, + "connect_timeout": "5s", + "type": "STATIC", + "http_protocol_options": {}, + "loadAssignment": { + "clusterName": "prometheus_backend", + "endpoints": [ + { + "lbEndpoints": [ + { + "endpoint": { + "address": { + "socket_address": { + "address": "127.0.0.1", + "port_value": 20100 + } + } + } + } + ] + } + ] + } + } + ], + "listeners": [ + { + "name": "envoy_prometheus_metrics_listener", + "address": { + "socket_address": { + "address": "0.0.0.0", + "port_value": 9000 + } + }, + "filter_chains": [ + { + "filters": [ + { + "name": "envoy.filters.network.http_connection_manager", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager", + "stat_prefix": "envoy_prometheus_metrics", + "codec_type": "HTTP1", + "route_config": { + "name": "self_admin_route", + "virtual_hosts": [ + { + "name": "self_admin", + "domains": [ + "*" + ], + "routes": [ + { + "match": { + "path": "/scrape-path" + }, + "route": { + "cluster": "prometheus_backend", + "prefix_rewrite": "/stats/prometheus" + } + }, + { + "match": { + "prefix": "/" + }, + "direct_response": { + "status": 404 + } + } + ] + } + ] + }, + "http_filters": [ + { + "name": "envoy.filters.http.router", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router" + } + } + ] + } + } + ], + "transportSocket": { + "name": "tls", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext", + "commonTlsContext": { + "tlsCertificateSdsSecretConfigs": [ + { + "name": "prometheus_cert" + } + ], + "validationContextSdsSecretConfig": { + "trustedCa": { + "name": "prometheus_validation_context" + } + } + } + } + } + } + ] + } + ], + "secrets": [ + { + "name": "prometheus_cert", + "tlsCertificate": { + "certificateChain": { + "filename": "../../../test/key/ourdomain_server.cer" + }, + "privateKey": { + "filename": "../../../test/key/ourdomain_server.key" + } + } + }, + { + "name": "prometheus_validation_context", + "validationContext": { + "trustedCa": { + "filename": "../../../test/key/ourdomain.cer" + } + } + } + ] + }, + "stats_config": { + "stats_tags": [ + { + "regex": "^cluster\\.(?:passthrough~)?((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.custom_hash" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.service_subset" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.service" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.namespace" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.partition" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.datacenter" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.routing_type" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)", + "tag_name": "consul.destination.trust_domain" + }, + { + "regex": "^cluster\\.(?:passthrough~)?(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.target" + }, + { + "regex": "^cluster\\.(?:passthrough~)?(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)", + "tag_name": "consul.destination.full_target" + }, + { + "regex": "^(?:tcp|http)\\.upstream\\.(([^.]+)(?:\\.[^.]+)?(?:\\.[^.]+)?\\.[^.]+\\.)", + "tag_name": "consul.upstream.service" + }, + { + "regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.[^.]+)?(?:\\.[^.]+)?\\.([^.]+)\\.)", + "tag_name": "consul.upstream.datacenter" + }, + { + "regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.([^.]+))?(?:\\.[^.]+)?\\.[^.]+\\.)", + "tag_name": "consul.upstream.namespace" + }, + { + "regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.[^.]+)?(?:\\.([^.]+))?\\.[^.]+\\.)", + "tag_name": "consul.upstream.partition" + }, + { + "regex": "^cluster\\.((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.custom_hash" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.service_subset" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.service" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.namespace" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.datacenter" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)", + "tag_name": "consul.routing_type" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)", + "tag_name": "consul.trust_domain" + }, + { + "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.target" + }, + { + "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)", + "tag_name": "consul.full_target" + }, + { + "tag_name": "local_cluster", + "fixed_value": "test" + }, + { + "tag_name": "consul.source.service", + "fixed_value": "test" + }, + { + "tag_name": "consul.source.namespace", + "fixed_value": "default" + }, + { + "tag_name": "consul.source.partition", + "fixed_value": "default" + }, + { + "tag_name": "consul.source.datacenter", + "fixed_value": "dc1" + } + ], + "use_all_default_tags": true + }, + "dynamic_resources": { + "lds_config": { + "ads": {}, + "resource_api_version": "V3" + }, + "cds_config": { + "ads": {}, + "resource_api_version": "V3" + }, + "ads_config": { + "api_type": "DELTA_GRPC", + "transport_api_version": "V3", + "grpc_services": { + "initial_metadata": [ + { + "key": "x-consul-token", + "value": "" + } + ], + "envoy_grpc": { + "cluster_name": "local_agent" + } + } + } + } +} + diff --git a/command/connect/envoy/testdata/prometheus-metrics-tls-ca-path.golden b/command/connect/envoy/testdata/prometheus-metrics-tls-ca-path.golden new file mode 100644 index 000000000..c00fa54ed --- /dev/null +++ b/command/connect/envoy/testdata/prometheus-metrics-tls-ca-path.golden @@ -0,0 +1,320 @@ +{ + "admin": { + "access_log_path": "/dev/null", + "address": { + "socket_address": { + "address": "127.0.0.1", + "port_value": 19000 + } + } + }, + "node": { + "cluster": "test", + "id": "test-proxy", + "metadata": { + "namespace": "default", + "partition": "default" + } + }, + "static_resources": { + "clusters": [ + { + "name": "local_agent", + "ignore_health_on_host_removal": false, + "connect_timeout": "1s", + "type": "STATIC", + "http2_protocol_options": {}, + "loadAssignment": { + "clusterName": "local_agent", + "endpoints": [ + { + "lbEndpoints": [ + { + "endpoint": { + "address": { + "socket_address": { + "address": "127.0.0.1", + "port_value": 8502 + } + } + } + } + ] + } + ] + } + }, + { + "name": "prometheus_backend", + "ignore_health_on_host_removal": false, + "connect_timeout": "5s", + "type": "STATIC", + "http_protocol_options": {}, + "loadAssignment": { + "clusterName": "prometheus_backend", + "endpoints": [ + { + "lbEndpoints": [ + { + "endpoint": { + "address": { + "socket_address": { + "address": "127.0.0.1", + "port_value": 20100 + } + } + } + } + ] + } + ] + } + } + ], + "listeners": [ + { + "name": "envoy_prometheus_metrics_listener", + "address": { + "socket_address": { + "address": "0.0.0.0", + "port_value": 9000 + } + }, + "filter_chains": [ + { + "filters": [ + { + "name": "envoy.filters.network.http_connection_manager", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager", + "stat_prefix": "envoy_prometheus_metrics", + "codec_type": "HTTP1", + "route_config": { + "name": "self_admin_route", + "virtual_hosts": [ + { + "name": "self_admin", + "domains": [ + "*" + ], + "routes": [ + { + "match": { + "path": "/scrape-path" + }, + "route": { + "cluster": "prometheus_backend", + "prefix_rewrite": "/stats/prometheus" + } + }, + { + "match": { + "prefix": "/" + }, + "direct_response": { + "status": 404 + } + } + ] + } + ] + }, + "http_filters": [ + { + "name": "envoy.filters.http.router", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.filters.http.router.v3.Router" + } + } + ] + } + } + ], + "transportSocket": { + "name": "tls", + "typedConfig": { + "@type": "type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext", + "commonTlsContext": { + "tlsCertificateSdsSecretConfigs": [ + { + "name": "prometheus_cert" + } + ], + "validationContextSdsSecretConfig": { + "trustedCa": { + "name": "prometheus_validation_context" + } + } + } + } + } + } + ] + } + ], + "secrets": [ + { + "name": "prometheus_cert", + "tlsCertificate": { + "certificateChain": { + "filename": "../../../test/key/ourdomain_server.cer" + }, + "privateKey": { + "filename": "../../../test/key/ourdomain_server.key" + } + } + }, + { + "name": "prometheus_validation_context", + "validationContext": { + "watchedDirectory": { + "path": "../../../test/ca_path" + } + } + } + ] + }, + "stats_config": { + "stats_tags": [ + { + "regex": "^cluster\\.(?:passthrough~)?((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.custom_hash" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.service_subset" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.service" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.namespace" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.partition" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.datacenter" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.routing_type" + }, + { + "regex": "^cluster\\.(?:passthrough~)?((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)", + "tag_name": "consul.destination.trust_domain" + }, + { + "regex": "^cluster\\.(?:passthrough~)?(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.destination.target" + }, + { + "regex": "^cluster\\.(?:passthrough~)?(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)", + "tag_name": "consul.destination.full_target" + }, + { + "regex": "^(?:tcp|http)\\.upstream\\.(([^.]+)(?:\\.[^.]+)?(?:\\.[^.]+)?\\.[^.]+\\.)", + "tag_name": "consul.upstream.service" + }, + { + "regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.[^.]+)?(?:\\.[^.]+)?\\.([^.]+)\\.)", + "tag_name": "consul.upstream.datacenter" + }, + { + "regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.([^.]+))?(?:\\.[^.]+)?\\.[^.]+\\.)", + "tag_name": "consul.upstream.namespace" + }, + { + "regex": "^(?:tcp|http)\\.upstream\\.([^.]+(?:\\.[^.]+)?(?:\\.([^.]+))?\\.[^.]+\\.)", + "tag_name": "consul.upstream.partition" + }, + { + "regex": "^cluster\\.((?:([^.]+)~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.custom_hash" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:([^.]+)\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.service_subset" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.service" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.namespace" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?([^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.datacenter" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.([^.]+)\\.[^.]+\\.consul\\.)", + "tag_name": "consul.routing_type" + }, + { + "regex": "^cluster\\.((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.([^.]+)\\.consul\\.)", + "tag_name": "consul.trust_domain" + }, + { + "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+)\\.[^.]+\\.[^.]+\\.consul\\.)", + "tag_name": "consul.target" + }, + { + "regex": "^cluster\\.(((?:[^.]+~)?(?:[^.]+\\.)?[^.]+\\.[^.]+\\.(?:[^.]+\\.)?[^.]+\\.[^.]+\\.[^.]+)\\.consul\\.)", + "tag_name": "consul.full_target" + }, + { + "tag_name": "local_cluster", + "fixed_value": "test" + }, + { + "tag_name": "consul.source.service", + "fixed_value": "test" + }, + { + "tag_name": "consul.source.namespace", + "fixed_value": "default" + }, + { + "tag_name": "consul.source.partition", + "fixed_value": "default" + }, + { + "tag_name": "consul.source.datacenter", + "fixed_value": "dc1" + } + ], + "use_all_default_tags": true + }, + "dynamic_resources": { + "lds_config": { + "ads": {}, + "resource_api_version": "V3" + }, + "cds_config": { + "ads": {}, + "resource_api_version": "V3" + }, + "ads_config": { + "api_type": "DELTA_GRPC", + "transport_api_version": "V3", + "grpc_services": { + "initial_metadata": [ + { + "key": "x-consul-token", + "value": "" + } + ], + "envoy_grpc": { + "cluster_name": "local_agent" + } + } + } + } +} + From 6bcd0652708d2d6ff37c97babb77ca8d89427251 Mon Sep 17 00:00:00 2001 From: Kyle Havlovitz Date: Thu, 16 Jun 2022 18:26:25 -0700 Subject: [PATCH 055/149] Add changelog note --- .changelog/13481.txt | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 .changelog/13481.txt diff --git a/.changelog/13481.txt b/.changelog/13481.txt new file mode 100644 index 000000000..2cfd0da42 --- /dev/null +++ b/.changelog/13481.txt @@ -0,0 +1,4 @@ +```release-note:improvement +command: Add support for enabling TLS in the Envoy Prometheus endpoint via the `consul connect envoy` command. +Adds the `-prometheus-ca-file`, `-prometheus-ca-path`, `-prometheus-cert-file` and `-prometheus-key-file` flags. +``` From dfcd28048a273921b7b2ac772f93d4e31e188470 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Fri, 17 Jun 2022 09:04:52 -0700 Subject: [PATCH 056/149] referred to mesh gateway functionality in ECS overview --- website/content/docs/ecs/index.mdx | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/website/content/docs/ecs/index.mdx b/website/content/docs/ecs/index.mdx index e5c1f2e42..73c1cbf81 100644 --- a/website/content/docs/ecs/index.mdx +++ b/website/content/docs/ecs/index.mdx @@ -8,15 +8,13 @@ description: >- # AWS ECS -Consul service mesh applications can be deployed on [AWS Elastic Container Service](https://aws.amazon.com/ecs/) (ECS) -using either our official [Terraform modules](/docs/ecs/terraform/install) or without Terraform by [manually configuring -the task definition](/docs/ecs/manual/install). +You can deploy Consul service mesh applications to [AWS Elastic Container Service](https://aws.amazon.com/ecs/) (ECS) using either our official [Terraform modules](/docs/ecs/terraform/install) or by [manually configuring the task definition](/docs/ecs/manual/install). ## Service Mesh Using Consul on AWS ECS enables you to add your ECS tasks to the service mesh and take advantage of features such as zero-trust-security, intentions, observability, -traffic policy, and more. +traffic policy, and more. You can also connect service meshes so that services deployed across your infrastructure environments can communicate. ## Architecture From 85463445b412bb80ede96f9dc714f852c7dbfaa8 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Fri, 17 Jun 2022 12:31:11 -0500 Subject: [PATCH 057/149] Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- website/content/api-docs/peering.mdx | 19 ++++++++----------- .../cluster-peering/create-manage-peering.mdx | 8 ++++---- .../docs/connect/cluster-peering/index.mdx | 10 +++++----- 3 files changed, 17 insertions(+), 20 deletions(-) diff --git a/website/content/api-docs/peering.mdx b/website/content/api-docs/peering.mdx index fa396015f..8c3ca4259 100644 --- a/website/content/api-docs/peering.mdx +++ b/website/content/api-docs/peering.mdx @@ -81,9 +81,9 @@ $ curl --request POST \ } ``` -## Establish a Peering +## Establish a Peering Connection -This endpoint establishes a peering with a given peering token. +This endpoint establishes a peering connection with a given peering token. | Method | Path | Produces | | ------- | -------------------- | ------------------ | @@ -154,9 +154,9 @@ $ curl --request POST \ {} ``` -## Read a Peering +## Read a Peering Connection -This endpoint reads a peering with the given name. +This endpoint returns information about a peering connection for the specified peer name. | Method | Path | Produces | | ------ | ------------------ | ------------------ | @@ -205,14 +205,11 @@ $ curl --header "X-Consul-Token: b23b3cad-5ea1-4413-919e-c76884b9ad60" \ } ``` -## Delete a Peering +## Delete a Peering Connection -This endpoint marks a peering for deletion. Once marked, Consul will -delete all the data imported from the peer in the background. Only once -all associated data has been deleted will the peering actually disappear. -Until then, further reads can be performed on the peering and a `DeletedAt` -field will be populated with the timestamp of when the peering was -marked for deletion. +Call this endpoint to delete a peering connection. Consul deletes all data imported from the peer in the background. The peering connection is removed after all associated data has been deleted. +Operators can still read the peering connections while the data is being removed. A `DeletedAt` +field will be populated with the timestamp of when the peering was marked for deletion. | Method | Path | Produces | | -------- | ------------------ | -------- | diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index db0d1bc6b..f0849cd60 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -47,7 +47,7 @@ Create a JSON file that contains the first cluster's name and the peering token. ## Establish a connection between clusters -Next, use the peering token to establish a secure connection between the clusters. In the client agents of “cluster-02,” establish the peering using the HTTP API. This endpoint does not generate an output unless there is an error. +Next, use the peering token to establish a secure connection between the clusters. In the client agents of “cluster-02,” establish the peering connection using the HTTP API. This endpoint does not generate an output unless there is an error. ```shell-session $ curl --request POST --data @peering_token.json http://127.0.0.1:8500/v1/peering/establish @@ -55,7 +55,7 @@ $ curl --request POST --data @peering_token.json http://127.0.0.1:8500/v1/peerin In the peer parameter, specify a name for the first cluster. The `PeeringToken` parameter should include the entire peering token created in the first cluster. -When you connect server agents through cluster peering, they will peer their default partitions. To establish peerings for other partitions through server agents, you must specify the partitions you want to peer using the `Partition` field of the request body. +When you connect server agents through cluster peering, they will peer their default partitions. To establish peering connections for other partitions through server agents, you must specify the partitions you want to peer using the `Partition` field of the request body. ## Connect service endpoints @@ -117,7 +117,7 @@ Sources = [ -**Tip:** If the peer’s name is not specified in `Peer`, then Consul assumes that the service is in the local cluster. +If the peer’s name is not specified in `Peer`, then Consul assumes that the service is in the local cluster. Then, add the configuration entry to your cluster. @@ -138,7 +138,7 @@ A successful query will include service information in the output. ## Remove peering connections -After you create a peering connection between clusters in different datacenters, you can “unpeer” the connected clusters. Deleting a peering connection stops data replication to the peer and deletes imported data, including services and CA certificates. +After you create a peering connection between clusters in different datacenters, you can disconnect the peered clusters. Deleting a peering connection stops data replication to the peer and deletes imported data, including services and CA certificates. In “cluster-01,” request the deletion via the HTTP API. diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 1f5fd0053..a94f258f8 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -7,7 +7,7 @@ description: >- # What is Cluster Peering? -->**Cluster peering is currently in technical preview**: Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support. +~> **Cluster peering is currently in technical preview**: Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support. You can create peering connections between two or more independent clusters so that services deployed to different partitions or datacenters can to communicate. @@ -22,23 +22,23 @@ For detailed instructions on setting up cluster peering with the Consul CLI, ref ### Differences between WAN federation and cluster peering -WAN Federation and cluster peering are different ways to connect clusters. The most important distinction is that WAN Federation assumes clusters are owned by the same operators, so it maintains and replicates global states like ACLs and Configuration Entries. As a result, WAN Federation requires a “primary datacenter" to serve as an authority for replicated data. +WAN federation and cluster peering are different ways to connect clusters. The most important distinction is that WAN federation assumes clusters are owned by the same operators, so it maintains and replicates global states such as ACLs and configuration entries. As a result, WAN federation requires a _primary datacenter_ to serve as an authority for replicated data. -Regardless of whether you connect your clusters through WAN Federation or cluster peering, human and machine users can use either method to discover services in other clusters or dial them through the service mesh. +Regardless of whether you connect your clusters through WAN federation or cluster peering, human and machine users can use either method to discover services in other clusters or dial them through the service mesh. | | WAN Federation | Cluster Peering | | :----------------------------------------------- | :------------: | :-------------: | | Connects clusters across datacenters | ✅ | ✅ | | Shares support queries and service endpoints | ✅ | ✅ | | Connects clusters owned by different operators | ❌ | ✅ | -| Functions without declaring “primary datacenter” | ❌ | ✅ | +| Functions without declaring primary datacenter | ❌ | ✅ | | Shares key/value stores | ✅ | ❌ | | Uses gossip protocol | ✅ | ❌ | ## Technical preview constraints Not all features and functionality are available in the technical preview release. In particular, consider the following technical constraints: -- Mesh gateways for _service to service traffic_ between clusters are available. However, mesh gateways for _server to server traffic_ are not available. +- Mesh gateways for _server to server traffic_ are not available. However, mesh gateways for _service to service traffic_ between clusters are available. - Services exported to peered clusters must not be configured as HTTP. - Support for dynamic routing such as splits, custom routes, or redirects is not available. - The `consul intention CLI` command is not supported. To manage intentions that specify services in peered clusters, use [configuration entries](/docs/connect/config-entries/service-intentions). From 4c520323a564fb0732e91cbd8b585714ce0b7049 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Fri, 17 Jun 2022 12:35:35 -0500 Subject: [PATCH 058/149] Update website/content/docs/connect/cluster-peering/create-manage-peering.mdx --- .../docs/connect/cluster-peering/create-manage-peering.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx index f0849cd60..07dd3cf67 100644 --- a/website/content/docs/connect/cluster-peering/create-manage-peering.mdx +++ b/website/content/docs/connect/cluster-peering/create-manage-peering.mdx @@ -24,7 +24,7 @@ You can generate peering tokens and initiate connections using the Consul API on To begin the cluster peering process, generate a peering token in one of your clusters. The other cluster uses this token to establish the peering connection. -In “cluster-01,” issue a request for a peering token using the HTTP API. +In `cluster-01`, issue a request for a peering token using the [HTTP API](/api-docs/peering). ```shell-session $ curl --request POST --data '{"PeerName":"cluster-02"}' --url http://localhost:8500/v1/peering/token From 1e6ddc8f820b62f56d1d4eb1d32bf3922e1c3ce6 Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Fri, 17 Jun 2022 13:36:20 -0500 Subject: [PATCH 059/149] Apply suggestions from code review --- website/content/api-docs/peering.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/content/api-docs/peering.mdx b/website/content/api-docs/peering.mdx index 8c3ca4259..607ecf311 100644 --- a/website/content/api-docs/peering.mdx +++ b/website/content/api-docs/peering.mdx @@ -42,7 +42,7 @@ The table below shows this endpoint's support for - `Partition` `(string: "")` - The admin partition that the peering token is generated from. Uses `default` when not specified. -- `Datacenter` `(string: "")` - Specifies the datacenter. Defaults to the +- `Datacenter` `(string: "")` - Specifies the datacenter where the peering token is generated. Defaults to the agent's datacenter when not specified. - `Token` `(string: "")` - Specifies the ACL token to use in the request. Takes precedence @@ -116,7 +116,7 @@ The table below shows this endpoint's support for - `PeeringToken` `(string: )` - The peering token fetched from the peer cluster. -- `Datacenter` `(string: "")` - Specifies the datacenter. Defaults to the +- `Datacenter` `(string: "")` - Specifies the datacenter where the peering token is generated. Defaults to the agent's datacenter when not specified. - `Token` `(string: "")` - Specifies the ACL token to use in the request. Takes precedence From e00c5c7554b8c4e651c0d5a447d374abf5c5ad02 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Fri, 17 Jun 2022 12:58:47 -0700 Subject: [PATCH 060/149] updates to ECS Terraform install --- .../content/docs/ecs/terraform/install.mdx | 193 ++++++++++++------ 1 file changed, 136 insertions(+), 57 deletions(-) diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index e810e141a..b48e1736f 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -7,34 +7,41 @@ description: >- # Installation with Terraform -This topic describes how to use the [`mesh-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh. If you do not use Terraform, see the [Manual Installation](/docs/ecs/manual-installation) page to install Consul on ECS without Terraform. +This topic describes how to use HashiCorp’s Terraform modules to launch your application in AWS ECS as part of Consul service mesh. If you do not use Terraform, refer to the [Manual Installation](/docs/ecs/manual-installation) page to install Consul on ECS without Terraform. -This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. +This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. ## Overview -This topic describes the following procedure: +The following procedure describes the general workflow: 1. Create Terraform configuration files for the necessary components: - * [ECS task definition](#using-the-mesh-task-module): Use the `mesh-task` module to create an ECS task definition for Consul on ECS - * [ECS service](#ecs-service): Use the `aws_ecs_service` resource to create an ECS service that schedules service mesh tasks to run on ECS + * [ECS task definition](#create-the-task-definition): Use the HashiCorp Terraform modules to create the ECS task definition. + * [ECS service](#ecs-service): Use the `aws_ecs_service` resource to create an ECS service that schedules service mesh tasks to run on ECS. 2. [Run Terraform](#running-terraform) to deploy the resources in AWS +If you want to operate Consul with ACLs enabled (recommended), follow the instructions in the [Secure Configuration](/docs/ecs/terraform/secure-configuration) documentation. ACLs provide network security for production-grade deployments. + ## Prerequisites * You should have some familiarity with using Terraform. Refer to the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform. * You should also be familiar with AWS ECS before following these instructions. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for details. -## Using the Mesh Task Module +## Create the task definition -To run an application in ECS with Consul service mesh, you must create an ECS task definition, which includes your application container(s) -and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container. +To run an application in ECS with Consul service mesh, you must create an ECS task definition. The task defintion includes your application container(s) and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container. -The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers. +Create a Terraform configuration file and include the `mesh-task` module. The module automatically adds the necessary sidecar containers. -The following example shows a Terraform configuration file that creates a task definition with an application container called `example-client-app` in a file called `mesh-task.tf`: +If you intend to peer the service mesh to multiple Consul datacenters or partitions, you can also include the `gateway-task` module. The `gateway-task` enables connectivity between datacenters across service meshes. + +### Configure the mesh task module + +Create a Terraform configuration file (e.g., `mesh-task.tf`) and specify the `mesh-task` module in the `source` field. The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers. + +In the following example, the a Terraform configuration file called `mesh-task.tf` creates a task definition with an application container called `example-client-app`: @@ -83,7 +90,124 @@ The following fields are required. Refer to the [module reference documentation] | `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | | `consul_datacenter` | string | The name of your Consul datacenter. | -### Running Terraform +### Configure an ECS service for the mesh task module + +[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) are one of the most common +ways to start tasks using a task definition. + +To define an ECS service, reference the `mesh-task` module's `task_definition_arn` output value +in your `aws_ecs_service` resource. The following example shows how to include the service in the `mesh-task.tf` file. + + + +```hcl +module "my_task" { + source = "hashicorp/consul-ecs/aws//modules/mesh-task" + ... +} + +resource "aws_ecs_service" "my_task" { + name = "my_task_service" + task_definition = module.my_task.task_definition_arn + launch_type = "FARGATE" + propagate_tags = "TASK_DEFINITION" + ... +} +``` + + + +This is a partial configuration to highlight some important fields. +See the [`aws_ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) documentation for a complete reference. + +| Input Variable | Type | Description | +| ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The name of the ECS service. This is required by AWS but is not used by Consul service mesh. | +| `task_definition` | string | The task definition used to start tasks. Set this to the task definition ARN returned by the `mesh-task` module. | +| `launch_type` | string | The launch type. Consul on ECS supports the `FARGATE` and `EC2` launch types. | +| `propagate_tags` | string | This must be set to `TASK_DEFINITION` so that tags added by `mesh-task` to the task definition are copied to tasks. | + +After including the ECS service in your Terraform configuration, run `terraform apply` +from your project directory to create the ECS service resource. The ECS service will +soon start your application in a task. The task will automatically register itself +into the Consul service catalog during startup. + +-> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true` +in their [`network_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#network_configuration) block so that ECS can pull the Docker images. + +### Configure the gateway task module + +Add the `gateway-task` to your Terraform configuration if you want to federate multiple service meshes across Consul datacenters over the WAN. Refer to [WAN Federation via Mesh Gateways](/docs/connect/gateways/mesh-gateway/wan-federation-via-mesh-gateways) to learn more about the federation deployment model. + +You must add and configure a `gateway-task` for each Consul datacenter in your network. The module creates an ECS service and a task definition that includes the following containers: + +* Consul client +* Envoy gateway proxy +* Mesh init + +You will need to provide inputs for the artifacts created by the `gateway-task` module. + +#### Task definition + +The `kind` parameter is the only required input. The value must be set to `mesh-gateway`. The following table describes optional inputs for the task definition. Refer to the [example gateway task configuration](#example-gateway-task-configuration) for a fully-configured task definition. + +| Input variable | Type | Description | +| --- | --- | --- | +| `kind` | string | Specifies the kind of gateway to create. The value must be set to `mesh-gateway`. | +| `lan_address` | string | Specifies the LAN address for the gateway. The address is also used as the service address. Defaults to the node address. | +| `lan_port` | integer | Specifies the LAN port for the gateway. Also used as the service port. Defaults to `8443`. | +| `wan_address` | string | Specifies the WAN address for the gateway. Defaults to the `lan_address`.
If the `assign_public_ip` is set to `true`, the WAN address will be set to the public IP address.
If the `load_balancer_target_group_arn` is specified but no value for `wan_address` is provided, then the WAN address will be set to the load balancer’s DNS name.
To set a static WAN address, specify an explicit value for `wan_address` and `wan_port`. | +| `wan_port` | integer | Specifies the WAN port for the gateway. Defaults to the `lan_port`. | +| `family` | string | Specifies the ECS task definition family. The family is also used as the Consul service name by default. | +| `requires_compatibilities` | list of strings | Specifies one or more launch types required by the task. Defaults to `[“FARGATE”, “EC2”]` | +| `cpu` | integer | Specifies the number of CPUs used by the task. Defaults to `256`. | +| `memory` | integer | Specifies the mount (in MiB) of memory used by the task. Default is `512`. | +| `task_role` | object | Specifies the ECS task role to include in the task definition. If not provided, a role is created. Defaults to `{ "arn": null, "id": null }` | +| `execution_role` | object | Specifies the ECS execution role to include in the task definition. If not provided, a role is created. Defaults to `{ "arn": null, "id": null }` | +| `additional_task_role_policies` | list of strings | Specifies additional policy ARNs to attach to the task role. Default is `[]`. | +| `addition_task_execution_role_policies` | | | +| `log_configuration` | object | Specifies configurations for the task definition log. See [LogConfiguration](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LogConfiguration.html) in the AWS documentation. Default is `{}`.| +| `tags` | object | Defines tags to add to all resources. Default is `{}`.| +| `consul_agent_configuration` | string | Specifies the contents of a configuration file for the Consul agent in HCL format. Default is `""`. | +| `consul_datacenter` | string | Specifies the name of the Consul datacenter that the client belongs to. Default is `dc1`.| +| `consul_service_name` | string | Specifies the name for the service when it registers will Consul. Defaults to the task family name. | +| `consul_service_tags` | list of strings | Defines a list of tags to include in the Consul service registration. Default is `[]`.| +| `consul_service_meta` | object | Defines metadata to attach to the Consul service registration. | +| `consul_image` | string | Specifies the Consul Docker image to use. Default is `public.ecr.aws/hashicorp/consul:1.13.0` | +| `consul_ecs_image` | string | Specifies the Consul on ECS Docker image to use. Default is `public.ecr.aws/hashicorp/consul-ecs:0.6.0` | +| `consul_namespace` | string | Specifies which Consul namespace to register the service. Default is `default`.| +| `consul_partition` | | Specifies which Consul admin partition to register the service. Default is `default`. | +| `envoy_image` | string | Specifies the name of the Envoy Docker image to use. Default is `envoyproxy/envoy-alpine:v1.21.2` | +| `retry_join` | list of strings | Defines a set of arguments to pass to the Consul agent [`-retry-join`](/docs/agent/config/cli-flags#_retry_join) flag. | +| `consul_server_ca_cert_arn` | string | Specifies the ARN of the Secrets Manager containing the Consul server CA certificate for Consul's internal remote procedure calls (RPC). | +| `consul_client_token_secret_arn` | | | +| `gossip_key_secret_arn` | string | Specifies the ARN of the Secrets Manager containing the Consul's gossip encryption key. | +| `acls` | Boolean | Set to `true` to enable Consul's [access control lists (ACLs)](/docs/security/acl/index). Default is `false`.| +| `acl_secret_name_prefix` | | | + +#### ECS service + +The ECS service is created as part of the `gateway-task` module configuration. The service can run one or more instances of the gateway. + +The following table describes the inputs for configuring the ECS service in your Terraform configuration file. All inputs are required. Refer to the [example gateway task configuration](#example-gateway-task-configuration) for a fully-configured task definition. + +| Input variable | Type | Description | +| --- | --- | --- | +| `ecs_cluster_arn` | string | Specifies the ECS cluster where tasks should be launched. | +| `launch_type` | string | Specifies the ECS service launch type. Can be either `fargate` or `ec2`. | +| `desired_count` | integer | Specifies the number instances for the service to create. Defaults to `0`. | +| `subnets` | string | Specifies the subnet IDs where the tasks will launch. | +| `security_group_ids` | string | Specifies the security group IDs to assign to the task ENI. | +| `assign_public_ip` | Boolean | Set to `true` to create a task accessible at a public IP address. Default is `false`.
If set to `true` and `wan_address` is not configured, the WAN address will be set to the public IP of the task. | +| `load_balancer_target_group_arn` | string | Specifies the ARN of an existing load balancer target group. The load balancer target group allows ingress to the gateway task.
No additional load balancer configuration is necessary. Only NLBs and ALBs are supported. The container name and port will be automatically set based on other fields. | + +#### Mesh init + +The `mesh-init` container is a short-lived container that sets up the initial configurations for Consul and Envoy (refer to [Task Startup](/docs/ecs/architecture#task-startup) for additional information). The `gateway-task` module automatically configures the `mesh-init` container based on the inputs specified in the [task definition](#task-definition) and [ECS service](#ecs-service) configuration. + +### Example gateway task configuration + +## Run Terraform You will need to run Terraform to create the task definition. @@ -132,52 +256,7 @@ Terraform should be run in your project directory as follows. Terraform automatically reads all files in the current directory that have a `.tf` file extension. Refer to the [Terraform documentation](https://www.terraform.io/docs) for more information and Terraform best practices. -## ECS Service - -[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) are one of the most common -ways to start tasks using a task definition. - -To define an ECS service, reference the `mesh-task` module's `task_definition_arn` output value -in your `aws_ecs_service` resource. The following example shows how to include the service in the `mesh-task.tf` file. - - - -```hcl -module "my_task" { - source = "hashicorp/consul-ecs/aws//modules/mesh-task" - ... -} - -resource "aws_ecs_service" "my_task" { - name = "my_task_service" - task_definition = module.my_task.task_definition_arn - launch_type = "FARGATE" - propagate_tags = "TASK_DEFINITION" - ... -} -``` - - - -This is a partial configuration to highlight some important fields. -See the [`aws_ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) documentation for a complete reference. - -| Input Variable | Type | Description | -| ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------- | -| `name` | string | The name of the ECS service. This is required by AWS but is not used by Consul service mesh. | -| `task_definition` | string | The task definition used to start tasks. Set this to the task definition ARN returned by the `mesh-task` module. | -| `launch_type` | string | The launch type. Consul on ECS supports the `FARGATE` and `EC2` launch types. | -| `propagate_tags` | string | This must be set to `TASK_DEFINITION` so that tags added by `mesh-task` to the task definition are copied to tasks. | - -After including the ECS service in your Terraform configuration, run `terraform apply` -from your project directory to create the ECS service resource. The ECS service will -soon start your application in a task. The task will automatically register itself -into the Consul service catalog during startup. - --> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true` -in their [`network_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#network_configuration) block so that ECS can pull the Docker images. - -## Routing +## Configure routes Now that your tasks are registered in the mesh, you're able to use the service mesh to route between them. From 1b1cfa900ede63abc41a885d5393806567143215 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Fri, 17 Jun 2022 14:15:29 -0700 Subject: [PATCH 061/149] minor tweaks to TF install --- .../content/docs/ecs/terraform/install.mdx | 23 ++++++++++--------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index b48e1736f..cbf06aeaf 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -176,7 +176,7 @@ The `kind` parameter is the only required input. The value must be set to `mesh- | `consul_image` | string | Specifies the Consul Docker image to use. Default is `public.ecr.aws/hashicorp/consul:1.13.0` | | `consul_ecs_image` | string | Specifies the Consul on ECS Docker image to use. Default is `public.ecr.aws/hashicorp/consul-ecs:0.6.0` | | `consul_namespace` | string | Specifies which Consul namespace to register the service. Default is `default`.| -| `consul_partition` | | Specifies which Consul admin partition to register the service. Default is `default`. | +| `consul_partition` | string | Specifies which Consul admin partition to register the service. Default is `default`. | | `envoy_image` | string | Specifies the name of the Envoy Docker image to use. Default is `envoyproxy/envoy-alpine:v1.21.2` | | `retry_join` | list of strings | Defines a set of arguments to pass to the Consul agent [`-retry-join`](/docs/agent/config/cli-flags#_retry_join) flag. | | `consul_server_ca_cert_arn` | string | Specifies the ARN of the Secrets Manager containing the Consul server CA certificate for Consul's internal remote procedure calls (RPC). | @@ -318,21 +318,22 @@ module "web" { } ``` -## Bind Address +## Configure the bind address To ensure that your application only receives traffic through the service mesh, -you must change the address that your application is listening on to only the loopback address -(also known as `localhost`, `lo`, and `127.0.0.1`) -so that only the sidecar proxy running in the same task can make requests to it. +you must change the address that your application is listening on to only the loopback address. The loopback address is also called `localhost`, `lo`, and `127.0.0.1`. +Binding to the loopback address allows the sidecar proxy running in the same task to only make requests within the service mesh. -If your application is listening on all interfaces, e.g. `0.0.0.0`, then other +If your application is listening on all interfaces, e.g., `0.0.0.0`, then other applications can call it directly, bypassing its sidecar proxy. Changing the listening address is specific to the language and framework you're using in your application. Regardless of which language/framework you're using, -it's a good practice to make the address configurable via environment variable. +it is a good practice to make the address configurable via environment variable. -For example in Go, you would use: +The following examples demonstrate how to bind the loopback address in golang and Django (Python): + + ```go s := &http.Server{ @@ -342,13 +343,13 @@ s := &http.Server{ log.Fatal(s.ListenAndServe()) ``` -In Django you'd use: - ```bash python manage.py runserver "127.0.0.1:8080" ``` -## Next Steps + + +## Next steps - Follow the [Secure Configuration](/docs/ecs/terraform/secure-configuration) to get production-ready. - Now that your applications are running in the service mesh, read about From 4850a1d4c1c965d0fd0f9c0c6b44530a27fe27a0 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Fri, 17 Jun 2022 14:42:51 -0700 Subject: [PATCH 062/149] tweaks to the secure TF install section --- .../ecs/terraform/secure-configuration.mdx | 47 ++++++++++++------- 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/website/content/docs/ecs/terraform/secure-configuration.mdx b/website/content/docs/ecs/terraform/secure-configuration.mdx index 6a932960e..9489ee14b 100644 --- a/website/content/docs/ecs/terraform/secure-configuration.mdx +++ b/website/content/docs/ecs/terraform/secure-configuration.mdx @@ -7,16 +7,22 @@ description: >- # Secure Configuration -For a production-ready installation of Consul on ECS, you will need to make sure that the cluster is secured. -A secure Consul cluster should include the following: +This topic describes how to enable Consul security features for your production workloads. The following overview describes the process: -1. [TLS Encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. -1. [Gossip Encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. -1. [Access Control (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. +1. Enable the security features on your Consul server cluster per the [Prerequisites](#prerequisites). +1. Deploy the ACL controller. +1. Deploy your services. --> **NOTE:** This page assumes that you have already configured your Consul server with the above features. +## Prerequisites -## Deploy ACL Controller +Implement the following configurations before proceeding: + +1. [TLS encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. +1. [Gossip encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. +1. [Access control lists (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. + + +## Deploy the ACL controller Before deploying your service, you will need to deploy the [ACL controller](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller) so that it can provision the necessary tokens for tasks on the service mesh. To learn more about the ACL Controller, please see [Automatic ACL Token Provisioning](/docs/ecs/architecture#automatic-acl-token-provisioning). @@ -60,18 +66,17 @@ module "acl_controller" { ``` The `name_prefix` parameter is used to prefix any secrets that the ACL controller will -update in AWS Secrets Manager. +update in AWS Secrets Manager. The `name_prefix` parameter value must be unique for each ECS cluster where you are deploying this controller. --> **NOTE:** Make sure that the `name_prefix` is unique for each ECS cluster where you are -deploying this controller. +## Deploy your services -## Deploy Services +Follow the instructions described in [Create a task definition](/docs/ecs/terraform/install#create-the-task-definition) to create the basic configuration for the task module. Add the following additional configurations to make the configuration production-ready. -Once the ACL controller is up and running, you will be able to deploy services on the mesh using the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task). -Start with the basic configuration for the [Task Module](/docs/ecs/terraform/install#task-module) and specify additional settings to make the configuration production-ready. +### Create an AWS Secrets Manager secret -First, you will need to create an AWS Secrets Manager secret for the gossip encryption key that the Consul clients -should use. +The secret stores the gossip encryption key that the Consul clients will use. + + ```hcl resource "aws_secretsmanager_secret" "gossip_key" { @@ -83,8 +88,11 @@ resource "aws_secretsmanager_secret_version" "gossip_key" { secret_string = "" } ``` + -Next, add the following configurations to enable secure deployment. Note that the `acl_secret_name_prefix` +### Enable secure deployment + +Add the following configurations to enable secure deployment. The `acl_secret_name_prefix` should be the same as the `name_prefix` you provide to the ACL controller module. ```hcl @@ -104,5 +112,8 @@ module "my_task" { } ``` -Now you can deploy your services! Follow the rest of the steps in the [Installation instructions](/docs/ecs/terraform/install#task-module) -to deploy and connect your services. +Complete the following steps described in the Installation with Terraform chapter to deploy and connect your services: + +1. [Run Terraform](/docs/ecs/terraform/install#run-terraform) +1. [Configure routes](/docs/ecs/terraform/install#configure-routes) +1. [Configure the bind address](/docs/ecs/terraform/install#configure-the-bind-address) From 1cee20a6442e5a67c3c1eeea181dfb86802aefb1 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Fri, 17 Jun 2022 14:57:37 -0700 Subject: [PATCH 063/149] Added note about manually creating mesh gw not being supported --- website/content/docs/ecs/manual/install.mdx | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/website/content/docs/ecs/manual/install.mdx b/website/content/docs/ecs/manual/install.mdx index 1046c69c6..7f671bb19 100644 --- a/website/content/docs/ecs/manual/install.mdx +++ b/website/content/docs/ecs/manual/install.mdx @@ -7,7 +7,9 @@ description: >- # Manual Installation -The following instructions describe how to manually create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) without Terraform. Refer to the [Consul ECS Terraform module](/docs/ecs/terraform/install) documentation for an alternative method for installing Consul on ECS. +The following instructions describe how to manually create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) without Terraform. Refer to the [Consul ECS Terraform module](/docs/ecs/terraform/install) documentation for an alternative method for installing Consul on ECS. + +If you intend to peer the service mesh to multiple Consul datacenters or partitions, you must use the Consul ECS Terraform module to install your service mesh on ECS. Manually configuring mesh gateways without using the `gateway-task` Terraform module is not supported. This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. @@ -17,13 +19,13 @@ You should have some familiarity with AWS ECS. See [What is Amazon Elastic Conta ## Task Definition -You must create a task definition, which includes the following containers: +Create a task definition configured to create the containers: * Your application container * An Envoy sidecar-proxy container * A Consul client container * A `consul-ecs-mesh-init` container for service mesh setup -* Optionally, a `consul-ecs-health-sync` container to sync ECS health checks into Consul +* (Optional) A `consul-ecs-health-sync` container to sync ECS health checks into Consul ## Top-level fields From c5c5ef7845ad6bfd7108cfaebb892c94f601667a Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Fri, 17 Jun 2022 15:13:48 -0700 Subject: [PATCH 064/149] tweaks to the secure configuration for manually installing consul ecs --- .../docs/ecs/manual/secure-configuration.mdx | 115 +++++++++++++----- 1 file changed, 84 insertions(+), 31 deletions(-) diff --git a/website/content/docs/ecs/manual/secure-configuration.mdx b/website/content/docs/ecs/manual/secure-configuration.mdx index c96d0afdf..fa96934be 100644 --- a/website/content/docs/ecs/manual/secure-configuration.mdx +++ b/website/content/docs/ecs/manual/secure-configuration.mdx @@ -7,41 +7,36 @@ description: >- # Secure Configuration -For a production-ready installation of Consul on ECS, you will need to make sure that the cluster is secured. -A secure Consul cluster should include the following: - -1. [TLS Encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. -1. [Gossip Encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. -1. [Access Control (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. - --> **NOTE:** In this topic, we assume that you have already configured your Consul server with the security-related features. +This topic describes how to enable Consul security features for your production workloads. ## Prerequisites -* You should already have followed the [installation instructions](/docs/ecs/manual/install) to understand how to define -the necessary components of the task definition for Consul on ECS. -* You should be familiar with [specifying sensitive data](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) on ECS. -* You should be familiar with configuring Consul's secure features, including how to create ACL tokens and policies. Refer to the following [Learn Guides](https://learn.hashicorp.com/collections/consul/security) for an introduction and the [ACL system](/docs/security/acl) documentation for more information. +The following features must be configured for your Consul server cluster: + +* [TLS encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. +* [Gossip encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. +* [Access control lists (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. + +You should already have followed the [installation instructions](/docs/ecs/manual/install) to understand how to define the necessary components of the task definition for Consul on ECS. + +You should be familiar with [specifying sensitive data](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) on ECS. + +You should be familiar with configuring Consul's secure features, including how to create ACL tokens and policies. Refer to the [ACL system documentation](/docs/security/acl) and [Day 1: Security tutorials](https://learn.hashicorp.com/collections/consul/security) for an introduction and additional information. ## ACL Tokens +Tokens are artifacts within the ACL system that authenticate users, services, and Consul agents. Tokens are linked to policies that specify the resources the token bearer has access to when making requests in the network. + You must create two types of ACL tokens for Consul on ECS: * **Client tokens:** used by the `consul-client` containers to join the Consul cluster * **Service tokens:** used by sidecar containers for service registration and health syncing -The following sections describe the ACL polices which must be associated with these token types. +This section describes how to manually create ACL tokens. You can install the ACL controller, however, to ease the burden of creating tokens. The ACL controller can automatically create ACL tokens for Consul on ECS. Refer to the [ACL Controller](/docs/manual/acl-controller) documentation for installation details. --> **NOTE:** This section describes how operators would create ACL tokens by hand. To ease operator -burden, the ACL Controller can automatically create ACL tokens for Consul on ECS. Refer to the -[ACL Controller](/docs/manual/acl-controller) page for installation details. +### Define policies -### Create Consul client token - -You must create a token for the Consul client. This is a shared token used by the `consul-client` -containers to join the Consul cluster. - -The following is the ACL policy needed for the Consul client token: +Confiture the following ACL policy for the Consul client token: ```hcl node_prefix "" { @@ -52,22 +47,80 @@ service_prefix "" { } ``` -This policy allows `node:write` for any node name, which is necessary because the Consul node -names on ECS are not known until runtime. +The policy allows `node:write` for any node name, which is necessary because the Consul node names on ECS are not known until runtime. + +You can add the policy in Consul using the [`consul acl policy create`](/commands/acl/policy/create) command or the [`[PUT] /v1/acl/policy`](/api-docs/acl/policies#create-a-policy) API endpoint. + +If you intend to create a gateway for connecting multiple Consul datacenters, you will need additional policies to specify the permission scope. + +#### Mesh gateway policy + +Mesh gateways must run in the default namespace. + +```hcl +namespace "default" { // If namespaces enabled + service "" { + policy = "write" + } +} +namespace_prefix "" { // If namespaces enabled + service_prefix "" { + policy = "read" + } + node_prefix "" { + policy = "read" + } +} +agent_prefix "" { + policy = "read" +} +``` +#### Terminating gateway policy + +```hcl +partition "" { // If partitions enabled + namespace "" { // If namespaces enabled + service "" { + policy = "write" + } + node_prefix "" { + policy = "read" + } + } +} +``` + +#### Ingress gateway policy + +```hcl +partition "" { // If partitions enabled + namespace "" { // If namespaces enabled + service "" { + policy = "write" + } + node_prefix "" { + policy = "read" + } + service_prefix "" { + policy = "read" + } + } +} +``` + ### Create service tokens -Service tokens should be associated with a [service identity](/docs/security/acl#service-identities). -The service identity includes `service:write` permissions for the service and sidecar proxy. +Create the Consul client token and the service tokens after adding the necessary policies. Service tokens should be associated with a service identity. The service identity includes `service:write` permissions for the service and sidecar proxy. -The following example shows how to use the Consul CLI to create a service token for a service named `example-client-app`: +You can create tokens using the [`consul acl token create`](/commands/acl/token/create) command or the [`[PUT] /v1/acl/token`](/api-docs/acl/tokens#create-a-token) API endpoint. +The following example shows how to use the Consul CLI to create a service token for a service named example-client-app: -```shell -consul acl token create -service-identity=example-client-app ... +```shell-session +$ consul acl token create -service-identity=example-client-app ... ``` +You will need to create one service token for each registered Consul service in ECS, including when new services are added to the service mesh. --> **NOTE**: You will need to create one service token for each registered Consul service in ECS, -including when new services are added to the service mesh. ## Secret storage From 7249a0326e8f476cf9ba28fe8ee96999cbc4bdf9 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Fri, 17 Jun 2022 15:17:48 -0700 Subject: [PATCH 065/149] tweaks to the enterprise section for ecs mesh gateways --- website/content/docs/ecs/enterprise.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/website/content/docs/ecs/enterprise.mdx b/website/content/docs/ecs/enterprise.mdx index 6ee42d5be..7a6a53545 100644 --- a/website/content/docs/ecs/enterprise.mdx +++ b/website/content/docs/ecs/enterprise.mdx @@ -7,20 +7,19 @@ description: >- # Consul Enterprise -Consul on ECS supports running Consul Enterprise by specifying the Consul Enterprise -Docker image in the Terraform module parameters. +You can run Consul Enterprise on ECS by specifying the Consul Enterprise Docker image in the Terraform module parameters. -## How To Use Consul Enterprise +## Specify the Consul image -When instantiating the [`mesh-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) module, -set the parameter `consul_image` to a Consul Enterprise image, e.g. `hashicorp/consul-enterprise:1.10.0-ent`: +When instantiating the [`mesh-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) or [`gateway-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/gateway-task) module, +set the parameter `consul_image` to a Consul Enterprise image. The following example instructs the `mesh-task` module to import Consul Enterprise version 1.12.0: ```hcl module "my_task" { source = "hashicorp/consul-ecs/aws//modules/mesh-task" version = "" - consul_image = "hashicorp/consul-enterprise:-ent" + consul_image = "hashicorp/consul-enterprise:1.12.0-ent" ... } ``` @@ -62,11 +61,12 @@ If client support is required for any of the features, then you must use a Consu ### Admin Partitions and Namespaces -Consul on ECS supports [admin partitions](/docs/enterprise/admin-partitions) and [namespaces](/docs/enterprise/namespaces) when Consul Enterprise servers and clients are used. -These features have the following requirements: +Consul on ECS supports [admin partitions](/docs/enterprise/admin-partitions) and [namespaces](/docs/enterprise/namespaces) when Consul Enterprise servers and clients are used. These features have the following requirements: + * ACLs must be enabled. * ACL controller must run in the ECS cluster. * `mesh-tasks` must use a Consul Enterprise client image. +* `gateway-task`s must use a Consul Enterprise client image. The ACL controller automatically manages ACL policies and token provisioning for clients and services on the service mesh. It also creates admin partitions and namespaces if they do not already exist. From 902e76d3042f2b2b33aad7ab68eed56381747b5f Mon Sep 17 00:00:00 2001 From: Freddy Date: Fri, 17 Jun 2022 18:40:38 -0600 Subject: [PATCH 066/149] Additional service mesh docs updates for peering (#13464) This PR covers two sets of changes: - Documenting the new `destination_peer` for proxy upstream definitions. - Updating the exported-services config entry documentation. Updates to the `exported-services` config entry include: - As of 1.13.0 it is no longer only for Consul Enterprise - A `PeerName` is now a possible consumer for an exported service. - Added examples for OSS and Enterprise - Linked to peering docs --- .../config-entries/exported-services.mdx | 591 ++++++++++++++++-- .../registration/service-registration.mdx | 21 +- 2 files changed, 543 insertions(+), 69 deletions(-) diff --git a/website/content/docs/connect/config-entries/exported-services.mdx b/website/content/docs/connect/config-entries/exported-services.mdx index b822a039c..60a247e43 100644 --- a/website/content/docs/connect/config-entries/exported-services.mdx +++ b/website/content/docs/connect/config-entries/exported-services.mdx @@ -8,22 +8,20 @@ description: >- # Exported Services - +This topic describes the `exported-services` configuration entry type. The `exported-services` configuration entry enables Consul to export service instances to other clusters from a single file and connect services across clusters. For additional information, refer to [Cluster Peering](/docs/connect/cluster-peering) and [Admin Partitions](/docs/enterprise/admin-partitions). -This topic describes the `exported-services` configuration entry type. The `exported-services` configuration entry enables Consul to export service instances to other admin partitions from a single file. This enables your services to be networked across admin partitions. See [Admin Partitions](/docs/enterprise/admin-partitions) for additional information. - --> **v1.11.0+:** This config entry is supported in Consul Enterprise versions 1.11.0+. +-> **v1.11.0+:** This config entry is supported in Consul versions 1.11.0+. ## Introduction -You can configure Consul to export services contained in an admin partition to one or more additional partitions by declaring the `exported-services` configuration entry in the `kind` field. This enables you to route traffic between services in different clusters that share a single set of Consul servers. +To configure Consul to export services contained in a Consul Enterprise admin partition or Consul OSS datacenter to one or more additional clusters, create a new configuration entry and declare `exported-services` in the `kind` field. This configuration entry enables you to route traffic between services in different clusters. -You can configure the settings defined in the `exported-services` configuration entry to apply to all namespaces and federated datacenters. +You can configure the settings defined in the `exported-services` configuration entry to apply to all namespaces in a Consul Enterprise admin partition. ## Requirements -- A Consul Enterprise binary -- A corresponding partition that the configuration entry can export to. For example, the `exported-services` configuration entry for a partition named `frontend` requires an existing `frontend` partition. +- A 1.11.0+ Consul Enteprise binary or a 1.13.0+ Consul OSS binary. +- **Enterprise Only**: A corresponding partition that the configuration entry can export from. For example, the `exported-services` configuration entry for a partition named `frontend` requires an existing `frontend` partition. ## Usage @@ -37,6 +35,56 @@ You can configure the settings defined in the `exported-services` configuration Configure the following parameters to define a `exported-services` configuration entry: + + + + +```hcl +Kind = "exported-services" +Name = "default" +Services = [ + { + Name = "" + Consumers = [ + { + PeerName = "" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: ExportedServices +metadata: + name: default +spec: + services: + - name: + consumers: + - peerName: +``` + +```json +"Kind": "exported-services", +"Name": "default", +"Services": [ + { + "Name": "", + "Consumers": [ + { + "PeerName": "" + } + ] + } +] +``` + + + + + ```hcl @@ -49,8 +97,8 @@ Services = [ Namespace = "" Consumers = [ { - Partition = "" - }, + PeerName = "" + } ] } ] @@ -66,7 +114,7 @@ spec: - name: namespace: consumers: - - partition: + - peerName: ``` ```json @@ -75,18 +123,73 @@ spec: "Name": "", "Services": [ { - "Consumers": [ - { - "Partition": "" - } - ], "Name": "", "Namespace": "" + "Consumers": [ + { + "PeerName": "" + } + ] } ] ``` + + + + + +```hcl +Kind = "exported-services" +Partition = "" +Name = "" +Services = [ + { + Name = "" + Namespace = "" + Consumers = [ + { + Partition = "" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +kind: ExportedServices +metadata: + name: +spec: + services: + - name: + namespace: + consumers: + - partition: +``` + +```json +"Kind": "exported-services", +"Partition": "", +"Name": "", +"Services": [ + { + "Name": "", + "Namespace": "" + "Consumers": [ + { + "Partition": "" + } + ] + } +] +``` + + + + ### Configuration Parameters @@ -94,25 +197,113 @@ The following table describes the parameters associated with the `exported-servi | Parameter | Description | Required | Default | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -------- | ------- | -| `Kind` | String value that enables the configuration entry. The value should always be `exported-services` (HCL and JSON) or `ExportedServices` (YAML) | Required | None | -| `Partition` | String value that specifies the name of the partition that contains the services you want to export. | Required | None | -| `Name` | String value that specifies the name of the partition that contains the services you want to export. | Required | None | -| `Services` | List of objects that specify which services to export. See [`Services`](#services) for details. | Required | None | -| `Meta` | Object that defines a map of the max 64 key/value pairs. | Optional | None | +| `Kind` | String value that enables the configuration entry. The value should always be `exported-services` (HCL and JSON) or `ExportedServices` (YAML) | Required | None | +| `Partition` | String value that specifies the name of the partition that contains the services you want to export. | Required | None | +| `Name` | String value that specifies the name of the partition that contains the services you want to export. Must be `default` in Consul OSS. | Required | None | +| `Services` | List of objects that specify which services to export. For details, refer to [`Services`](#services). | Required | None | +| `Meta` | Object that defines a map of the max 64 key/value pairs. | Optional | None | ### Services -The `Services` parameter contains one or more lists of parameters that specify which services to export, which namespaces the services reside, and the destination partition for the exported services. Each list in the `Services` block must contain the following parameters: +The `Services` parameter contains a list of one or more parameters that specify which services to export, which namespaces the services reside, and the destination cluster for the exported services. Each item in the `Services` list must contain the following parameters: -- `Name`: Specifies the name of the service to export. You can use a asterisk wildcard (`*`) to include all services in the namespace. -- `Namespace`: Specifies the namespace containing the services to export. You can use a asterisk wildcard (`*`) to include all namespaces in the partition. -- `Consumers`: Specifies one ore more objects that identify a destination partition for the exported services. +- `Name`: Specifies the name of the service to export. You can use an asterisk wildcard (`*`) to include all services in the namespace. +- `Namespace`: Specifies the namespace containing the services to export. You can use an asterisk wildcard (`*`) to include all namespaces in the partition. +- `Consumers`: Specifies one or more objects that identify a destination cluster for the exported services. -## Example +### Consumers -The following example configures the agent to export the `billing` service from the `default` namespace of the `finance` admin partition to the `frontend` and `backend` partitions. Additionally, all services in all namespaces within the `finance` partition will be exported to the `monitoring` partition. +The `Consumers` parameter contains a list of one or more parameters that specify the destination cluster for +an exported service. Each item in the `Consumers` list must contain exactly one of the following parameters: - +- `PeerName`: Specifies the name of the peered cluster to export the service to. +A asterisk wildcard (`*`) cannot be specified as the `PeerName`. Added in Consul 1.13.0. +- `Partition`: Specifies an admin partition in the datacenter to export the service to. +A asterisk wildcard (`*`) cannot be specified as the `Partition`. + +## Examples + + +### Exporting services to peered clusters + + + + + +The following example configures Consul to export the `payments` and `refunds` services to the peered `web-shop` cluster. + + + +```hcl +Kind = "exported-services" +Name = "default" + +Services = [ + { + Name = "payments" + Consumers = [ + { + PeerName = "web-shop" + }, + ] + }, + { + Name = "refunds" + Consumers = [ + { + PeerName = "web-shop" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: default +spec: + services: + - name: payments + consumers: + - peerName: web-shop + - name: refunds + consumers: + - peerName: web-shop +``` + +```json +"Kind": "exported-services", + "Name": "default", + "Services": [ + { + "Name": "payments", + "Consumers": [ + { + "PeerName": "web-shop" + }, + ], + }, + { + "Name": "refunds", + "Consumers": [ + { + "PeerName": "web-shop" + } + ] + } + ] +``` + + + + + + +The following example configures Consul to export the `payments` and `refunds` services from the `billing` namespace of the `finance` admin partition to the `web-shop` peer. + + ```hcl Kind = "exported-services" @@ -121,23 +312,20 @@ Name = "finance" Services = [ { - Name = "billing" - Namespace = "default" + Name = "payments" + Namespace = "billing" Consumers = [ { - Partition = "frontend" + PeerName = "web-shop" }, - { - Partition = "backend" - } ] }, { - Name = "*" - Namespace = "*" + Name = "refunds" + Namespace = "billing" Consumers = [ { - Partition = "monitoring" + PeerName = "web-shop" } ] } @@ -151,15 +339,14 @@ metadata: name: finance spec: services: - - name: mesh-gateway - namespace: default + - name: payments + namespace: billing consumers: - - partition: default - - name: billing - namespace: default + - peerName: web-shop + - name: refunds + namespace: billing consumers: - - partition: frontend - - partition: backend + - peerName: web-shop ``` ```json @@ -168,41 +355,327 @@ spec: "Name": "finance", "Services": [ { + "Name": "payments", + "Namespace": "billing" "Consumers": [ { - "Partition": "frontend" + "PeerName": "web-shop" }, - { - "Partition": "backend" - } ], - "Name": "billing", - "Namespace": "default" }, { + "Name": "refunds", + "Namespace": "billing", "Consumers": [ { - "Partition": "monitoring" + "PeerName": "web-shop" } - ], - "Name": "*", - "Namespace": "*" + ] } ] ``` + + + + +The following example configures Consul to export the `payments` and `refunds` services from the `billing` namespace of the `finance` admin partition to the `web-shop` partition. + + + +```hcl +Kind = "exported-services" +Partition = "finance" +Name = "finance" + +Services = [ + { + Name = "payments" + Namespace = "billing" + Consumers = [ + { + Partition = "web-shop" + }, + ] + }, + { + Name = "refunds" + Namespace = "billing" + Consumers = [ + { + Partition = "web-shop" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: finance +spec: + services: + - name: payments + namespace: billing + consumers: + - partition: web-shop + - name: refunds + namespace: billing + consumers: + - partition: web-shop +``` + +```json +"Kind": "exported-services", + "Partition": "finance", + "Name": "finance", + "Services": [ + { + "Name": "payments", + "Namespace": "billing" + "Consumers": [ + { + "Partition": "web-shop" + }, + ], + }, + { + "Name": "refunds", + "Namespace": "billing", + "Consumers": [ + { + "Partition": "web-shop" + } + ] + } + ] +``` + + + + + +### Exporting all services + + + + +The following example configures Consul to export all services in the datacenter to the peered `monitoring` and `platform` clusters. + + + +```hcl +Kind = "exported-services" +Name = "default" + +Services = [ + { + Name = "*" + Consumers = [ + { + PeerName = "monitoring" + }, + { + PeerName = "platform" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: default +spec: + services: + - name: * + consumers: + - peerName: monitoring + - peerName: platform +``` + +```json +"Kind": "exported-services", + "Name": "default", + "Services": [ + { + "Name": "*", + "Namespace": "*" + "Consumers": [ + { + "PeerName": "monitoring" + }, + { + "PeerName": "platform" + } + ] + } + ] +``` + + + + + + +The following example configures Consul to export all services in all namespaces of the `finance` partition to the peered `monitoring` and `platform` clusters. + + + +```hcl +Kind = "exported-services" +Partition = "finance" +Name = "finance" + +Services = [ + { + Name = "*" + Namespace = "*" + Consumers = [ + { + PeerName = "monitoring" + }, + { + PeerName = "platform" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: finance +spec: + services: + - name: * + namespace: * + consumers: + - peerName: monitoring + - peerName: platform +``` + +```json +"Kind": "exported-services", + "Partition": "finance", + "Name": "finance", + "Services": [ + { + "Name": "*", + "Namespace": "*" + "Consumers": [ + { + "PeerName": "monitoring" + }, + { + "PeerName": "platform" + } + ] + } + ] +``` + + + + + + +The following example configures Consul to export all services in all namespaces of the `finance` partition to the `monitoring` and `platform` partitions. + + + +```hcl +Kind = "exported-services" +Partition = "finance" +Name = "finance" + +Services = [ + { + Name = "*" + Namespace = "*" + Consumers = [ + { + Partition = "monitoring" + }, + { + Partition = "platform" + } + ] + } +] +``` + +```yaml +apiVersion: consul.hashicorp.com/v1alpha1 +Kind: ExportedServices +metadata: + name: finance +spec: + services: + - name: * + namespace: * + consumers: + - partition: monitoring + - partition: platform +``` + +```json +"Kind": "exported-services", + "Partition": "finance", + "Name": "finance", + "Services": [ + { + "Name": "*", + "Namespace": "*" + "Consumers": [ + { + "Partition": "monitoring" + }, + { + "Partition": "platform" + } + ] + } + ] +``` + + + + ## Reading Services -When an exported service has been imported to another partition, you can use the `health` REST API endpoint to query the service on the consumer partition. The following example queries the `finance` partition for the imported `billing` service: +When an exported service has been imported to another cluster, you can use the `health` REST API endpoint to query the service on the consumer cluster. + + + + +The following example queries the `finance` peer for the imported `payments` service: ```shell-session -$ curl 'localhost:8500/v1/health/connect/billing?partition=finance' +$ curl 'localhost:8500/v1/health/service/payments?peer=finance' ``` + -An ACL token with `service:write` permissions is required for the partition from which the query is made. If the call in the previous example is made from a service named `web` in a partition named `frontend`, then the request will require a token with `write` permissions to `web` in the `frontend` partition. + -Exports are available to all services in the consumer partition. In the previous example, any service with `write` permissions for the `frontend` partition will be able to read exports. +The following example queries the `finance` partition for the imported `payments` service: -See [Health HTTP Endpoint](/api-docs/health) for additional information. +```shell-session +$ curl 'localhost:8500/v1/health/service/payments?partition=finance' +``` + + + +An ACL token with `service:write` permissions is required for the cluster the query is made from. If the call in the previous example is made from a service named `web` in a partition named `frontend`, then the request requires a token with `write` permissions to `web` in the `frontend` partition. + +Exports are available to all services in the consumer cluster. In the previous example, any service with `write` permissions for the `frontend` partition can read exports. + +For additional information, refer to [Health HTTP Endpoint](/api-docs/health). diff --git a/website/content/docs/connect/registration/service-registration.mdx b/website/content/docs/connect/registration/service-registration.mdx index 3deac820e..66b1aa96f 100644 --- a/website/content/docs/connect/registration/service-registration.mdx +++ b/website/content/docs/connect/registration/service-registration.mdx @@ -35,13 +35,13 @@ port = "destination_service_name": "", "" : "" }, - "port": + "port": } ``` -The following table describes the parameters that must be added to the service definition to declare the service as a proxy. +The following table describes the parameters that must be added to the service definition to declare the service as a proxy. | Parameter | Description | Required | Default | | --- | --- | --- | --- | @@ -82,8 +82,8 @@ proxy = { ### Sidecar Proxy Configuration -Many service mesh proxies are deployed as sidecars. -Sidecar proxies are co-located with the single service instance they represent and proxy all inbound traffic to. +Many service mesh proxies are deployed as sidecars. +Sidecar proxies are co-located with the single service instance they represent and proxy all inbound traffic to. Specify the following parameters in the `proxy` code block to configure a sidecar proxy in its own service registration: @@ -99,7 +99,7 @@ The following example includes values for all available options when registering -```hcl +```hcl kind = "connect-proxy" name = "redis-proxy" port = 8181 @@ -166,6 +166,7 @@ You can configure the service mesh proxy to create listeners for upstream servic | --- | --- | --- | --- | |`destination_name` | String value that specifies the name of the service or prepared query to route the service mesh to. The prepared query should be the name or the ID of the prepared query. | Required | None | | `destination_namespace` | String value that specifies the namespace containing the upstream service. | Optional | `default` | +| `destination_peer` | String value that specifies the name of the peer cluster containing the upstream service. | Optional | None | | `destination_partition` | String value that specifies the name of the admin partition containing the upstream service. | Optional | `default` | | `local_bind_port` | Integer value that specifies the port to bind a local listener to. The application will make outbound connections to the upstream from the local port. | Required | None | | `local_bind_address` | String value that specifies the address to bind a local listener to. The application will make outbound connections to the upstream service from the local bind address. | Optional | `127.0.0.1` | @@ -265,7 +266,7 @@ You can configure which mode a proxy operates in by specifying `"direct"` or `"t * `transparent`: In this mode, inbound and outbound application traffic is captured and redirected through the proxy. This mode does not enable the traffic redirection. It directs Consul to configure Envoy as if traffic is already being redirected. * `direct`: In this mode, the proxy's listeners must be dialed directly by the local application and other proxies. -You can also specify an empty string (`""`), which configures the proxy to operate in the default mode. The default mode is inherited from parent parameters in the following order of precedence: +You can also specify an empty string (`""`), which configures the proxy to operate in the default mode. The default mode is inherited from parent parameters in the following order of precedence: 1. Proxy service's `Proxy` configuration 1. The `service-defaults` configuration for the service. @@ -294,12 +295,12 @@ registrations](/docs/discovery/services#service-definition-parameter-case). - `outbound_listener_port` `(int: 15001)` - The port the proxy should listen on for outbound traffic. This must be the port where outbound application traffic is captured and redirected to. - `dialed_directly` `(bool: false)` - Determines whether this proxy instance's IP address can be dialed - directly by transparent proxies. Typically transparent proxies dial upstreams using the "virtual" - tagged address, which load balances across instances. Dialing individual instances can be helpful - in cases like stateful services such as a database cluster with a leader. + directly by transparent proxies. Transparent proxies typically dial upstreams using the "virtual" + tagged address, which load balances across instances. A database cluster with a leader is an example + where dialing individual instances can be helpful. ~> **Note:** Dynamic routing rules such as failovers and redirects do not apply to services dialed directly. - Additionally, the connection is proxied using a TCP proxy with a connection timeout of 5 seconds. + Additionally, the connection is proxied using a TCP proxy with a connection timeout of 5 seconds. ### Mesh Gateway Configuration Reference From 569cf68daa98cd5b9c4e3271c0832b9e720e4182 Mon Sep 17 00:00:00 2001 From: Chris Thain Date: Mon, 20 Jun 2022 09:07:44 -0700 Subject: [PATCH 067/149] Add mesh gateway configuration examples. --- .../content/docs/ecs/terraform/install.mdx | 134 +++++++++++++++++- 1 file changed, 133 insertions(+), 1 deletion(-) diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index 0e690e560..122601912 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -79,7 +79,139 @@ The following fields are required. Refer to the [module reference documentation] | `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. | | `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. | | `port` | integer | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | -| `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | +| `retry_join` | list | This is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | + +## Configuring a Mesh Gateway + +The `gateway-task` Terraform module can be used to deploy mesh gateways to enable service to service communication across the WAN. +Mesh gateways can also be used to federate service mesh traffic across Consul admin partitions and Consul datacenters over the WAN. + +~> This topic requires familiarity with [mesh gateways](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters). + +Using the `gateway-task` module to deploy mesh gateways requires that all Consul server and client agents in all datacenters have TLS and gossip encryption enabled. +Mesh gateways operate by sniffing and extracting the server name indication (SNI) header from the service mesh session and routing the connection to the appropriate destination based on the server name requested. + +The following example shows a Terraform configuration that creates a mesh gateway task called `my-gateway` in a file called `mesh-gateway.tf`: + + + +```hcl +module "my_mesh_gateway" { + source = "hashicorp/consul-ecs/aws//modules/gateway-task" + version = "" + kind = "mesh-gateway" + + family = "my-gateway" + ecs_cluster_arn = "" + subnets = [""] + retry_join = ["
"] + tls = true + consul_server_ca_cert_arn = "" + gossip_key_secret_arn = "" +} +``` + + + +The following fields are required. Refer to the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/gateway-task?tab=inputs) for a complete reference. + +| Input Variable | Type | Description | +| --------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `source` | string | Must be set to the source location of the `gateway-task` module, `hashicorp/consul-ecs/aws//modules/gateway-task`. | +| `version` | string | Must be set to the version of the `gateway-task` module. | +| `kind` | string | The kind of gateway to create. Must be set to `"mesh-gateway"` to create a mesh-gateway. | +| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. | +| `ecs_cluster_arn` | string | ARN of the ECS cluster to deploy the mesh gateway task to. | +| `subnets` | list | The list of subnet IDs associated with the mesh gateway task. | +| `retry_join` | list | This is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul client agent, which specifies the locations of the Consul servers in the local datacenter. | +| `consul_server_ca_cert_arn` | string | ARN of the Secrets Manager secret that contains the Consul CA certificate. | +| `gossip_key_secret_arn` | string | ARN of the Secrets Manager secret that contains the Consul gossip encryption key. | + + +### Mesh Gateway Ingress + +To route traffic between datacenters, mesh gateways need to be reachable over the WAN. +Providing the `lb_enabled = true` flag will cause the `gateway-task` module to automatically deploy and configure a Network Load Balancer for ingress to the mesh-gateway. +You also need to provide the VPC identifier and at least one public subnet to associate with the load balancer. + + + +```hcl +module "my_mesh_gateway" { + ... + + lb_enabled = true + lb_vpc_id = "" + lb_subnets = [""] +} +``` + + + +Alternatively, you can manually configure ingress to the mesh gateway and provide the `wan_address` and `wan_port` inputs to the +`gateway-task` module. + +~> Mesh gateways route L4 TCP connections and do not terminate mTLS sessions. If you manually configure an Elastic Load Balancer for ingress to a mesh gateway you must use a Network Load Balancer or a Classic Load Balancer. + + + + +```hcl +module "my_mesh_gateway" { + ... + + wan_address = "" + wan_port = +} +``` + + + +The `wan_port` field is optional. If it is not provided, port `8443` is used by default. + +### ACLs + +The following example shows how to configure the `gateway-task` when ACLs are enabled. + + + +```hcl +module "my_mesh_gateway" { + ... + + acls = true + consul_http_addr = "" + consul_https_ca_cert_arn = "" +} +``` + + + +The `consul_http_addr` input is the HTTP `address:port` of the Consul server and is required for the mesh gateway task to log in to Consul via the IAM Auth Method to obtain its client and service tokens. +The `consul_https_ca_cert_arn` input is the ARN of the Secrets Manager secret that contains the certificate for the Consul HTTPS API. + + +### WAN Federation with Mesh Gateways + +The following example shows how to configure the `gateway-task` module to enable [WAN federation via mesh gateways](/docs/connect/gateways/mesh-gateway/wan-federation-via-mesh-gateways). + + + +```hcl +module "my_mesh_gateway" { + ... + + consul_datacenter = "" + consul_primary_datacenter = "" + enable_mesh_gateway_wan_federation = true + enable_acl_token_replication = true +} +``` + + + +~> When federating Consul datacenters over the WAN with ACLs enabled, [ACL Token replication](/docs/security/acl/acl-federated-datacenters) must be enabled on all server and client agents in all datacenters. + ### Running Terraform From 7e8b21a0e0d85cb92f791e53b223c3b4fc567480 Mon Sep 17 00:00:00 2001 From: Nitya Dhanushkodi Date: Mon, 20 Jun 2022 09:16:31 -0700 Subject: [PATCH 068/149] add peering helm value --- website/content/docs/k8s/helm.mdx | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/website/content/docs/k8s/helm.mdx b/website/content/docs/k8s/helm.mdx index 53274177a..9d3037ea3 100644 --- a/website/content/docs/k8s/helm.mdx +++ b/website/content/docs/k8s/helm.mdx @@ -61,6 +61,12 @@ Use these links to navigate to a particular top-level stanza. (see `-domain` (https://www.consul.io/docs/agent/config/cli-flags#_domain)) and the domain services synced from Consul into Kubernetes will have, e.g. `service-name.service.consul`. + - `peering` ((#v-global-peering)) - [Experimental] Configures the Cluster Peering feature. Requires Consul v1.13+ and Consul-K8s v0.45+. + + - `enabled` ((#v-global-peering-enabled)) (`boolean: false`) - If true, the Helm chart will enable Cluster Peering for the cluster. This will enable peering controllers and + allow use of the PeeringAcceptor and PeeringDialer CRDs to establish peerings for service mesh. + @type boolean + - `adminPartitions` ((#v-global-adminpartitions)) - Enabling `adminPartitions` allows creation of Admin Partitions in Kubernetes clusters. It additionally indicates that you are running Consul Enterprise v1.11+ with a valid Consul Enterprise license. Admin partitions enables deploying services across partitions, while sharing From 39ff759d86348e949d9cda6af5f7e58b8764ed44 Mon Sep 17 00:00:00 2001 From: Nitya Dhanushkodi Date: Mon, 20 Jun 2022 09:34:59 -0700 Subject: [PATCH 069/149] upstream annotation --- .../docs/k8s/annotations-and-labels.mdx | 125 ++++++++++++------ 1 file changed, 87 insertions(+), 38 deletions(-) diff --git a/website/content/docs/k8s/annotations-and-labels.mdx b/website/content/docs/k8s/annotations-and-labels.mdx index b98346563..9f5608fba 100644 --- a/website/content/docs/k8s/annotations-and-labels.mdx +++ b/website/content/docs/k8s/annotations-and-labels.mdx @@ -69,53 +69,102 @@ Resource annotations could be used on the Kubernetes pod to control connect-inje local port to listen for those connections. When transparent proxy is enabled, this annotation is optional. - - Services + - There are a few formats this annotation can take: - The name of the service is the name of the service registered with Consul. You can optionally specify datacenters with this annotation. + - Unlabeled: + The unlabeled annotation format supports specifying service name, Consul Enterprise Namespaces and Partitions, + and datacenter. To use [cluster peering](/docs/connect/cluster-peering/k8s) with upstreams, use the labeled format below. + - Service Name + To specify the upstream service, you can provide the name of the service, and optionally, it's datacenter. + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter]" + ``` - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter]" - ``` + - Consul Enterprise Namespace + When using Consul Enterprise Namespaces [1.7+], your upstream services may be running in different namespaces. + The upstream namespace can be specified after the service name as `[service-name].[namespace]`. See [Consul + Enterprise Namespaces](#consul-enterprise-namespaces) below for more details on configuring the injector. + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace]:[port]:[optional datacenter]" + ``` + -> **NOTE:** If the namespace is not specified it will default to the namespace + of the source service. - - Consul Enterprise Namespaces + ~> **WARNING:** Setting a namespace when not using Consul Enterprise or using a version < 1.7 + is not supported. It will be treated as part of the service name. - If running Consul Enterprise 1.7+, your upstream services may be running in different - namespaces. The upstream namespace can be specified after the service name - as `[service-name].[namespace]`. See [Consul Enterprise Namespaces](#consul-enterprise-namespaces) - below for more details on configuring the injector. + - Consul Enterprise Admin Partition + When using Consul Enterprise Admin Partitions [1.11+], your upstream services may be running in a different + partition. You must specify the namespace when specifying a partition, and the datacenter must be your local + datacenter if specified. Communicating across partitions using this method is only supported within a + datacenter. For cross partition communication across datacenters, see [cluster + peering](/docs/connect/cluster-peering/k8s). + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace].[service-partition]:[port]:[optional datacenter]" + ``` + - [Prepared Query](/docs/connect/proxies#dynamic-upstreams-require-native-integration): + Prepared query upstreams can use the format below. + ```yaml + annotations: + 'consul.hashicorp.com/connect-service-upstreams': 'prepared_query:[query name]:[port]' + ``` - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace]:[port]:[optional datacenter]" - ``` + - Labeled (requires consul-k8s v0.45.0+): + The labeled format is required when using the cluster peering feature and specifying an upstream in another + peer. It also supports specifying Consul Enterprise Namespace, Partition, or Datacenter. Only one of peer, + datacenter, or partition can be specified. + - Service Name + To specify the upstream service, you can provide the name of the service, followed by `.svc`. + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc:[port]" + ``` + - Peer or Datacenter + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-peer].peer:[port]" + ``` + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-dc].dc:[port]" + ``` + - Consul Enterprise + You can specify a Consul Enterprise Namespace + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns:[port]" + ``` + When specifying a peer, datacenter or Consul Enterprise Admin Partition when namespaces are enabled, you must + provide the namespace and one of (peer, datacenter, partition). + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-peer].peer:[port]" + ``` + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-partition].ap:[port]" + ``` + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-dc].dc:[port]" + ``` - -> **NOTE:** If the namespace is not specified it will default to the namespace - of the source service. + - Multiple Upstreams - ~> **WARNING:** Setting a namespace when not using Consul Enterprise or using a version < 1.7 - is not supported. It will be treated as part of the service name. + If you would like to specify multiple services or upstreams, delimit them with commas. They can be of any of the Unlabeled, Labeled, or Prepared Query formats above when using the supported versions for the formats. - - [Prepared Query](/docs/connect/proxies#dynamic-upstreams-require-native-integration) + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],[service-name]:[port]:[optional datacenter]" + ``` - ```yaml - annotations: - 'consul.hashicorp.com/connect-service-upstreams': 'prepared_query:[query name]:[port]' - ``` - - - Multiple Upstreams - - If you would like to specify multiple services or upstreams, delimit them with commas - - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],[service-name]:[port]:[optional datacenter]" - ``` - - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],prepared_query:[query name]:[port]" - ``` + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],prepared_query:[query name]:[port],[service-name].svc:[port]" + ``` - `consul.hashicorp.com/envoy-extra-args` - A space-separated list of [arguments](https://www.envoyproxy.io/docs/envoy/latest/operations/cli) to be passed to the injected envoy binary. From e33d4e3cb26c9cf075f5755fe3245683309a7696 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Mon, 20 Jun 2022 15:25:15 -0400 Subject: [PATCH 070/149] ReferenceGrant -> ReferencePolicy --- .../api-gateway/upgrade-specific-versions.mdx | 56 +++++++++---------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index a7312bde3..43cf27d5d 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -12,7 +12,7 @@ This topic describes how to upgrade Consul API Gateway. ## v0.3.0 -Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with a `secret` defined in a different namespace now require a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) that explicitly allows `Gateways` from the gateway's namesapce to use `Secrets` in the secret's namespace. +Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with a `secret` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows `Gateways` from the gateway's namesapce to use `Secrets` in the secret's namespace. ### Requirements @@ -23,7 +23,7 @@ Ensure that the following requirements are met prior to upgrading: - `kubectl` should be configured to point to the cluster containing the installation you are upgrading. - You should have the following permission rights on your Kubernetes cluster: - `Gateway.read` - - `ReferenceGrant.create` + - `ReferencePolicy.create` - (Optional) The [jq](https://stedolan.github.io/jq/download/) command line processor for JSON can be installed, which will ease route retrieval during the upgrade process. ### Procedure @@ -97,7 +97,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Inspect the `secret` entries for each of the routes. - If a `namespace` field is not defined in the `secret` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `secret`. Otherwise, note the `namespace` field values for `secret` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferenceGrant` that explicitly allows each cross-namespace secret-to-gateway pair to prevent the route from breaking (see [step 5](#create-secret-reference-grant)). + If a `namespace` field is not defined in the `secret` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `secret`. Otherwise, note the `namespace` field values for `secret` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace secret-to-gateway pair to prevent the route from breaking (see [step 5](#create-secret-reference-grant)). After completing this step, you will have a list of all secrets similar to the following: @@ -114,14 +114,14 @@ Ensure that the following requirements are met prior to upgrading: Proceed with the [standard-upgrade](#standard-upgrade) if your list is empty. -1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `Gateways` and filter for secrets that require a `ReferenceGrant`. +1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `Gateways` and filter for secrets that require a `ReferencePolicy`. ```shell-session $ kubectl get Gateway -o json -A | jq -r '.items[] | {gateway_name: .metadata.name, gateway_namespace: .metadata.namespace, kind: .kind, crossNamespaceSecrets: ( .metadata.namespace as $parentnamespace | .spec.listeners[] .tls.certificateRefs[] | select(.namespace != null and .namespace != $parentnamespace ) )} ' ``` - The output will resemble the following response if gateways that require a new `ReferenceGrant` are returned: + The output will resemble the following response if gateways that require a new `ReferencePolicy` are returned: @@ -144,20 +144,20 @@ Ensure that the following requirements are met prior to upgrading: If your output is empty, proceed with the [standard-upgrade](#standard-upgrade). -1. Using the list of secrets you created earlier as a guide, create a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) to allow each gateway cross namespace secret access. - The `ReferenceGrant` explicitly allows each cross-namespace gateway to secret pair. The `ReferenceGrant` must be created in the same `namespace` as the `Secret`. +1. Using the list of secrets you created earlier as a guide, create a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) to allow each gateway cross namespace secret access. + The `ReferencePolicy` explicitly allows each cross-namespace gateway to secret pair. The `ReferencePolicy` must be created in the same `namespace` as the `Secret`. - Skip to the next step if you've already created a `ReferenceGrant`. + Skip to the next step if you've already created a `ReferencePolicy`. - The following example `ReferenceGrant` enables `example-gateway` in `gateway-namespace` to utilize secrets in the `secret-namespace` namespace: + The following example `ReferencePolicy` enables `example-gateway` in `gateway-namespace` to utilize secrets in the `secret-namespace` namespace: - + ```yaml apiVersion: gateway.networking.k8s.io/v1alpha2 - kind: ReferenceGrant + kind: ReferencePolicy metadata: name: reference-grant namespace: secret-namespace @@ -173,20 +173,20 @@ Ensure that the following requirements are met prior to upgrading: -1. If you have already created a `ReferenceGrant`, modify it to allow your route and save it as `referencegrant.yaml`. Note that each `ReferenceGrant` only supports one `to` field and one `from` field (refer the [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencegrant/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferenceGrant`s. +1. If you have already created a `ReferencePolicy`, modify it to allow your route and save it as `referencepolicy.yaml`. Note that each `ReferencePolicy` only supports one `to` field and one `from` field (refer the [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencepolicy/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferencePolicy`s. 1. Issue the following command to apply it to your cluster: ```shell-session - $ kubectl apply --filename referencegrant.yaml + $ kubectl apply --filename referencepolicy.yaml ``` - Repeat this step as needed until each of your cross-namespace secrets have a corresponding `ReferenceGrant`. + Repeat this step as needed until each of your cross-namespace secrets have a corresponding `ReferencePolicy`. ## v0.2.0 -Consul API Gateway v0.2.0 introduces a breaking change for people upgrading from Consul API Gateway v0.1.0. Routes with a `backendRef` defined in a different namespace now require a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. +Consul API Gateway v0.2.0 introduces a breaking change for people upgrading from Consul API Gateway v0.1.0. Routes with a `backendRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. ### Requirements @@ -198,7 +198,7 @@ Ensure that the following requirements are met prior to upgrading: - You should have the following permission rights on your Kubernetes cluster: - `HTTPRoute.read` - `TCPRoute.read` - - `ReferenceGrant.create` + - `ReferencePolicy.create` - (Optional) The [jq](https://stedolan.github.io/jq/download/) command line processor for JSON can be installed, which will ease route retrieval during the upgrade process. ### Procedure @@ -273,7 +273,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Inspect the `backendRefs` entries for each of the routes. - If a `namespace` field is not defined in the `backendRef` or if the namespace matches the namespace of the route, then no additional action is required for the `backendRef`. Otherwise, note the `group`, `kind`, `name`, and `namespace` field values for `backendRef` configurations that have a `namespace` defined that do not match the namespace of the parent route. You must also note the `kind` and `namespace` of the parent route. You will need these to create a `ReferenceGrant` that explicitly allows each cross-namespace route-to-service pair to prevent the route from breaking (see [step 5](#create-reference-grant)). + If a `namespace` field is not defined in the `backendRef` or if the namespace matches the namespace of the route, then no additional action is required for the `backendRef`. Otherwise, note the `group`, `kind`, `name`, and `namespace` field values for `backendRef` configurations that have a `namespace` defined that do not match the namespace of the parent route. You must also note the `kind` and `namespace` of the parent route. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace route-to-service pair to prevent the route from breaking (see [step 5](#create-reference-grant)). After completing this step, you will have a list of all routes similar to the following: @@ -304,7 +304,7 @@ Ensure that the following requirements are met prior to upgrading: Proceed with [standard-upgrade](#standard-upgrade) if your list is empty. -1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `HTTPRoutes` and `TCPRoutes` and filter for routes that require a `ReferenceGrant`. +1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `HTTPRoutes` and `TCPRoutes` and filter for routes that require a `ReferencePolicy`. ```shell-session $ kubectl get HTTPRoute,TCPRoute -o json -A | jq -r '.items[] | {name: .metadata.name, namespace: .metadata.namespace, kind: .kind, crossNamespaceBackendReferences: ( .metadata.namespace as $parentnamespace | .spec.rules[] .backendRefs[] | select(.namespace != null and .namespace != $parentnamespace ) )} ' @@ -312,7 +312,7 @@ Ensure that the following requirements are met prior to upgrading: Note that the command retrieves all `HTTPRoutes` and `TCPRoutes`. `TLSRoutes` and `UDPRoutes` are not supported in v0.1.0. - The output will resemble the following response if routes that require a new `ReferenceGrant` are returned: + The output will resemble the following response if routes that require a new `ReferencePolicy` are returned: @@ -350,20 +350,20 @@ Ensure that the following requirements are met prior to upgrading: If your output is empty, proceed with the [standard-upgrade](#standard-upgrade). -1. Using the list of routes you created earlier as a guide, create a [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferenceGrant) to allow cross namespace traffic for each route service pair. - The `ReferenceGrant` explicitly allows each cross-namespace route to service pair to prevent the route from breaking. The `ReferenceGrant` must be created in the same `namespace` as the backend `Service`. +1. Using the list of routes you created earlier as a guide, create a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) to allow cross namespace traffic for each route service pair. + The `ReferencePolicy` explicitly allows each cross-namespace route to service pair to prevent the route from breaking. The `ReferencePolicy` must be created in the same `namespace` as the backend `Service`. - Skip to the next step if you've already created a `ReferenceGrant`. + Skip to the next step if you've already created a `ReferencePolicy`. - The following example `ReferenceGrant` enables `HTTPRoute` traffic from the `example-namespace` to Kubernetes Services in the `web-backend` namespace: + The following example `ReferencePolicy` enables `HTTPRoute` traffic from the `example-namespace` to Kubernetes Services in the `web-backend` namespace: - + ```yaml apiVersion: gateway.networking.k8s.io/v1alpha2 - kind: ReferenceGrant + kind: ReferencePolicy metadata: name: reference-grant namespace: gateway-namespace @@ -380,15 +380,15 @@ Ensure that the following requirements are met prior to upgrading: -1. If you have already created a `ReferenceGrant`, modify it to allow your route and save it as `referencegrant.yaml`. Note that each `ReferenceGrant` only supports one `to` field and one `from` field (refer the [`ReferenceGrant`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencegrant/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferenceGrant`s. +1. If you have already created a `ReferencePolicy`, modify it to allow your route and save it as `referencepolicy.yaml`. Note that each `ReferencePolicy` only supports one `to` field and one `from` field (refer the [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencepolicy/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferencePolicy`s. 1. Issue the following command to apply it to your cluster: ```shell-session - $ kubectl apply --filename referencegrant.yaml + $ kubectl apply --filename referencepolicy.yaml ``` - Repeat this step as needed until each of your cross-namespace routes have a corresponding `ReferenceGrant`. + Repeat this step as needed until each of your cross-namespace routes have a corresponding `ReferencePolicy`. ## Standard Upgrade From f6d0220af8848b9ca5fa9893efd8188272d54213 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Mon, 20 Jun 2022 12:38:21 -0700 Subject: [PATCH 071/149] incorporated examples from @cthain --- .../content/docs/ecs/terraform/install.mdx | 309 ++++++++---------- 1 file changed, 139 insertions(+), 170 deletions(-) diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index 7118ae12a..3a072c8cc 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -24,10 +24,11 @@ The following procedure describes the general workflow: If you want to operate Consul with ACLs enabled (recommended), follow the instructions in the [Secure Configuration](/docs/ecs/terraform/secure-configuration) documentation. ACLs provide network security for production-grade deployments. -## Prerequisites +## Requirements * You should have some familiarity with using Terraform. Refer to the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform. * You should also be familiar with AWS ECS before following these instructions. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for details. +* If you intend to [use the `gateway-task` module to deploy mesh gateways](#configure-the-gateway-task-module), all Consul server and client agents in all datacenters must have TLS and gossip encryption enabled (refer to [Encryption](/docs/security/encryption) for instructions). ## Create the task definition @@ -88,138 +89,6 @@ The following fields are required. Refer to the [module reference documentation] | `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. | | `port` | integer | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | | `retry_join` | list | This is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | - -## Configuring a Mesh Gateway - -The `gateway-task` Terraform module can be used to deploy mesh gateways to enable service to service communication across the WAN. -Mesh gateways can also be used to federate service mesh traffic across Consul admin partitions and Consul datacenters over the WAN. - -~> This topic requires familiarity with [mesh gateways](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters). - -Using the `gateway-task` module to deploy mesh gateways requires that all Consul server and client agents in all datacenters have TLS and gossip encryption enabled. -Mesh gateways operate by sniffing and extracting the server name indication (SNI) header from the service mesh session and routing the connection to the appropriate destination based on the server name requested. - -The following example shows a Terraform configuration that creates a mesh gateway task called `my-gateway` in a file called `mesh-gateway.tf`: - - - -```hcl -module "my_mesh_gateway" { - source = "hashicorp/consul-ecs/aws//modules/gateway-task" - version = "" - kind = "mesh-gateway" - - family = "my-gateway" - ecs_cluster_arn = "" - subnets = [""] - retry_join = ["
"] - consul_server_ca_cert_arn = "" - gossip_key_secret_arn = "" -} -``` - - - -The following fields are required. Refer to the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/gateway-task?tab=inputs) for a complete reference. - -| Input Variable | Type | Description | -| --------------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `source` | string | Must be set to the source location of the `gateway-task` module, `hashicorp/consul-ecs/aws//modules/gateway-task`. | -| `version` | string | Must be set to the version of the `gateway-task` module. | -| `kind` | string | The kind of gateway to create. Must be set to `"mesh-gateway"` to create a mesh-gateway. | -| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. | -| `ecs_cluster_arn` | string | ARN of the ECS cluster to deploy the mesh gateway task to. | -| `subnets` | list | The list of subnet IDs associated with the mesh gateway task. | -| `retry_join` | list | This is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul client agent, which specifies the locations of the Consul servers in the local datacenter. | -| `consul_server_ca_cert_arn` | string | ARN of the Secrets Manager secret that contains the Consul CA certificate. | -| `gossip_key_secret_arn` | string | ARN of the Secrets Manager secret that contains the Consul gossip encryption key. | - - -### Mesh Gateway Ingress - -To route traffic between datacenters, mesh gateways need to be reachable over the WAN. -Providing the `lb_enabled = true` flag will cause the `gateway-task` module to automatically deploy and configure a Network Load Balancer for ingress to the mesh-gateway. -You also need to provide the VPC identifier and at least one public subnet to associate with the load balancer. - - - -```hcl -module "my_mesh_gateway" { - ... - - lb_enabled = true - lb_vpc_id = "" - lb_subnets = [""] -} -``` - - - -Alternatively, you can manually configure ingress to the mesh gateway and provide the `wan_address` and `wan_port` inputs to the -`gateway-task` module. - -~> Mesh gateways route L4 TCP connections and do not terminate mTLS sessions. If you manually configure an Elastic Load Balancer for ingress to a mesh gateway you must use a Network Load Balancer or a Classic Load Balancer. - - - - -```hcl -module "my_mesh_gateway" { - ... - - wan_address = "" - wan_port = -} -``` - - - -The `wan_port` field is optional. If it is not provided, port `8443` is used by default. - -### ACLs - -The following example shows how to configure the `gateway-task` when ACLs are enabled. - - - -```hcl -module "my_mesh_gateway" { - ... - - acls = true - consul_http_addr = "" - consul_https_ca_cert_arn = "" -} -``` - - - -The `consul_http_addr` input is the HTTP `address:port` of the Consul server and is required for the mesh gateway task to log in to Consul via the IAM Auth Method to obtain its client and service tokens. -The `consul_https_ca_cert_arn` input is the ARN of the Secrets Manager secret that contains the certificate for the Consul HTTPS API. - - -### WAN Federation with Mesh Gateways - -The following example shows how to configure the `gateway-task` module to enable [WAN federation via mesh gateways](/docs/connect/gateways/mesh-gateway/wan-federation-via-mesh-gateways). - - - -```hcl -module "my_mesh_gateway" { - ... - - consul_datacenter = "" - consul_primary_datacenter = "" - enable_mesh_gateway_wan_federation = true - enable_acl_token_replication = true -} -``` - - - -~> When federating Consul datacenters over the WAN with ACLs enabled, [ACL Token replication](/docs/security/acl/acl-federated-datacenters) must be enabled on all server and client agents in all datacenters. - - ### Configure an ECS service for the mesh task module [ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) are one of the most common @@ -267,53 +136,55 @@ in their [`network_configuration`](https://registry.terraform.io/providers/hashi ### Configure the gateway task module -Add the `gateway-task` to your Terraform configuration if you want to federate multiple service meshes across Consul datacenters over the WAN. Refer to [WAN Federation via Mesh Gateways](/docs/connect/gateways/mesh-gateway/wan-federation-via-mesh-gateways) to learn more about the federation deployment model. +Add the `gateway-task` to your Terraform configuration if you want to deploy a mesh gateway. Mesh gateways enable service to service communication across the WAN, as well as federate service mesh traffic across Consul admin partitions and Consul datacenters over the WAN. Refer to the following documentation to learn more about mesh gateways: -You must add and configure a `gateway-task` for each Consul datacenter in your network. The module creates an ECS service and a task definition that includes the following containers: +* [WAN Federation via Mesh Gateways](/docs/connect/gateways/mesh-gateway/wan-federation-via-mesh-gateways) +* [Service-to-service Traffic Across Datacenters](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters). + +You must add and configure a `gateway-task` for each Consul datacenter in your network. You must also enable TLS and gossip encryption on all server and client agents in all data centers per the [Requirements](#requirements). Mesh gateways operate by sniffing and extracting the server name indication (SNI) header from the service mesh session and routing the connection to the appropriate destination based on the server name requested. + +The module creates an ECS service and a task definition that includes the following containers: * Consul client * Envoy gateway proxy * Mesh init -You will need to provide inputs for the artifacts created by the `gateway-task` module. +You will need to provide inputs for the artifacts created by the `gateway-task` module. The following example defines a mesh gateway task called `my-gateway` in a file called `mesh-gateway.tf`: -#### Task definition + -The `kind` parameter is the only required input. The value must be set to `mesh-gateway`. The following table describes optional inputs for the task definition. Refer to the [example gateway task configuration](#example-gateway-task-configuration) for a fully-configured task definition. +```hcl +module "my_mesh_gateway" { + source = "hashicorp/consul-ecs/aws//modules/gateway-task" + version = "" + kind = "mesh-gateway" + + family = "my-gateway" + ecs_cluster_arn = "" + subnets = [""] + retry_join = ["
"] + consul_server_ca_cert_arn = "" + gossip_key_secret_arn = "" +} +``` + + + +The following fields are required. Refer to the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/gateway-task?tab=inputs) for a complete reference. | Input variable | Type | Description | -| --- | --- | --- | -| `kind` | string | Specifies the kind of gateway to create. The value must be set to `mesh-gateway`. | -| `lan_address` | string | Specifies the LAN address for the gateway. The address is also used as the service address. Defaults to the node address. | -| `lan_port` | integer | Specifies the LAN port for the gateway. Also used as the service port. Defaults to `8443`. | -| `wan_address` | string | Specifies the WAN address for the gateway. Defaults to the `lan_address`.
If the `assign_public_ip` is set to `true`, the WAN address will be set to the public IP address.
If the `load_balancer_target_group_arn` is specified but no value for `wan_address` is provided, then the WAN address will be set to the load balancer’s DNS name.
To set a static WAN address, specify an explicit value for `wan_address` and `wan_port`. | -| `wan_port` | integer | Specifies the WAN port for the gateway. Defaults to the `lan_port`. | -| `family` | string | Specifies the ECS task definition family. The family is also used as the Consul service name by default. | -| `requires_compatibilities` | list of strings | Specifies one or more launch types required by the task. Defaults to `[“FARGATE”, “EC2”]` | -| `cpu` | integer | Specifies the number of CPUs used by the task. Defaults to `256`. | -| `memory` | integer | Specifies the mount (in MiB) of memory used by the task. Default is `512`. | -| `task_role` | object | Specifies the ECS task role to include in the task definition. If not provided, a role is created. Defaults to `{ "arn": null, "id": null }` | -| `execution_role` | object | Specifies the ECS execution role to include in the task definition. If not provided, a role is created. Defaults to `{ "arn": null, "id": null }` | -| `additional_task_role_policies` | list of strings | Specifies additional policy ARNs to attach to the task role. Default is `[]`. | -| `addition_task_execution_role_policies` | | | -| `log_configuration` | object | Specifies configurations for the task definition log. See [LogConfiguration](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LogConfiguration.html) in the AWS documentation. Default is `{}`.| -| `tags` | object | Defines tags to add to all resources. Default is `{}`.| -| `consul_agent_configuration` | string | Specifies the contents of a configuration file for the Consul agent in HCL format. Default is `""`. | -| `consul_datacenter` | string | Specifies the name of the Consul datacenter that the client belongs to. Default is `dc1`.| -| `consul_service_name` | string | Specifies the name for the service when it registers will Consul. Defaults to the task family name. | -| `consul_service_tags` | list of strings | Defines a list of tags to include in the Consul service registration. Default is `[]`.| -| `consul_service_meta` | object | Defines metadata to attach to the Consul service registration. | -| `consul_image` | string | Specifies the Consul Docker image to use. Default is `public.ecr.aws/hashicorp/consul:1.13.0` | -| `consul_ecs_image` | string | Specifies the Consul on ECS Docker image to use. Default is `public.ecr.aws/hashicorp/consul-ecs:0.6.0` | -| `consul_namespace` | string | Specifies which Consul namespace to register the service. Default is `default`.| -| `consul_partition` | string | Specifies which Consul admin partition to register the service. Default is `default`. | -| `envoy_image` | string | Specifies the name of the Envoy Docker image to use. Default is `envoyproxy/envoy-alpine:v1.21.2` | -| `retry_join` | list of strings | Defines a set of arguments to pass to the Consul agent [`-retry-join`](/docs/agent/config/cli-flags#_retry_join) flag. | -| `consul_server_ca_cert_arn` | string | Specifies the ARN of the Secrets Manager containing the Consul server CA certificate for Consul's internal remote procedure calls (RPC). | -| `consul_client_token_secret_arn` | | | +| --- | --- | --- | +| `source` | string | Specifies the source location of the `gateway-task` module. Must be set to `hashicorp/consul-ecs/aws//modules/gateway-task`. | +| `version` | string | Specifies the version of the `gateway-task` module. | +| `kind` | string | Declares the kind of gateway to create. Must be set to `mesh-gateway` to create a mesh-gateway. | +| `family` | string | Specifies the [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. | +| `ecs_cluster_arn` | string | Specifies the ARN of the ECS cluster where the mesh gateway task should be launched. | +| `subnets` | list of strings | Specifies the subnet IDs where the task will be launched. | +| `retry_join` | list of strings | Defines a set of arguments to pass to the Consul agent [`-retry-join`](/docs/agent/config/cli-flags#_retry_join) flag. The arguments specify locations of the Consul servers in the local datacenter that Consul client agents can connect to. | +| `consul_server_ca_cert_arn` | string | Specifies the ARN of the Secrets Manager containing the Consul server CA certificate | | `gossip_key_secret_arn` | string | Specifies the ARN of the Secrets Manager containing the Consul's gossip encryption key. | -| `acls` | Boolean | Set to `true` to enable Consul's [access control lists (ACLs)](/docs/security/acl/index). Default is `false`.| -| `acl_secret_name_prefix` | | | + +Refer to the [gateway task configuration examples](#gateway-task-configuration-examples) for additional example configurations. #### ECS service @@ -335,7 +206,105 @@ The following table describes the inputs for configuring the ECS service in your The `mesh-init` container is a short-lived container that sets up the initial configurations for Consul and Envoy (refer to [Task Startup](/docs/ecs/architecture#task-startup) for additional information). The `gateway-task` module automatically configures the `mesh-init` container based on the inputs specified in the [task definition](#task-definition) and [ECS service](#ecs-service) configuration. -### Example gateway task configuration +#### Gateway task configuration examples + +The following examples illustrate how to configure the `gateway-task` for different use cases. + +##### Ingress + +Mesh gateways need to be reachable over the WAN to route traffic between datacenters. Configure the following options in the `gateway-task` to enable ingress through the mesh gateway. + +| Option | Type | Description | +| --- | --- | --- | +| `lb_enabled` | Boolean | Set to `true` to automatically deploy and configure a network load balancer for ingress to the mesh gateway. | +| `lb_vpc_id` | string | Specifies the VPC in which to launch the load balancer. | +| `lb_subnets` | list of strings | Specifies one or more public subnets to associate with the load balancer. | + + + +```hcl +module "my_mesh_gateway" { + ... + + lb_enabled = true + lb_vpc_id = "" + lb_subnets = [""] +} +``` + + + +Alternatively, you can manually configure ingress to the mesh gateway and provide the `wan_address` and `wan_port` inputs to the `gateway-task` module. The `wan_port` field is optional. Port `8443` is used by default. + + + +```hcl +module "my_mesh_gateway" { + ... + + wan_address = "" + wan_port = +} +``` + + + +Mesh gateways route L4 TCP connections and do not terminate mTLS sessions. If you manually configure [AWS Elastic Load Balancing](https://aws.amazon.com/elasticloadbalancing/) for ingress to a mesh gateway, you must use an AWS [Network Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/introduction.html) or a [Classic Load Balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/classic/introduction.html). + + +#### ACLs + +Configure the following options in the `gateway-task` when ACLs are enabled. + +| Option | Type | Description | +| --- | --- | --- | +| `acl` | Boolean | Set to `true` if ACLs are enabled. | +| `consul_http_addr` | string | Specifies the HTTP `address:port` of the Consul server. Required for the mesh gateway task to log into Consul via the IAM Auth Method to obtain its client and service tokens. | +| `consul_https_ca_cert_arn` | string | Specifies ARN of the Secrets Manager secret that contains the certificate for the Consul HTTPS API. | + + + +```hcl +module "my_mesh_gateway" { + ... + + acls = true + consul_http_addr = "" + consul_https_ca_cert_arn = "" +} +``` + + + +#### WAN federation + +Configure the following options in the `gateway-task` to enable [WAN federation via mesh gateways](/docs/connect/gateways/mesh-gateway/wan-federation-via-mesh-gateways). + +| Option | Type | Description | +| --- | --- | --- | +| `consul_datacenter` | string | Specifies the name of the local Consul datacenter. | +| `consul_primary_datacenter` | string | Specifies the name of the primary Consul datacenter. | +| `enable_mesh_gateway_wan_federation` | Boolean | Set to `true` to enable WAN federation. | +| `enable_acl_token_replication` | Boolean | Set to `true` to enable ACL token replication and allow the creation of local tokens secondary datacenters. | + +The following example shows how to configure the `gateway-task` module. + + + +```hcl +module "my_mesh_gateway" { + ... + + consul_datacenter = "" + consul_primary_datacenter = "" + enable_mesh_gateway_wan_federation = true + enable_acl_token_replication = true +} +``` + + + +When federating Consul datacenters over the WAN with ACLs enabled, [ACL Token replication](/docs/security/acl/acl-federated-datacenters) must be enabled on all server and client agents in all datacenters. ## Run Terraform From 3ded4522fb04fb626572cfe52b0c5a399688500c Mon Sep 17 00:00:00 2001 From: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Date: Mon, 20 Jun 2022 15:28:50 -0500 Subject: [PATCH 072/149] Apply suggestions from code review Co-authored-by: Freddy --- website/content/docs/connect/cluster-peering/index.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index a94f258f8..73980ceb2 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -38,6 +38,7 @@ Regardless of whether you connect your clusters through WAN federation or cluste ## Technical preview constraints Not all features and functionality are available in the technical preview release. In particular, consider the following technical constraints: +- Consul ACLs must be disabled or the ACL `default_policy` must be set to `allow`. - Mesh gateways for _server to server traffic_ are not available. However, mesh gateways for _service to service traffic_ between clusters are available. - Services exported to peered clusters must not be configured as HTTP. - Support for dynamic routing such as splits, custom routes, or redirects is not available. From 49e0c16d4d4a17b6e3d7a4689ab16224c675335d Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Mon, 20 Jun 2022 16:30:58 -0400 Subject: [PATCH 073/149] Address surprise format of reference-grant --- .../docs/api-gateway/upgrade-specific-versions.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 43cf27d5d..03220d24c 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -159,7 +159,7 @@ Ensure that the following requirements are met prior to upgrading: apiVersion: gateway.networking.k8s.io/v1alpha2 kind: ReferencePolicy metadata: - name: reference-grant + name: reference-policy namespace: secret-namespace spec: from: @@ -273,7 +273,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Inspect the `backendRefs` entries for each of the routes. - If a `namespace` field is not defined in the `backendRef` or if the namespace matches the namespace of the route, then no additional action is required for the `backendRef`. Otherwise, note the `group`, `kind`, `name`, and `namespace` field values for `backendRef` configurations that have a `namespace` defined that do not match the namespace of the parent route. You must also note the `kind` and `namespace` of the parent route. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace route-to-service pair to prevent the route from breaking (see [step 5](#create-reference-grant)). + If a `namespace` field is not defined in the `backendRef` or if the namespace matches the namespace of the route, then no additional action is required for the `backendRef`. Otherwise, note the `group`, `kind`, `name`, and `namespace` field values for `backendRef` configurations that have a `namespace` defined that do not match the namespace of the parent route. You must also note the `kind` and `namespace` of the parent route. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace route-to-service pair to prevent the route from breaking (see [step 5](#create-reference-policy)). After completing this step, you will have a list of all routes similar to the following: @@ -348,7 +348,7 @@ Ensure that the following requirements are met prior to upgrading: If your output is empty, proceed with the [standard-upgrade](#standard-upgrade). - + 1. Using the list of routes you created earlier as a guide, create a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) to allow cross namespace traffic for each route service pair. The `ReferencePolicy` explicitly allows each cross-namespace route to service pair to prevent the route from breaking. The `ReferencePolicy` must be created in the same `namespace` as the backend `Service`. @@ -365,7 +365,7 @@ Ensure that the following requirements are met prior to upgrading: apiVersion: gateway.networking.k8s.io/v1alpha2 kind: ReferencePolicy metadata: - name: reference-grant + name: reference-policy namespace: gateway-namespace spec: from: From 8890e3ae696df787f6d801a4e31343eac705bb5f Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Mon, 20 Jun 2022 16:33:49 -0400 Subject: [PATCH 074/149] Fix broken link --- .../content/docs/api-gateway/upgrade-specific-versions.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 03220d24c..b6a475888 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -97,7 +97,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Inspect the `secret` entries for each of the routes. - If a `namespace` field is not defined in the `secret` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `secret`. Otherwise, note the `namespace` field values for `secret` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace secret-to-gateway pair to prevent the route from breaking (see [step 5](#create-secret-reference-grant)). + If a `namespace` field is not defined in the `secret` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `secret`. Otherwise, note the `namespace` field values for `secret` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace secret-to-gateway pair to prevent the route from breaking (see [step 5](#create-secret-reference-policy)). After completing this step, you will have a list of all secrets similar to the following: @@ -142,7 +142,7 @@ Ensure that the following requirements are met prior to upgrading: If your output is empty, proceed with the [standard-upgrade](#standard-upgrade). - + 1. Using the list of secrets you created earlier as a guide, create a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) to allow each gateway cross namespace secret access. The `ReferencePolicy` explicitly allows each cross-namespace gateway to secret pair. The `ReferencePolicy` must be created in the same `namespace` as the `Secret`. From c779d224e23fdcdfa9f1cc58d289e9295af7ad2b Mon Sep 17 00:00:00 2001 From: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Date: Mon, 20 Jun 2022 13:53:48 -0700 Subject: [PATCH 075/149] Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --- website/content/docs/ecs/enterprise.mdx | 10 +++---- website/content/docs/ecs/manual/install.mdx | 12 ++++----- .../docs/ecs/manual/secure-configuration.mdx | 26 +++++++++---------- .../ecs/terraform/secure-configuration.mdx | 4 +-- 4 files changed, 26 insertions(+), 26 deletions(-) diff --git a/website/content/docs/ecs/enterprise.mdx b/website/content/docs/ecs/enterprise.mdx index 7a6a53545..eaedba7e6 100644 --- a/website/content/docs/ecs/enterprise.mdx +++ b/website/content/docs/ecs/enterprise.mdx @@ -11,7 +11,7 @@ You can run Consul Enterprise on ECS by specifying the Consul Enterprise Docker ## Specify the Consul image -When instantiating the [`mesh-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) or [`gateway-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/gateway-task) module, +When you set up an instance of the [`mesh-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) or [`gateway-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/gateway-task) module, set the parameter `consul_image` to a Consul Enterprise image. The following example instructs the `mesh-task` module to import Consul Enterprise version 1.12.0: ```hcl @@ -63,10 +63,10 @@ If client support is required for any of the features, then you must use a Consu Consul on ECS supports [admin partitions](/docs/enterprise/admin-partitions) and [namespaces](/docs/enterprise/namespaces) when Consul Enterprise servers and clients are used. These features have the following requirements: -* ACLs must be enabled. -* ACL controller must run in the ECS cluster. -* `mesh-tasks` must use a Consul Enterprise client image. -* `gateway-task`s must use a Consul Enterprise client image. +- ACLs must be enabled. +- ACL controller must run in the ECS cluster. +- `mesh-task` must use a Consul Enterprise client image. +- `gateway-task` must use a Consul Enterprise client image. The ACL controller automatically manages ACL policies and token provisioning for clients and services on the service mesh. It also creates admin partitions and namespaces if they do not already exist. diff --git a/website/content/docs/ecs/manual/install.mdx b/website/content/docs/ecs/manual/install.mdx index 7f671bb19..7e20b3ea0 100644 --- a/website/content/docs/ecs/manual/install.mdx +++ b/website/content/docs/ecs/manual/install.mdx @@ -7,7 +7,7 @@ description: >- # Manual Installation -The following instructions describe how to manually create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) without Terraform. Refer to the [Consul ECS Terraform module](/docs/ecs/terraform/install) documentation for an alternative method for installing Consul on ECS. +The following instructions describe how to use the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) to manually create the ECS task definition without Terraform. If you prefer to use Terraform, refer to [Consul ECS Terraform module](/docs/ecs/terraform/install). If you intend to peer the service mesh to multiple Consul datacenters or partitions, you must use the Consul ECS Terraform module to install your service mesh on ECS. Manually configuring mesh gateways without using the `gateway-task` Terraform module is not supported. @@ -19,12 +19,12 @@ You should have some familiarity with AWS ECS. See [What is Amazon Elastic Conta ## Task Definition -Create a task definition configured to create the containers: +Configure a task definition that creates the containers: -* Your application container -* An Envoy sidecar-proxy container -* A Consul client container -* A `consul-ecs-mesh-init` container for service mesh setup +- Your application container +- An Envoy sidecar-proxy container +- A Consul client container +- A `consul-ecs-mesh-init` container for service mesh setup * (Optional) A `consul-ecs-health-sync` container to sync ECS health checks into Consul ## Top-level fields diff --git a/website/content/docs/ecs/manual/secure-configuration.mdx b/website/content/docs/ecs/manual/secure-configuration.mdx index fa96934be..02a4bd4a8 100644 --- a/website/content/docs/ecs/manual/secure-configuration.mdx +++ b/website/content/docs/ecs/manual/secure-configuration.mdx @@ -13,11 +13,11 @@ This topic describes how to enable Consul security features for your production The following features must be configured for your Consul server cluster: -* [TLS encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. -* [Gossip encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. -* [Access control lists (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. +- [TLS encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. +- [Gossip encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. +- [Access control lists (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. -You should already have followed the [installation instructions](/docs/ecs/manual/install) to understand how to define the necessary components of the task definition for Consul on ECS. +You should already have followed the [manual installation instructions](/docs/ecs/manual/install) to define the necessary components of the task definition for Consul on ECS. You should be familiar with [specifying sensitive data](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) on ECS. @@ -36,7 +36,7 @@ This section describes how to manually create ACL tokens. You can install the AC ### Define policies -Confiture the following ACL policy for the Consul client token: +Configure the following ACL policy for the Consul client token: ```hcl node_prefix "" { @@ -58,12 +58,12 @@ If you intend to create a gateway for connecting multiple Consul datacenters, yo Mesh gateways must run in the default namespace. ```hcl -namespace "default" { // If namespaces enabled +namespace "default" { ## If namespaces enabled service "" { policy = "write" } } -namespace_prefix "" { // If namespaces enabled +namespace_prefix "" { ## If namespaces enabled service_prefix "" { policy = "read" } @@ -78,8 +78,8 @@ agent_prefix "" { #### Terminating gateway policy ```hcl -partition "" { // If partitions enabled - namespace "" { // If namespaces enabled +partition "" { ## If partitions enabled + namespace "" { ## If namespaces enabled service "" { policy = "write" } @@ -93,8 +93,8 @@ partition "" { // If partitions enabled #### Ingress gateway policy ```hcl -partition "" { // If partitions enabled - namespace "" { // If namespaces enabled +partition "" { ## If partitions enabled + namespace "" { ## If namespaces enabled service "" { policy = "write" } @@ -114,12 +114,12 @@ partition "" { // If partitions enabled Create the Consul client token and the service tokens after adding the necessary policies. Service tokens should be associated with a service identity. The service identity includes `service:write` permissions for the service and sidecar proxy. You can create tokens using the [`consul acl token create`](/commands/acl/token/create) command or the [`[PUT] /v1/acl/token`](/api-docs/acl/tokens#create-a-token) API endpoint. -The following example shows how to use the Consul CLI to create a service token for a service named example-client-app: +The following example shows how to use the Consul CLI to create a service token for a service named `example-client-app`: ```shell-session $ consul acl token create -service-identity=example-client-app ... ``` -You will need to create one service token for each registered Consul service in ECS, including when new services are added to the service mesh. +You need to create one service token for each registered Consul service in ECS. When you add new services to the service mesh, you must create new tokens for each service. ## Secret storage diff --git a/website/content/docs/ecs/terraform/secure-configuration.mdx b/website/content/docs/ecs/terraform/secure-configuration.mdx index 9489ee14b..521141da8 100644 --- a/website/content/docs/ecs/terraform/secure-configuration.mdx +++ b/website/content/docs/ecs/terraform/secure-configuration.mdx @@ -66,7 +66,7 @@ module "acl_controller" { ``` The `name_prefix` parameter is used to prefix any secrets that the ACL controller will -update in AWS Secrets Manager. The `name_prefix` parameter value must be unique for each ECS cluster where you are deploying this controller. +update in AWS Secrets Manager. The `name_prefix` parameter value must be unique for each ECS cluster where you deploy this controller. ## Deploy your services @@ -74,7 +74,7 @@ Follow the instructions described in [Create a task definition](/docs/ecs/terraf ### Create an AWS Secrets Manager secret -The secret stores the gossip encryption key that the Consul clients will use. +The secret stores the gossip encryption key that the Consul clients use. From 1bd3909a716e4a867b70a259892577a7c4661a4d Mon Sep 17 00:00:00 2001 From: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Date: Mon, 20 Jun 2022 14:11:05 -0700 Subject: [PATCH 076/149] Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> --- .../content/docs/ecs/terraform/install.mdx | 43 ++++++++++--------- 1 file changed, 22 insertions(+), 21 deletions(-) diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index 3a072c8cc..10950eff9 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -9,7 +9,7 @@ description: >- This topic describes how to use HashiCorp’s Terraform modules to launch your application in AWS ECS as part of Consul service mesh. If you do not use Terraform, refer to the [Manual Installation](/docs/ecs/manual-installation) page to install Consul on ECS without Terraform. -This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. +This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the guides in the [Getting Started](/docs/ecs#getting-started) section for complete and runnable examples. ## Overview @@ -17,8 +17,8 @@ The following procedure describes the general workflow: 1. Create Terraform configuration files for the necessary components: - * [ECS task definition](#create-the-task-definition): Use the HashiCorp Terraform modules to create the ECS task definition. - * [ECS service](#ecs-service): Use the `aws_ecs_service` resource to create an ECS service that schedules service mesh tasks to run on ECS. + - [ECS task definition](#create-the-task-definition): Use the HashiCorp Terraform modules to create the ECS task definition. + - [ECS service](#ecs-service): Use the `aws_ecs_service` resource to create an ECS service that schedules service mesh tasks to run on ECS. 2. [Run Terraform](#running-terraform) to deploy the resources in AWS @@ -32,17 +32,17 @@ If you want to operate Consul with ACLs enabled (recommended), follow the instru ## Create the task definition -To run an application in ECS with Consul service mesh, you must create an ECS task definition. The task defintion includes your application container(s) and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container. +To run an application in ECS with Consul service mesh, you must create an ECS task definition. The task definition includes your application containers and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container. Create a Terraform configuration file and include the `mesh-task` module. The module automatically adds the necessary sidecar containers. -If you intend to peer the service mesh to multiple Consul datacenters or partitions, you can also include the `gateway-task` module. The `gateway-task` enables connectivity between datacenters across service meshes. +If you intend to peer the service mesh to multiple Consul datacenters or partitions, you can also include the `gateway-task` module. The module enables connectivity between datacenters across service meshes. ### Configure the mesh task module -Create a Terraform configuration file (e.g., `mesh-task.tf`) and specify the `mesh-task` module in the `source` field. The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers. +Create a Terraform configuration file and specify the `mesh-task` module in the `source` field. The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) automatically includes the necessary sidecar containers. -In the following example, the a Terraform configuration file called `mesh-task.tf` creates a task definition with an application container called `example-client-app`: +In the following example, the Terraform configuration file called `mesh-task.tf` creates a task definition with an application container called `example-client-app`: @@ -116,19 +116,18 @@ resource "aws_ecs_service" "my_task" { -This is a partial configuration to highlight some important fields. -See the [`aws_ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) documentation for a complete reference. +The example shows a partially configured ECS service to highlight significant fields. Refer to [`aws_ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) for a complete configuration reference. | Input Variable | Type | Description | | ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------- | -| `name` | string | The name of the ECS service. This is required by AWS but is not used by Consul service mesh. | -| `task_definition` | string | The task definition used to start tasks. Set this to the task definition ARN returned by the `mesh-task` module. | +| `name` | string | The name of the ECS service. This name is required by AWS but is not used by Consul service mesh. | +| `task_definition` | string | The task definition used to start tasks. Set this option to the task definition ARN returned by the `mesh-task` module. | | `launch_type` | string | The launch type. Consul on ECS supports the `FARGATE` and `EC2` launch types. | -| `propagate_tags` | string | This must be set to `TASK_DEFINITION` so that tags added by `mesh-task` to the task definition are copied to tasks. | +| `propagate_tags` | string | This option must be set to `TASK_DEFINITION` so that tags added by `mesh-task` to the task definition are copied to tasks. | After including the ECS service in your Terraform configuration, run `terraform apply` -from your project directory to create the ECS service resource. The ECS service will -soon start your application in a task. The task will automatically register itself +from your project directory to create the ECS service resource. The ECS service +then starts your application in a task. The task automatically registers itself into the Consul service catalog during startup. -> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true` @@ -145,9 +144,9 @@ You must add and configure a `gateway-task` for each Consul datacenter in your n The module creates an ECS service and a task definition that includes the following containers: -* Consul client -* Envoy gateway proxy -* Mesh init +- Consul client +- Envoy gateway proxy +- Mesh init You will need to provide inputs for the artifacts created by the `gateway-task` module. The following example defines a mesh gateway task called `my-gateway` in a file called `mesh-gateway.tf`: @@ -197,14 +196,16 @@ The following table describes the inputs for configuring the ECS service in your | `ecs_cluster_arn` | string | Specifies the ECS cluster where tasks should be launched. | | `launch_type` | string | Specifies the ECS service launch type. Can be either `fargate` or `ec2`. | | `desired_count` | integer | Specifies the number instances for the service to create. Defaults to `0`. | -| `subnets` | string | Specifies the subnet IDs where the tasks will launch. | +| `subnets` | string | Specifies the subnet IDs where the tasks launch. | | `security_group_ids` | string | Specifies the security group IDs to assign to the task ENI. | | `assign_public_ip` | Boolean | Set to `true` to create a task accessible at a public IP address. Default is `false`.
If set to `true` and `wan_address` is not configured, the WAN address will be set to the public IP of the task. | -| `load_balancer_target_group_arn` | string | Specifies the ARN of an existing load balancer target group. The load balancer target group allows ingress to the gateway task.
No additional load balancer configuration is necessary. Only NLBs and ALBs are supported. The container name and port will be automatically set based on other fields. | +| `load_balancer_target_group_arn` | string | Specifies the ARN of an existing load balancer target group. The load balancer target group allows ingress to the gateway task.
No additional load balancer configuration is necessary. Only NLBs and ALBs are supported. The container name and port are set automatically based on other fields. | #### Mesh init -The `mesh-init` container is a short-lived container that sets up the initial configurations for Consul and Envoy (refer to [Task Startup](/docs/ecs/architecture#task-startup) for additional information). The `gateway-task` module automatically configures the `mesh-init` container based on the inputs specified in the [task definition](#task-definition) and [ECS service](#ecs-service) configuration. +The `mesh-init` container is a short-lived container that sets up the initial configurations for Consul and Envoy. The `gateway-task` module automatically configures the `mesh-init` container based on the inputs specified in the [task definition](#task-definition) and [ECS service](#ecs-service) configuration. + +For additional information, refer to [Task Startup](/docs/ecs/architecture#task-startup) for additional information. #### Gateway task configuration examples @@ -217,7 +218,7 @@ Mesh gateways need to be reachable over the WAN to route traffic between datacen | Option | Type | Description | | --- | --- | --- | | `lb_enabled` | Boolean | Set to `true` to automatically deploy and configure a network load balancer for ingress to the mesh gateway. | -| `lb_vpc_id` | string | Specifies the VPC in which to launch the load balancer. | +| `lb_vpc_id` | string | Specifies the VPC to launch the load balancer in. | | `lb_subnets` | list of strings | Specifies one or more public subnets to associate with the load balancer. | From cfd9e2e41da895a22f2ca75899b50ed990b6f3dc Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Mon, 20 Jun 2022 14:14:18 -0700 Subject: [PATCH 077/149] fixed links to TF install examples --- website/content/docs/ecs/terraform/install.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index 10950eff9..d52aeeaf4 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -189,7 +189,7 @@ Refer to the [gateway task configuration examples](#gateway-task-configuration-e The ECS service is created as part of the `gateway-task` module configuration. The service can run one or more instances of the gateway. -The following table describes the inputs for configuring the ECS service in your Terraform configuration file. All inputs are required. Refer to the [example gateway task configuration](#example-gateway-task-configuration) for a fully-configured task definition. +The following table describes the inputs for configuring the ECS service in your Terraform configuration file. All inputs are required. Refer to the [gateway task configuration examples](#gateway-task-configuration-examples) for a fully-configured task definition. | Input variable | Type | Description | | --- | --- | --- | From b724f8b3ab7255b7a5d86db86389dbbb362d67b9 Mon Sep 17 00:00:00 2001 From: David Yu Date: Mon, 20 Jun 2022 16:07:38 -0700 Subject: [PATCH 078/149] docs: Use "error" to use standard log level value (#13507) * docs: Use "error" to use standard log level value --- website/content/commands/monitor.mdx | 2 +- website/content/commands/snapshot/agent.mdx | 2 +- website/content/docs/agent/config/cli-flags.mdx | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/website/content/commands/monitor.mdx b/website/content/commands/monitor.mdx index 51899cdc5..58abbeede 100644 --- a/website/content/commands/monitor.mdx +++ b/website/content/commands/monitor.mdx @@ -32,6 +32,6 @@ Usage: `consul monitor [options]` - `-log-level` - The log level of the messages to show. By default this is "info". This log level can be more verbose than what the agent is configured to run at. Available log levels are "trace", "debug", "info", - "warn", and "err". + "warn", and "error". - `-log-json` - Toggles whether the messages are streamed in JSON format. By default this is false. diff --git a/website/content/commands/snapshot/agent.mdx b/website/content/commands/snapshot/agent.mdx index f70525389..136a0f065 100644 --- a/website/content/commands/snapshot/agent.mdx +++ b/website/content/commands/snapshot/agent.mdx @@ -229,7 +229,7 @@ if desired. provided, this will be disabled. Defaults to "72h". - `-log-level` - Controls verbosity of snapshot agent logs. Valid options are - "TRACE", "DEBUG", "INFO", "WARN", "ERR". Defaults to "INFO". + "trace", "debug", "info", "warn", "error". Defaults to "info". - `-service` - The service name to used when registering the agent with Consul. Registering helps monitor running agents and the leader registers an additional diff --git a/website/content/docs/agent/config/cli-flags.mdx b/website/content/docs/agent/config/cli-flags.mdx index b55f44129..caf3b6444 100644 --- a/website/content/docs/agent/config/cli-flags.mdx +++ b/website/content/docs/agent/config/cli-flags.mdx @@ -468,7 +468,7 @@ information. - `-log-level` ((#\_log_level)) - The level of logging to show after the Consul agent has started. This defaults to "info". The available log levels are - "trace", "debug", "info", "warn", and "err". You can always connect to an agent + "trace", "debug", "info", "warn", and "error". You can always connect to an agent via [`consul monitor`](/commands/monitor) and use any log level. Also, the log level can be changed during a config reload. From 8aee4fecb5c491423dc3e2420e69f0b7113febf4 Mon Sep 17 00:00:00 2001 From: Jeff-Apple <79924108+Jeff-Apple@users.noreply.github.com> Date: Mon, 20 Jun 2022 17:00:11 -0700 Subject: [PATCH 079/149] API GW 0.3 Release Notes and updated Tech Specs --- .../content/docs/api-gateway/tech-specs.mdx | 5 +- .../consul-api-gateway/v0_2_x.mdx | 6 +- .../consul-api-gateway/v0_3_x.mdx | 59 +++++++++++++++++++ website/data/docs-nav-data.json | 8 ++- 4 files changed, 71 insertions(+), 7 deletions(-) create mode 100644 website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx diff --git a/website/content/docs/api-gateway/tech-specs.mdx b/website/content/docs/api-gateway/tech-specs.mdx index bfd9e893c..e49934919 100644 --- a/website/content/docs/api-gateway/tech-specs.mdx +++ b/website/content/docs/api-gateway/tech-specs.mdx @@ -18,9 +18,10 @@ Verify that your environment meets the following requirements prior to using Con Your datacenter must meet the following requirements prior to configuring the Consul API Gateway: - Kubernetes 1.21+ + - Kubernetes 1.24 is not supported at this time. - `kubectl` 1.21+ -- Consul 1.12.0+ -- HashiCorp Consul Helm chart 0.43.0+ +- Consul 1.11.2+ +- HashiCorp Consul Helm chart 0.45.0+ - Consul Service Mesh must be deployed on the Kubernetes cluster that API Gateway is deployed on. - Envoy: Envoy proxy support is determined by the Consul version deployed. Refer to [Envoy Integration](/docs/connect/proxies/envoy) for details. diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_2_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_2_x.mdx index cb5370b3f..3643b7262 100644 --- a/website/content/docs/release-notes/consul-api-gateway/v0_2_x.mdx +++ b/website/content/docs/release-notes/consul-api-gateway/v0_2_x.mdx @@ -18,9 +18,9 @@ description: >- was reachable, it didn’t allow users to set the more granular restrictions or permissions that they may expect. - This version of API Gateway implements Cross Namespace Reference Policies - and requires them when routes are in a different namespace than the services - (as specified by the `backendRef`) they are routing traffic to. + This version of API Gateway implements Cross Namespace Reference Policies + and requires them when routes are in a different namespace than the services + (as specified by the `backendRef`) they are routing traffic to. ## Supported Software diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx new file mode 100644 index 000000000..1d177d008 --- /dev/null +++ b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx @@ -0,0 +1,59 @@ +--- +layout: docs +page_title: 0.3.x +description: >- + Consul API Gateway release notes for version 0.3.x +--- + +# Consul API Gateway 0.3.0 + +## Release Highlights + +- **Improved High Availability and Scalability via Multiple Instances per Gateway:** + This release introduces support for deploying multiple instances of a logical + gateway (i.e. a K8s `gateway` config object). You can use a external load + balancing service to distribute incoming traffic across the instances of a + gateway. This provides higher availability of gateway services and also allows + a single logical gateway to handle a volume of network traffic that is much + greater than what a single instance of a gateway can. + + Administrators can set the number of instances independently for each logical + gateway. Administrators can also set, in the `gatewayClassConfig`, the + minimum, maximum and default number of gateway instances. These are applied + per `gateway` to all `gateways` in that `gatewayClass`. The minimum and + maximum settings will override settings configured on any given `gateway`. + + Note: There is a hard limit of eight (8) instances per `gateway` and it can + not be overridden. This hard limit will probably be increased in the future. + +- **Cross Namespace Reference Policies for Certificates:** + Reference policies is a security feature introduced in the 0.2 release. In + this release, Reference Policies are now required for TLS certificates when a + `listener` has a `certificateRef` that is in a different `namespace`. + + +## Supported Software + +- Consul 1.11.2+ +- HashiCorp Consul Helm chart 0.45.0+ +- Kubernetes 1.21+ + - Kubernetes 1.24 is not supported at this time. +- Kubectl 1.21+ +- Envoy proxy support is determined by the Consul version deployed. Refer to + [Envoy Integration](/docs/connect/proxies/envoy) for details. + +## Kubernetes Gateway API Specification + +Supported version of the Gateway API spec: `v1alpha2`(v0.4.1) + +## Upgrading + +For detailed information on upgrading, please refer to the [upgrade details page](/docs/api-gateway/upgrade-specific-versions) + +## Change logs + +The changelogs for this major release version and any maintenance versions are listed below. + +-> **Note**: These links will take you to the changelogs on the GitHub website. + +- [0.3.0](https://github.com/hashicorp/consul-api-gateway/releases/tag/v0.3.0) diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index f009f5611..3785b4182 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -1232,12 +1232,16 @@ "title": "Consul API Gateway", "routes": [ { - "title": "v0.1.x", - "path": "release-notes/consul-api-gateway/v0_1_x" + "title": "v0.3.x", + "path": "release-notes/consul-api-gateway/v0_3_x" }, { "title": "v0.2.x", "path": "release-notes/consul-api-gateway/v0_2_x" + }, + { + "title": "v0.1.x", + "path": "release-notes/consul-api-gateway/v0_1_x" } ] }, From 6158df1d538b219179b08db88e81734274359705 Mon Sep 17 00:00:00 2001 From: Nitya Dhanushkodi Date: Mon, 20 Jun 2022 17:15:33 -0700 Subject: [PATCH 080/149] Apply suggestions from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- .../docs/k8s/annotations-and-labels.mdx | 46 +++++++------------ 1 file changed, 16 insertions(+), 30 deletions(-) diff --git a/website/content/docs/k8s/annotations-and-labels.mdx b/website/content/docs/k8s/annotations-and-labels.mdx index 9f5608fba..e8b55dc31 100644 --- a/website/content/docs/k8s/annotations-and-labels.mdx +++ b/website/content/docs/k8s/annotations-and-labels.mdx @@ -72,41 +72,32 @@ Resource annotations could be used on the Kubernetes pod to control connect-inje - There are a few formats this annotation can take: - Unlabeled: - The unlabeled annotation format supports specifying service name, Consul Enterprise Namespaces and Partitions, - and datacenter. To use [cluster peering](/docs/connect/cluster-peering/k8s) with upstreams, use the labeled format below. - - Service Name - To specify the upstream service, you can provide the name of the service, and optionally, it's datacenter. + Use the unlabeled annotation format to specify a service name, Consul Enterprise namespaces and partitions, + and datacenters. To use [cluster peering](/docs/connect/cluster-peering/k8s) with upstreams, use the following labeled format. + - Service name: Place the service name at the beginning of the annotation to specify the upstream service. You can also append the datacenter where the service is deployed (optional). ```yaml annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter]" ``` - - Consul Enterprise Namespace - When using Consul Enterprise Namespaces [1.7+], your upstream services may be running in different namespaces. - The upstream namespace can be specified after the service name as `[service-name].[namespace]`. See [Consul - Enterprise Namespaces](#consul-enterprise-namespaces) below for more details on configuring the injector. + - Namespace (requires Consul Enterprise 1.7+): Upstream services may be running in different a namespace. Place the upstream namespace after the service name. Refer to [Consul Enterprise Namespaces](#consul-enterprise-namespaces) for additional details about configuring the injector. ```yaml annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace]:[port]:[optional datacenter]" ``` - -> **NOTE:** If the namespace is not specified it will default to the namespace - of the source service. + If the namespace is not specified, the annotation will default to the namespace of the source service. - ~> **WARNING:** Setting a namespace when not using Consul Enterprise or using a version < 1.7 - is not supported. It will be treated as part of the service name. + If you are not using Consul Enterprise 1.7+, Consul will interpret the value placed in the namespace position as part of the service name. - - Consul Enterprise Admin Partition - When using Consul Enterprise Admin Partitions [1.11+], your upstream services may be running in a different - partition. You must specify the namespace when specifying a partition, and the datacenter must be your local - datacenter if specified. Communicating across partitions using this method is only supported within a + - Admin partitions (requires Consul Enterprise 1.11+): Upstream services may be running in a different + partition. You must specify the namespace when specifying a partition. Place the partition name after the namespace. If you specify the name of the datacenter (optional), it must be the local datacenter. Communicating across partitions using this method is only supported within a datacenter. For cross partition communication across datacenters, see [cluster peering](/docs/connect/cluster-peering/k8s). ```yaml annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace].[service-partition]:[port]:[optional datacenter]" ``` - - [Prepared Query](/docs/connect/proxies#dynamic-upstreams-require-native-integration): - Prepared query upstreams can use the format below. + - [Prepared queries](/docs/connect/proxies#dynamic-upstreams-require-native-integration): Prepend the annotation with `prepared_query` and place the name of the query at the beginning of the string. ```yaml annotations: 'consul.hashicorp.com/connect-service-upstreams': 'prepared_query:[query name]:[port]' @@ -114,15 +105,13 @@ Resource annotations could be used on the Kubernetes pod to control connect-inje - Labeled (requires consul-k8s v0.45.0+): The labeled format is required when using the cluster peering feature and specifying an upstream in another - peer. It also supports specifying Consul Enterprise Namespace, Partition, or Datacenter. Only one of peer, - datacenter, or partition can be specified. - - Service Name - To specify the upstream service, you can provide the name of the service, followed by `.svc`. + peer. You can specify a Consul Enterprise namespace, partition, or datacenter. The format supports only one peer, datacenter, or partition. + - Service name: Place the service name at the beginning of the annotation followed by `.svc` to specify the upstream service. ```yaml annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc:[port]" ``` - - Peer or Datacenter + - Peer or datacenter: Place the peer or datacenter after `svc.` followed by either `peer` or `dc` and the port number. ```yaml annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-peer].peer:[port]" @@ -131,14 +120,13 @@ Resource annotations could be used on the Kubernetes pod to control connect-inje annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-dc].dc:[port]" ``` - - Consul Enterprise - You can specify a Consul Enterprise Namespace + - Namespace (required Consul Enterprise): Place the namespace after `svc.` followed by `ns` and the port number. ```yaml annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns:[port]" ``` - When specifying a peer, datacenter or Consul Enterprise Admin Partition when namespaces are enabled, you must - provide the namespace and one of (peer, datacenter, partition). + When specifying a peer, datacenter, or admin partition when namespaces are enabled, you must + provide the namespace and a peer, datacenter, or partition. ```yaml annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-peer].peer:[port]" @@ -152,9 +140,7 @@ Resource annotations could be used on the Kubernetes pod to control connect-inje "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-dc].dc:[port]" ``` - - Multiple Upstreams - - If you would like to specify multiple services or upstreams, delimit them with commas. They can be of any of the Unlabeled, Labeled, or Prepared Query formats above when using the supported versions for the formats. + - Multiple upstreams: Delimit multiple services or upstreams with commas. You can specify any of the unlabeled, labeled, or prepared query formats when using the supported versions for the formats. ```yaml annotations: From 190344a0ff451bdbbd0e960f771a3fd021803517 Mon Sep 17 00:00:00 2001 From: Nitya Dhanushkodi Date: Mon, 20 Jun 2022 17:21:55 -0700 Subject: [PATCH 081/149] un nest the sections --- .../docs/k8s/annotations-and-labels.mdx | 131 +++++++++--------- 1 file changed, 66 insertions(+), 65 deletions(-) diff --git a/website/content/docs/k8s/annotations-and-labels.mdx b/website/content/docs/k8s/annotations-and-labels.mdx index e8b55dc31..1e4d5533c 100644 --- a/website/content/docs/k8s/annotations-and-labels.mdx +++ b/website/content/docs/k8s/annotations-and-labels.mdx @@ -67,27 +67,27 @@ Resource annotations could be used on the Kubernetes pod to control connect-inje - `consul.hashicorp.com/connect-service-upstreams` - The list of upstream services that this pod needs to connect to via Connect along with a static local port to listen for those connections. When transparent proxy is enabled, - this annotation is optional. + this annotation is optional. There are a few formats this annotation can take: - - There are a few formats this annotation can take: - - - Unlabeled: - Use the unlabeled annotation format to specify a service name, Consul Enterprise namespaces and partitions, - and datacenters. To use [cluster peering](/docs/connect/cluster-peering/k8s) with upstreams, use the following labeled format. - - Service name: Place the service name at the beginning of the annotation to specify the upstream service. You can also append the datacenter where the service is deployed (optional). - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter]" - ``` - - - Namespace (requires Consul Enterprise 1.7+): Upstream services may be running in different a namespace. Place the upstream namespace after the service name. Refer to [Consul Enterprise Namespaces](#consul-enterprise-namespaces) for additional details about configuring the injector. - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace]:[port]:[optional datacenter]" - ``` - If the namespace is not specified, the annotation will default to the namespace of the source service. - - If you are not using Consul Enterprise 1.7+, Consul will interpret the value placed in the namespace position as part of the service name. + - Unlabeled: + Use the unlabeled annotation format to specify a service name, Consul Enterprise namespaces and partitions, and + datacenters. To use [cluster peering](/docs/connect/cluster-peering/k8s) with upstreams, use the following + labeled format. + - Service name: Place the service name at the beginning of the annotation to specify the upstream service. You can + also append the datacenter where the service is deployed (optional). + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter]" + ``` + - Namespace (requires Consul Enterprise 1.7+): Upstream services may be running in different a namespace. Place + the upstream namespace after the service name. Refer to [Consul Enterprise + Namespaces](#consul-enterprise-namespaces) for additional details about configuring the injector. + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace]:[port]:[optional datacenter]" + ``` + If the namespace is not specified, the annotation will default to the namespace of the source service. + If you are not using Consul Enterprise 1.7+, Consul will interpret the value placed in the namespace position as part of the service name. - Admin partitions (requires Consul Enterprise 1.11+): Upstream services may be running in a different partition. You must specify the namespace when specifying a partition. Place the partition name after the namespace. If you specify the name of the datacenter (optional), it must be the local datacenter. Communicating across partitions using this method is only supported within a @@ -97,60 +97,61 @@ Resource annotations could be used on the Kubernetes pod to control connect-inje annotations: "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace].[service-partition]:[port]:[optional datacenter]" ``` - - [Prepared queries](/docs/connect/proxies#dynamic-upstreams-require-native-integration): Prepend the annotation with `prepared_query` and place the name of the query at the beginning of the string. + - [Prepared queries](/docs/connect/proxies#dynamic-upstreams-require-native-integration): Prepend the annotation + with `prepared_query` and place the name of the query at the beginning of the string. + ```yaml + annotations: + 'consul.hashicorp.com/connect-service-upstreams': 'prepared_query:[query name]:[port]' + ``` + + - Labeled (requires consul-k8s v0.45.0+): + The labeled format is required when using the cluster peering feature and specifying an upstream in another + peer. You can specify a Consul Enterprise namespace, partition, or datacenter. The format supports only one peer, datacenter, or partition. + - Service name: Place the service name at the beginning of the annotation followed by `.svc` to specify the upstream service. ```yaml annotations: - 'consul.hashicorp.com/connect-service-upstreams': 'prepared_query:[query name]:[port]' + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc:[port]" ``` - - - Labeled (requires consul-k8s v0.45.0+): - The labeled format is required when using the cluster peering feature and specifying an upstream in another - peer. You can specify a Consul Enterprise namespace, partition, or datacenter. The format supports only one peer, datacenter, or partition. - - Service name: Place the service name at the beginning of the annotation followed by `.svc` to specify the upstream service. - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc:[port]" - ``` - - Peer or datacenter: Place the peer or datacenter after `svc.` followed by either `peer` or `dc` and the port number. - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-peer].peer:[port]" - ``` - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-dc].dc:[port]" - ``` - - Namespace (required Consul Enterprise): Place the namespace after `svc.` followed by `ns` and the port number. - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns:[port]" - ``` - When specifying a peer, datacenter, or admin partition when namespaces are enabled, you must - provide the namespace and a peer, datacenter, or partition. - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-peer].peer:[port]" - ``` - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-partition].ap:[port]" - ``` - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-dc].dc:[port]" - ``` - - - Multiple upstreams: Delimit multiple services or upstreams with commas. You can specify any of the unlabeled, labeled, or prepared query formats when using the supported versions for the formats. - + - Peer or datacenter: Place the peer or datacenter after `svc.` followed by either `peer` or `dc` and the port number. ```yaml annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],[service-name]:[port]:[optional datacenter]" + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-peer].peer:[port]" ``` - ```yaml annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],prepared_query:[query name]:[port],[service-name].svc:[port]" + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-dc].dc:[port]" ``` + - Namespace (required Consul Enterprise): Place the namespace after `svc.` followed by `ns` and the port number. + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns:[port]" + ``` + When specifying a peer, datacenter, or admin partition when namespaces are enabled, you must + provide the namespace and a peer, datacenter, or partition. + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-peer].peer:[port]" + ``` + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-partition].ap:[port]" + ``` + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].svc.[service-namespace].ns.[service-dc].dc:[port]" + ``` + + - Multiple upstreams: Delimit multiple services or upstreams with commas. You can specify any of the unlabeled, labeled, or prepared query formats when using the supported versions for the formats. + + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],[service-name]:[port]:[optional datacenter]" + ``` + + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name]:[port]:[optional datacenter],prepared_query:[query name]:[port],[service-name].svc:[port]" + ``` - `consul.hashicorp.com/envoy-extra-args` - A space-separated list of [arguments](https://www.envoyproxy.io/docs/envoy/latest/operations/cli) to be passed to the injected envoy binary. From d0f7152d1f53328173998326e4b2f93291b1d2b1 Mon Sep 17 00:00:00 2001 From: Nitya Dhanushkodi Date: Mon, 20 Jun 2022 17:28:19 -0700 Subject: [PATCH 082/149] unnest partitions --- .../content/docs/k8s/annotations-and-labels.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/website/content/docs/k8s/annotations-and-labels.mdx b/website/content/docs/k8s/annotations-and-labels.mdx index 1e4d5533c..c036bb77e 100644 --- a/website/content/docs/k8s/annotations-and-labels.mdx +++ b/website/content/docs/k8s/annotations-and-labels.mdx @@ -89,14 +89,14 @@ Resource annotations could be used on the Kubernetes pod to control connect-inje If the namespace is not specified, the annotation will default to the namespace of the source service. If you are not using Consul Enterprise 1.7+, Consul will interpret the value placed in the namespace position as part of the service name. - - Admin partitions (requires Consul Enterprise 1.11+): Upstream services may be running in a different - partition. You must specify the namespace when specifying a partition. Place the partition name after the namespace. If you specify the name of the datacenter (optional), it must be the local datacenter. Communicating across partitions using this method is only supported within a - datacenter. For cross partition communication across datacenters, see [cluster - peering](/docs/connect/cluster-peering/k8s). - ```yaml - annotations: - "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace].[service-partition]:[port]:[optional datacenter]" - ``` + - Admin partitions (requires Consul Enterprise 1.11+): Upstream services may be running in a different + partition. You must specify the namespace when specifying a partition. Place the partition name after the namespace. If you specify the name of the datacenter (optional), it must be the local datacenter. Communicating across partitions using this method is only supported within a + datacenter. For cross partition communication across datacenters, see [cluster + peering](/docs/connect/cluster-peering/k8s). + ```yaml + annotations: + "consul.hashicorp.com/connect-service-upstreams":"[service-name].[service-namespace].[service-partition]:[port]:[optional datacenter]" + ``` - [Prepared queries](/docs/connect/proxies#dynamic-upstreams-require-native-integration): Prepend the annotation with `prepared_query` and place the name of the query at the beginning of the string. ```yaml From 247b56def355384491d1455a25bc809457de4a78 Mon Sep 17 00:00:00 2001 From: "Chris S. Kim" Date: Mon, 20 Jun 2022 22:47:14 -0400 Subject: [PATCH 083/149] Pass trust domain to RBAC to validate and fix use of wrong peer trust bundles (#13508) --- agent/proxycfg/connect_proxy.go | 14 +++---- agent/proxycfg/manager_test.go | 4 +- agent/proxycfg/snapshot.go | 26 ++++++------ agent/proxycfg/state_test.go | 14 +++---- agent/xds/clusters.go | 8 ++-- agent/xds/listeners.go | 11 +++-- agent/xds/rbac.go | 42 +++++++++---------- agent/xds/rbac_test.go | 18 ++++---- ...deny-all-and-path-allow--httpfilter.golden | 4 +- ...fault-allow-deny-all-and-path-allow.golden | 6 +-- ...-deny-all-and-path-deny--httpfilter.golden | 6 +-- ...efault-allow-deny-all-and-path-deny.golden | 6 +-- ...ault-allow-kitchen-sink--httpfilter.golden | 12 +++--- .../rbac/default-allow-kitchen-sink.golden | 12 +++--- .../default-allow-one-deny--httpfilter.golden | 2 +- .../rbac/default-allow-one-deny.golden | 2 +- .../rbac/default-allow-path-allow.golden | 2 +- ...default-allow-path-deny--httpfilter.golden | 2 +- .../rbac/default-allow-path-deny.golden | 2 +- ...w-service-wildcard-deny--httpfilter.golden | 2 +- ...default-allow-service-wildcard-deny.golden | 2 +- ...with-kitchen-sink-perms--httpfilter.golden | 2 +- ...e-intention-with-kitchen-sink-perms.golden | 2 +- ...ath-deny-and-path-allow--httpfilter.golden | 2 +- ...-allow-two-path-deny-and-path-allow.golden | 2 +- ...default-deny-allow-deny--httpfilter.golden | 4 +- .../rbac/default-deny-allow-deny.golden | 4 +- ...deny-all-and-path-allow--httpfilter.golden | 2 +- ...fault-deny-kitchen-sink--httpfilter.golden | 12 +++--- .../rbac/default-deny-kitchen-sink.golden | 12 +++--- ...t-deny-mixed-precedence--httpfilter.golden | 2 +- .../rbac/default-deny-mixed-precedence.golden | 2 +- .../default-deny-one-allow--httpfilter.golden | 2 +- .../rbac/default-deny-one-allow.golden | 2 +- ...default-deny-path-allow--httpfilter.golden | 2 +- ...eny-peered-kitchen-sink--httpfilter.golden | 2 +- .../default-deny-peered-kitchen-sink.golden | 2 +- ...-service-wildcard-allow--httpfilter.golden | 2 +- ...default-deny-service-wildcard-allow.golden | 2 +- ...with-kitchen-sink-perms--httpfilter.golden | 2 +- ...ath-deny-and-path-allow--httpfilter.golden | 2 +- 41 files changed, 135 insertions(+), 126 deletions(-) diff --git a/agent/proxycfg/connect_proxy.go b/agent/proxycfg/connect_proxy.go index 0f62767c6..cdeea5ead 100644 --- a/agent/proxycfg/connect_proxy.go +++ b/agent/proxycfg/connect_proxy.go @@ -22,8 +22,8 @@ func (s *handlerConnectProxy) initialize(ctx context.Context) (ConfigSnapshot, e snap.ConnectProxy.WatchedDiscoveryChains = make(map[UpstreamID]context.CancelFunc) snap.ConnectProxy.WatchedUpstreams = make(map[UpstreamID]map[string]context.CancelFunc) snap.ConnectProxy.WatchedUpstreamEndpoints = make(map[UpstreamID]map[string]structs.CheckServiceNodes) - snap.ConnectProxy.WatchedPeerTrustBundles = make(map[string]context.CancelFunc) - snap.ConnectProxy.PeerTrustBundles = make(map[string]*pbpeering.PeeringTrustBundle) + snap.ConnectProxy.WatchedUpstreamPeerTrustBundles = make(map[string]context.CancelFunc) + snap.ConnectProxy.UpstreamPeerTrustBundles = make(map[string]*pbpeering.PeeringTrustBundle) snap.ConnectProxy.WatchedGateways = make(map[UpstreamID]map[string]context.CancelFunc) snap.ConnectProxy.WatchedGatewayEndpoints = make(map[UpstreamID]map[string]structs.CheckServiceNodes) snap.ConnectProxy.WatchedServiceChecks = make(map[structs.ServiceID][]structs.CheckType) @@ -212,7 +212,7 @@ func (s *handlerConnectProxy) initialize(ctx context.Context) (ConfigSnapshot, e } // Check whether a watch for this peer exists to avoid duplicates. - if _, ok := snap.ConnectProxy.WatchedPeerTrustBundles[uid.Peer]; !ok { + if _, ok := snap.ConnectProxy.WatchedUpstreamPeerTrustBundles[uid.Peer]; !ok { peerCtx, cancel := context.WithCancel(ctx) if err := s.dataSources.TrustBundle.Notify(peerCtx, &pbpeering.TrustBundleReadRequest{ Name: uid.Peer, @@ -222,7 +222,7 @@ func (s *handlerConnectProxy) initialize(ctx context.Context) (ConfigSnapshot, e return snap, fmt.Errorf("error while watching trust bundle for peer %q: %w", uid.Peer, err) } - snap.ConnectProxy.WatchedPeerTrustBundles[uid.Peer] = cancel + snap.ConnectProxy.WatchedUpstreamPeerTrustBundles[uid.Peer] = cancel } continue } @@ -270,7 +270,7 @@ func (s *handlerConnectProxy) handleUpdate(ctx context.Context, u UpdateEvent, s } peer := strings.TrimPrefix(u.CorrelationID, peerTrustBundleIDPrefix) if resp.Bundle != nil { - snap.ConnectProxy.PeerTrustBundles[peer] = resp.Bundle + snap.ConnectProxy.UpstreamPeerTrustBundles[peer] = resp.Bundle } case u.CorrelationID == peeringTrustBundlesWatchID: @@ -279,9 +279,9 @@ func (s *handlerConnectProxy) handleUpdate(ctx context.Context, u UpdateEvent, s return fmt.Errorf("invalid type for response: %T", u.Result) } if len(resp.Bundles) > 0 { - snap.ConnectProxy.PeeringTrustBundles = resp.Bundles + snap.ConnectProxy.InboundPeerTrustBundles = resp.Bundles } - snap.ConnectProxy.PeeringTrustBundlesSet = true + snap.ConnectProxy.InboundPeerTrustBundlesSet = true case u.CorrelationID == intentionsWatchID: resp, ok := u.Result.(*structs.IndexedIntentionMatches) diff --git a/agent/proxycfg/manager_test.go b/agent/proxycfg/manager_test.go index 08a54b11b..a62bbf2d3 100644 --- a/agent/proxycfg/manager_test.go +++ b/agent/proxycfg/manager_test.go @@ -238,7 +238,7 @@ func TestManager_BasicLifecycle(t *testing.T) { }, PassthroughUpstreams: map[UpstreamID]map[string]map[string]struct{}{}, PassthroughIndices: map[string]indexedTarget{}, - PeerTrustBundles: map[string]*pbpeering.PeeringTrustBundle{}, + UpstreamPeerTrustBundles: map[string]*pbpeering.PeeringTrustBundle{}, PeerUpstreamEndpoints: map[UpstreamID]structs.CheckServiceNodes{}, PeerUpstreamEndpointsUseHostnames: map[UpstreamID]struct{}{}, }, @@ -299,7 +299,7 @@ func TestManager_BasicLifecycle(t *testing.T) { }, PassthroughUpstreams: map[UpstreamID]map[string]map[string]struct{}{}, PassthroughIndices: map[string]indexedTarget{}, - PeerTrustBundles: map[string]*pbpeering.PeeringTrustBundle{}, + UpstreamPeerTrustBundles: map[string]*pbpeering.PeeringTrustBundle{}, PeerUpstreamEndpoints: map[UpstreamID]structs.CheckServiceNodes{}, PeerUpstreamEndpointsUseHostnames: map[UpstreamID]struct{}{}, }, diff --git a/agent/proxycfg/snapshot.go b/agent/proxycfg/snapshot.go index e405dd369..e56c169e8 100644 --- a/agent/proxycfg/snapshot.go +++ b/agent/proxycfg/snapshot.go @@ -44,13 +44,13 @@ type ConfigSnapshotUpstreams struct { // endpoints of an upstream. WatchedUpstreamEndpoints map[UpstreamID]map[string]structs.CheckServiceNodes - // WatchedPeerTrustBundles is a map of (PeerName -> CancelFunc) in order to cancel + // WatchedUpstreamPeerTrustBundles is a map of (PeerName -> CancelFunc) in order to cancel // watches for peer trust bundles any time the list of upstream peers changes. - WatchedPeerTrustBundles map[string]context.CancelFunc + WatchedUpstreamPeerTrustBundles map[string]context.CancelFunc - // PeerTrustBundles is a map of (PeerName -> PeeringTrustBundle). + // UpstreamPeerTrustBundles is a map of (PeerName -> PeeringTrustBundle). // It is used to store trust bundles for upstream TLS transport sockets. - PeerTrustBundles map[string]*pbpeering.PeeringTrustBundle + UpstreamPeerTrustBundles map[string]*pbpeering.PeeringTrustBundle // WatchedGateways is a map of UpstreamID -> (map of GatewayKey.String() -> // CancelFunc) in order to cancel watches for mesh gateways @@ -128,8 +128,8 @@ func gatewayKeyFromString(s string) GatewayKey { type configSnapshotConnectProxy struct { ConfigSnapshotUpstreams - PeeringTrustBundlesSet bool - PeeringTrustBundles []*pbpeering.PeeringTrustBundle + InboundPeerTrustBundlesSet bool + InboundPeerTrustBundles []*pbpeering.PeeringTrustBundle WatchedServiceChecks map[structs.ServiceID][]structs.CheckType // TODO: missing garbage collection PreparedQueryEndpoints map[UpstreamID]structs.CheckServiceNodes // DEPRECATED:see:WatchedUpstreamEndpoints @@ -152,8 +152,8 @@ func (c *configSnapshotConnectProxy) isEmpty() bool { len(c.WatchedDiscoveryChains) == 0 && len(c.WatchedUpstreams) == 0 && len(c.WatchedUpstreamEndpoints) == 0 && - len(c.WatchedPeerTrustBundles) == 0 && - len(c.PeerTrustBundles) == 0 && + len(c.WatchedUpstreamPeerTrustBundles) == 0 && + len(c.UpstreamPeerTrustBundles) == 0 && len(c.WatchedGateways) == 0 && len(c.WatchedGatewayEndpoints) == 0 && len(c.WatchedServiceChecks) == 0 && @@ -161,7 +161,7 @@ func (c *configSnapshotConnectProxy) isEmpty() bool { len(c.UpstreamConfig) == 0 && len(c.PassthroughUpstreams) == 0 && len(c.IntentionUpstreams) == 0 && - !c.PeeringTrustBundlesSet && + !c.InboundPeerTrustBundlesSet && !c.MeshConfigSet && len(c.PeerUpstreamEndpoints) == 0 && len(c.PeerUpstreamEndpointsUseHostnames) == 0 @@ -637,7 +637,7 @@ func (s *ConfigSnapshot) Clone() (*ConfigSnapshot, error) { snap.ConnectProxy.WatchedUpstreams = nil snap.ConnectProxy.WatchedGateways = nil snap.ConnectProxy.WatchedDiscoveryChains = nil - snap.ConnectProxy.WatchedPeerTrustBundles = nil + snap.ConnectProxy.WatchedUpstreamPeerTrustBundles = nil case structs.ServiceKindTerminatingGateway: snap.TerminatingGateway.WatchedServices = nil snap.TerminatingGateway.WatchedIntentions = nil @@ -652,7 +652,7 @@ func (s *ConfigSnapshot) Clone() (*ConfigSnapshot, error) { snap.IngressGateway.WatchedUpstreams = nil snap.IngressGateway.WatchedGateways = nil snap.IngressGateway.WatchedDiscoveryChains = nil - snap.IngressGateway.WatchedPeerTrustBundles = nil + snap.IngressGateway.WatchedUpstreamPeerTrustBundles = nil // only ingress-gateway snap.IngressGateway.LeafCertWatchCancel = nil } @@ -676,7 +676,7 @@ func (s *ConfigSnapshot) Leaf() *structs.IssuedCert { func (s *ConfigSnapshot) PeeringTrustBundles() []*pbpeering.PeeringTrustBundle { switch s.Kind { case structs.ServiceKindConnectProxy: - return s.ConnectProxy.PeeringTrustBundles + return s.ConnectProxy.InboundPeerTrustBundles case structs.ServiceKindMeshGateway: return s.MeshGateway.PeeringTrustBundles default: @@ -755,7 +755,7 @@ func (u *ConfigSnapshotUpstreams) PeeredUpstreamIDs() []UpstreamID { continue } - if _, ok := u.PeerTrustBundles[uid.Peer]; uid.Peer != "" && !ok { + if _, ok := u.UpstreamPeerTrustBundles[uid.Peer]; uid.Peer != "" && !ok { // The trust bundle for this upstream is not available yet, skip for now. continue } diff --git a/agent/proxycfg/state_test.go b/agent/proxycfg/state_test.go index 667946283..b61bc5eab 100644 --- a/agent/proxycfg/state_test.go +++ b/agent/proxycfg/state_test.go @@ -2572,15 +2572,15 @@ func TestState_WatchesAndUpdates(t *testing.T) { require.Len(t, snap.ConnectProxy.WatchedGateways, 0, "%+v", snap.ConnectProxy.WatchedGateways) require.Len(t, snap.ConnectProxy.WatchedGatewayEndpoints, 0, "%+v", snap.ConnectProxy.WatchedGatewayEndpoints) - require.Contains(t, snap.ConnectProxy.WatchedPeerTrustBundles, "peer-a", "%+v", snap.ConnectProxy.WatchedPeerTrustBundles) - require.Len(t, snap.ConnectProxy.PeerTrustBundles, 0, "%+v", snap.ConnectProxy.PeerTrustBundles) + require.Contains(t, snap.ConnectProxy.WatchedUpstreamPeerTrustBundles, "peer-a", "%+v", snap.ConnectProxy.WatchedUpstreamPeerTrustBundles) + require.Len(t, snap.ConnectProxy.UpstreamPeerTrustBundles, 0, "%+v", snap.ConnectProxy.UpstreamPeerTrustBundles) require.Len(t, snap.ConnectProxy.PeerUpstreamEndpoints, 0, "%+v", snap.ConnectProxy.PeerUpstreamEndpoints) require.Len(t, snap.ConnectProxy.WatchedServiceChecks, 0, "%+v", snap.ConnectProxy.WatchedServiceChecks) require.Len(t, snap.ConnectProxy.PreparedQueryEndpoints, 0, "%+v", snap.ConnectProxy.PreparedQueryEndpoints) - require.Len(t, snap.ConnectProxy.PeeringTrustBundles, 0, "%+v", snap.ConnectProxy.PeeringTrustBundles) - require.False(t, snap.ConnectProxy.PeeringTrustBundlesSet) + require.Len(t, snap.ConnectProxy.InboundPeerTrustBundles, 0, "%+v", snap.ConnectProxy.InboundPeerTrustBundles) + require.False(t, snap.ConnectProxy.InboundPeerTrustBundlesSet) }, }, { @@ -2655,7 +2655,7 @@ func TestState_WatchesAndUpdates(t *testing.T) { require.Equal(t, indexedRoots, snap.Roots) require.Equal(t, issuedCert, snap.ConnectProxy.Leaf) - prototest.AssertDeepEqual(t, peerTrustBundles.Bundles, snap.ConnectProxy.PeeringTrustBundles) + prototest.AssertDeepEqual(t, peerTrustBundles.Bundles, snap.ConnectProxy.InboundPeerTrustBundles) require.Len(t, snap.ConnectProxy.DiscoveryChain, 1, "%+v", snap.ConnectProxy.DiscoveryChain) require.Len(t, snap.ConnectProxy.WatchedUpstreams, 1, "%+v", snap.ConnectProxy.WatchedUpstreams) @@ -2663,8 +2663,8 @@ func TestState_WatchesAndUpdates(t *testing.T) { require.Len(t, snap.ConnectProxy.WatchedGateways, 1, "%+v", snap.ConnectProxy.WatchedGateways) require.Len(t, snap.ConnectProxy.WatchedGatewayEndpoints, 1, "%+v", snap.ConnectProxy.WatchedGatewayEndpoints) - require.Contains(t, snap.ConnectProxy.WatchedPeerTrustBundles, "peer-a", "%+v", snap.ConnectProxy.WatchedPeerTrustBundles) - require.Equal(t, peerTrustBundles.Bundles[0], snap.ConnectProxy.PeerTrustBundles["peer-a"], "%+v", snap.ConnectProxy.WatchedPeerTrustBundles) + require.Contains(t, snap.ConnectProxy.WatchedUpstreamPeerTrustBundles, "peer-a", "%+v", snap.ConnectProxy.WatchedUpstreamPeerTrustBundles) + require.Equal(t, peerTrustBundles.Bundles[0], snap.ConnectProxy.UpstreamPeerTrustBundles["peer-a"], "%+v", snap.ConnectProxy.WatchedUpstreamPeerTrustBundles) require.Len(t, snap.ConnectProxy.PeerUpstreamEndpoints, 1, "%+v", snap.ConnectProxy.PeerUpstreamEndpoints) require.NotNil(t, snap.ConnectProxy.PeerUpstreamEndpoints[extApiUID]) diff --git a/agent/xds/clusters.go b/agent/xds/clusters.go index 2acfa7c10..feb488a63 100644 --- a/agent/xds/clusters.go +++ b/agent/xds/clusters.go @@ -465,9 +465,9 @@ func (s *ResourceGenerator) makeDestinationClusters(cfgSnap *proxycfg.ConfigSnap cluster := s.makeDynamicForwardProxyCluster(cfgSnap, opts) // TODO (dans): might be relevant later for TLS addons like CA validation - //if err := s.injectGatewayServiceAddons(cfgSnap, cluster, svc, loadBalancer); err != nil { + // if err := s.injectGatewayServiceAddons(cfgSnap, cluster, svc, loadBalancer); err != nil { // return nil, err - //} + // } clusters = append(clusters, cluster) } return clusters, nil @@ -695,7 +695,7 @@ func (s *ResourceGenerator) makeUpstreamClusterForPeerService( rootPEMs := cfgSnap.RootPEMs() if uid.Peer != "" { - rootPEMs = cfgSnap.ConnectProxy.PeerTrustBundles[uid.Peer].ConcatenatedRootPEMs() + rootPEMs = cfgSnap.ConnectProxy.UpstreamPeerTrustBundles[uid.Peer].ConcatenatedRootPEMs() } // Enable TLS upstream with the configured client certificate. @@ -999,7 +999,7 @@ func (s *ResourceGenerator) makeUpstreamClustersForDiscoveryChain( rootPEMs := cfgSnap.RootPEMs() if uid.Peer != "" { - rootPEMs = cfgSnap.ConnectProxy.PeerTrustBundles[uid.Peer].ConcatenatedRootPEMs() + rootPEMs = cfgSnap.ConnectProxy.UpstreamPeerTrustBundles[uid.Peer].ConcatenatedRootPEMs() } commonTLSContext := makeCommonTLSContext( cfgSnap.Leaf(), diff --git a/agent/xds/listeners.go b/agent/xds/listeners.go index aab41f44b..a364ecce1 100644 --- a/agent/xds/listeners.go +++ b/agent/xds/listeners.go @@ -697,7 +697,8 @@ func (s *ResourceGenerator) injectConnectFilters(cfgSnap *proxycfg.ConfigSnapsho authzFilter, err := makeRBACNetworkFilter( cfgSnap.ConnectProxy.Intentions, cfgSnap.IntentionDefaultAllow, - cfgSnap.ConnectProxy.PeerTrustBundles, + cfgSnap.Roots.TrustDomain, + cfgSnap.ConnectProxy.InboundPeerTrustBundles, ) if err != nil { return err @@ -952,7 +953,8 @@ func (s *ResourceGenerator) makeInboundListener(cfgSnap *proxycfg.ConfigSnapshot httpAuthzFilter, err := makeRBACHTTPFilter( cfgSnap.ConnectProxy.Intentions, cfgSnap.IntentionDefaultAllow, - cfgSnap.ConnectProxy.PeerTrustBundles, + cfgSnap.Roots.TrustDomain, + cfgSnap.ConnectProxy.InboundPeerTrustBundles, ) if err != nil { return nil, err @@ -1009,7 +1011,8 @@ func (s *ResourceGenerator) makeInboundListener(cfgSnap *proxycfg.ConfigSnapshot filterOpts.httpAuthzFilter, err = makeRBACHTTPFilter( cfgSnap.ConnectProxy.Intentions, cfgSnap.IntentionDefaultAllow, - cfgSnap.ConnectProxy.PeerTrustBundles, + cfgSnap.Roots.TrustDomain, + cfgSnap.ConnectProxy.InboundPeerTrustBundles, ) if err != nil { return nil, err @@ -1307,6 +1310,7 @@ func (s *ResourceGenerator) makeFilterChainTerminatingGateway(cfgSnap *proxycfg. authFilter, err := makeRBACNetworkFilter( intentions, cfgSnap.IntentionDefaultAllow, + cfgSnap.Roots.TrustDomain, nil, // TODO(peering): verify intentions w peers don't apply to terminatingGateway ) if err != nil { @@ -1344,6 +1348,7 @@ func (s *ResourceGenerator) makeFilterChainTerminatingGateway(cfgSnap *proxycfg. opts.httpAuthzFilter, err = makeRBACHTTPFilter( intentions, cfgSnap.IntentionDefaultAllow, + cfgSnap.Roots.TrustDomain, nil, // TODO(peering): verify intentions w peers don't apply to terminatingGateway ) if err != nil { diff --git a/agent/xds/rbac.go b/agent/xds/rbac.go index 319c648e3..c5a651150 100644 --- a/agent/xds/rbac.go +++ b/agent/xds/rbac.go @@ -21,9 +21,10 @@ import ( func makeRBACNetworkFilter( intentions structs.Intentions, intentionDefaultAllow bool, - peerTrustBundles map[string]*pbpeering.PeeringTrustBundle, + trustDomain string, + peerTrustBundles []*pbpeering.PeeringTrustBundle, ) (*envoy_listener_v3.Filter, error) { - rules, err := makeRBACRules(intentions, intentionDefaultAllow, false, peerTrustBundles) + rules, err := makeRBACRules(intentions, intentionDefaultAllow, trustDomain, false, peerTrustBundles) if err != nil { return nil, err } @@ -38,9 +39,10 @@ func makeRBACNetworkFilter( func makeRBACHTTPFilter( intentions structs.Intentions, intentionDefaultAllow bool, - peerTrustBundles map[string]*pbpeering.PeeringTrustBundle, + trustDomain string, + peerTrustBundles []*pbpeering.PeeringTrustBundle, ) (*envoy_http_v3.HttpFilter, error) { - rules, err := makeRBACRules(intentions, intentionDefaultAllow, true, peerTrustBundles) + rules, err := makeRBACRules(intentions, intentionDefaultAllow, trustDomain, true, peerTrustBundles) if err != nil { return nil, err } @@ -53,6 +55,7 @@ func makeRBACHTTPFilter( func intentionListToIntermediateRBACForm( intentions structs.Intentions, + trustDomain string, isHTTP bool, trustBundlesByPeer map[string]*pbpeering.PeeringTrustBundle, ) []*rbacIntention { @@ -72,7 +75,7 @@ func intentionListToIntermediateRBACForm( continue } - rixn := intentionToIntermediateRBACForm(ixn, isHTTP, trustBundle) + rixn := intentionToIntermediateRBACForm(ixn, trustDomain, isHTTP, trustBundle) rbacIxns = append(rbacIxns, rixn) } return rbacIxns @@ -210,11 +213,12 @@ func removePermissionPrecedence(perms []*rbacPermission, intentionDefaultAction return out } -func intentionToIntermediateRBACForm(ixn *structs.Intention, isHTTP bool, bundle *pbpeering.PeeringTrustBundle) *rbacIntention { +func intentionToIntermediateRBACForm(ixn *structs.Intention, trustDomain string, isHTTP bool, bundle *pbpeering.PeeringTrustBundle) *rbacIntention { rixn := &rbacIntention{ Source: rbacService{ ServiceName: ixn.SourceServiceName(), Peer: ixn.SourcePeer, + TrustDomain: trustDomain, }, Precedence: ixn.Precedence, } @@ -426,25 +430,21 @@ func simplifyNotSourceSlice(notSources []rbacService) []rbacService { func makeRBACRules( intentions structs.Intentions, intentionDefaultAllow bool, + trustDomain string, isHTTP bool, - peerTrustBundles map[string]*pbpeering.PeeringTrustBundle, + peerTrustBundles []*pbpeering.PeeringTrustBundle, ) (*envoy_rbac_v3.RBAC, error) { - // Note that we DON'T explicitly validate the trust-domain matches ours. - // - // For now we don't validate the trust domain of the _destination_ at all. - // The RBAC policies below ignore the trust domain and it's implicit that - // the request is for the correct cluster. We might want to reconsider this - // later but plumbing in additional machinery to check the clusterID here - // is not really necessary for now unless the Envoys are badly configured. - // Our threat model _requires_ correctly configured and well behaved - // proxies given that they have ACLs to fetch certs and so can do whatever - // they want including not authorizing traffic at all or routing it do a - // different service than they auth'd against. - // TODO(banks,rb): Implement revocation list checking? + // TODO(peering): mkeeler asked that these maps come from proxycfg instead of + // being constructed in xds to save memory allocation and gc pressure. Low priority. + trustBundlesByPeer := make(map[string]*pbpeering.PeeringTrustBundle, len(peerTrustBundles)) + for _, ptb := range peerTrustBundles { + trustBundlesByPeer[ptb.PeerName] = ptb + } + // First build up just the basic principal matches. - rbacIxns := intentionListToIntermediateRBACForm(intentions, isHTTP, peerTrustBundles) + rbacIxns := intentionListToIntermediateRBACForm(intentions, trustDomain, isHTTP, trustBundlesByPeer) // Normalize: if we are in default-deny then all intentions must be allows and vice versa intentionDefaultAction := intentionActionFromBool(intentionDefaultAllow) @@ -641,7 +641,7 @@ const anyPath = `[^/]+` func makeSpiffePattern(src rbacService) string { var ( - host = anyPath // TODO(peering): We match trust domain on any value but should be defaulting to the local trust domain + host = src.TrustDomain ap = src.PartitionOrDefault() ns = src.NamespaceOrDefault() svc = src.Name diff --git a/agent/xds/rbac_test.go b/agent/xds/rbac_test.go index 358a91347..81fb32ad8 100644 --- a/agent/xds/rbac_test.go +++ b/agent/xds/rbac_test.go @@ -58,10 +58,13 @@ func TestRemoveIntentionPrecedence(t *testing.T) { ExportedPartition: "part1", }, } + testTrustDomain := "test.consul" var ( - nameWild = rbacService{ServiceName: structs.NewServiceName("*", nil)} - nameWeb = rbacService{ServiceName: structs.NewServiceName("web", nil)} + nameWild = rbacService{ServiceName: structs.NewServiceName("*", nil), + TrustDomain: testTrustDomain} + nameWeb = rbacService{ServiceName: structs.NewServiceName("web", nil), + TrustDomain: testTrustDomain} nameWildPeered = rbacService{ServiceName: structs.NewServiceName("*", nil), Peer: "peer1", TrustDomain: "peer1.domain", ExportedPartition: "part1"} nameWebPeered = rbacService{ServiceName: structs.NewServiceName("web", nil), @@ -439,7 +442,7 @@ func TestRemoveIntentionPrecedence(t *testing.T) { for name, tt := range tests { t.Run(name, func(t *testing.T) { - rbacIxns := intentionListToIntermediateRBACForm(tt.intentions, tt.http, testPeerTrustBundle) + rbacIxns := intentionListToIntermediateRBACForm(tt.intentions, testTrustDomain, tt.http, testPeerTrustBundle) intentionDefaultAction := intentionActionFromBool(tt.intentionDefaultAllow) rbacIxns = removeIntentionPrecedence(rbacIxns, intentionDefaultAction) @@ -472,13 +475,14 @@ func TestMakeRBACNetworkAndHTTPFilters(t *testing.T) { ixn.Permissions = perms return ixn } - testPeerTrustBundle := map[string]*pbpeering.PeeringTrustBundle{ - "peer1": { + testPeerTrustBundle := []*pbpeering.PeeringTrustBundle{ + { PeerName: "peer1", TrustDomain: "peer1.domain", ExportedPartition: "part1", }, } + testTrustDomain := "test.consul" sorted := func(ixns ...*structs.Intention) structs.Intentions { sort.SliceStable(ixns, func(i, j int) bool { return ixns[j].Precedence < ixns[i].Precedence @@ -797,7 +801,7 @@ func TestMakeRBACNetworkAndHTTPFilters(t *testing.T) { tt := tt t.Run(name, func(t *testing.T) { t.Run("network filter", func(t *testing.T) { - filter, err := makeRBACNetworkFilter(tt.intentions, tt.intentionDefaultAllow, testPeerTrustBundle) + filter, err := makeRBACNetworkFilter(tt.intentions, tt.intentionDefaultAllow, testTrustDomain, testPeerTrustBundle) require.NoError(t, err) t.Run("current", func(t *testing.T) { @@ -807,7 +811,7 @@ func TestMakeRBACNetworkAndHTTPFilters(t *testing.T) { }) }) t.Run("http filter", func(t *testing.T) { - filter, err := makeRBACHTTPFilter(tt.intentions, tt.intentionDefaultAllow, testPeerTrustBundle) + filter, err := makeRBACHTTPFilter(tt.intentions, tt.intentionDefaultAllow, testTrustDomain, testPeerTrustBundle) require.NoError(t, err) t.Run("current", func(t *testing.T) { diff --git a/agent/xds/testdata/rbac/default-allow-deny-all-and-path-allow--httpfilter.golden b/agent/xds/testdata/rbac/default-allow-deny-all-and-path-allow--httpfilter.golden index b44899f2a..b590b7135 100644 --- a/agent/xds/testdata/rbac/default-allow-deny-all-and-path-allow--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-allow-deny-all-and-path-allow--httpfilter.golden @@ -22,7 +22,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -35,7 +35,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-deny-all-and-path-allow.golden b/agent/xds/testdata/rbac/default-allow-deny-all-and-path-allow.golden index 092676776..51346b03f 100644 --- a/agent/xds/testdata/rbac/default-allow-deny-all-and-path-allow.golden +++ b/agent/xds/testdata/rbac/default-allow-deny-all-and-path-allow.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -34,7 +34,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -47,7 +47,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-deny-all-and-path-deny--httpfilter.golden b/agent/xds/testdata/rbac/default-allow-deny-all-and-path-deny--httpfilter.golden index 806b19c59..62f4ba97e 100644 --- a/agent/xds/testdata/rbac/default-allow-deny-all-and-path-deny--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-allow-deny-all-and-path-deny--httpfilter.golden @@ -22,7 +22,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -35,7 +35,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -64,7 +64,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-deny-all-and-path-deny.golden b/agent/xds/testdata/rbac/default-allow-deny-all-and-path-deny.golden index 092676776..51346b03f 100644 --- a/agent/xds/testdata/rbac/default-allow-deny-all-and-path-deny.golden +++ b/agent/xds/testdata/rbac/default-allow-deny-all-and-path-deny.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -34,7 +34,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -47,7 +47,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-kitchen-sink--httpfilter.golden b/agent/xds/testdata/rbac/default-allow-kitchen-sink--httpfilter.golden index 714203229..65da4d27f 100644 --- a/agent/xds/testdata/rbac/default-allow-kitchen-sink--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-allow-kitchen-sink--httpfilter.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/cron$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/cron$" } } } @@ -31,7 +31,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -46,7 +46,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -59,7 +59,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -73,7 +73,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/unsafe$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/unsafe$" } } } @@ -87,7 +87,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/cron$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/cron$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-kitchen-sink.golden b/agent/xds/testdata/rbac/default-allow-kitchen-sink.golden index a85359a55..21e94ee14 100644 --- a/agent/xds/testdata/rbac/default-allow-kitchen-sink.golden +++ b/agent/xds/testdata/rbac/default-allow-kitchen-sink.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/cron$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/cron$" } } } @@ -31,7 +31,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -46,7 +46,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -59,7 +59,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -73,7 +73,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/unsafe$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/unsafe$" } } } @@ -87,7 +87,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/cron$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/cron$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-one-deny--httpfilter.golden b/agent/xds/testdata/rbac/default-allow-one-deny--httpfilter.golden index 72be1fe60..f315c45e0 100644 --- a/agent/xds/testdata/rbac/default-allow-one-deny--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-allow-one-deny--httpfilter.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-one-deny.golden b/agent/xds/testdata/rbac/default-allow-one-deny.golden index 05dca90f8..45b6965e0 100644 --- a/agent/xds/testdata/rbac/default-allow-one-deny.golden +++ b/agent/xds/testdata/rbac/default-allow-one-deny.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-path-allow.golden b/agent/xds/testdata/rbac/default-allow-path-allow.golden index 05dca90f8..45b6965e0 100644 --- a/agent/xds/testdata/rbac/default-allow-path-allow.golden +++ b/agent/xds/testdata/rbac/default-allow-path-allow.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-path-deny--httpfilter.golden b/agent/xds/testdata/rbac/default-allow-path-deny--httpfilter.golden index 4f64c2734..aa06ebe3d 100644 --- a/agent/xds/testdata/rbac/default-allow-path-deny--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-allow-path-deny--httpfilter.golden @@ -23,7 +23,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-path-deny.golden b/agent/xds/testdata/rbac/default-allow-path-deny.golden index 05dca90f8..45b6965e0 100644 --- a/agent/xds/testdata/rbac/default-allow-path-deny.golden +++ b/agent/xds/testdata/rbac/default-allow-path-deny.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-service-wildcard-deny--httpfilter.golden b/agent/xds/testdata/rbac/default-allow-service-wildcard-deny--httpfilter.golden index be70c6725..0c69fa845 100644 --- a/agent/xds/testdata/rbac/default-allow-service-wildcard-deny--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-allow-service-wildcard-deny--httpfilter.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-service-wildcard-deny.golden b/agent/xds/testdata/rbac/default-allow-service-wildcard-deny.golden index 1951818cd..d685342e9 100644 --- a/agent/xds/testdata/rbac/default-allow-service-wildcard-deny.golden +++ b/agent/xds/testdata/rbac/default-allow-service-wildcard-deny.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-single-intention-with-kitchen-sink-perms--httpfilter.golden b/agent/xds/testdata/rbac/default-allow-single-intention-with-kitchen-sink-perms--httpfilter.golden index a3945edbb..ba1787d12 100644 --- a/agent/xds/testdata/rbac/default-allow-single-intention-with-kitchen-sink-perms--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-allow-single-intention-with-kitchen-sink-perms--httpfilter.golden @@ -227,7 +227,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-single-intention-with-kitchen-sink-perms.golden b/agent/xds/testdata/rbac/default-allow-single-intention-with-kitchen-sink-perms.golden index 05dca90f8..45b6965e0 100644 --- a/agent/xds/testdata/rbac/default-allow-single-intention-with-kitchen-sink-perms.golden +++ b/agent/xds/testdata/rbac/default-allow-single-intention-with-kitchen-sink-perms.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-two-path-deny-and-path-allow--httpfilter.golden b/agent/xds/testdata/rbac/default-allow-two-path-deny-and-path-allow--httpfilter.golden index 8eb2021cc..98f414209 100644 --- a/agent/xds/testdata/rbac/default-allow-two-path-deny-and-path-allow--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-allow-two-path-deny-and-path-allow--httpfilter.golden @@ -45,7 +45,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-allow-two-path-deny-and-path-allow.golden b/agent/xds/testdata/rbac/default-allow-two-path-deny-and-path-allow.golden index 05dca90f8..45b6965e0 100644 --- a/agent/xds/testdata/rbac/default-allow-two-path-deny-and-path-allow.golden +++ b/agent/xds/testdata/rbac/default-allow-two-path-deny-and-path-allow.golden @@ -19,7 +19,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-allow-deny--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-allow-deny--httpfilter.golden index ee099df84..3e48b7c56 100644 --- a/agent/xds/testdata/rbac/default-deny-allow-deny--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-allow-deny--httpfilter.golden @@ -21,7 +21,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -34,7 +34,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-allow-deny.golden b/agent/xds/testdata/rbac/default-deny-allow-deny.golden index b92c68935..d901cf42c 100644 --- a/agent/xds/testdata/rbac/default-deny-allow-deny.golden +++ b/agent/xds/testdata/rbac/default-deny-allow-deny.golden @@ -21,7 +21,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -34,7 +34,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-deny-all-and-path-allow--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-deny-all-and-path-allow--httpfilter.golden index 323526f48..003fdb24b 100644 --- a/agent/xds/testdata/rbac/default-deny-deny-all-and-path-allow--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-deny-all-and-path-allow--httpfilter.golden @@ -22,7 +22,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-kitchen-sink--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-kitchen-sink--httpfilter.golden index f61f892d4..426404d74 100644 --- a/agent/xds/testdata/rbac/default-deny-kitchen-sink--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-kitchen-sink--httpfilter.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/cron$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/cron$" } } } @@ -30,7 +30,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -45,7 +45,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -58,7 +58,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -72,7 +72,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/unsafe$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/unsafe$" } } } @@ -86,7 +86,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/cron$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/cron$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-kitchen-sink.golden b/agent/xds/testdata/rbac/default-deny-kitchen-sink.golden index 30d56485c..912b64d95 100644 --- a/agent/xds/testdata/rbac/default-deny-kitchen-sink.golden +++ b/agent/xds/testdata/rbac/default-deny-kitchen-sink.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/cron$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/cron$" } } } @@ -30,7 +30,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -45,7 +45,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } @@ -58,7 +58,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } @@ -72,7 +72,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/unsafe$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/unsafe$" } } } @@ -86,7 +86,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/cron$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/cron$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-mixed-precedence--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-mixed-precedence--httpfilter.golden index a37549e5e..c4795f46e 100644 --- a/agent/xds/testdata/rbac/default-deny-mixed-precedence--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-mixed-precedence--httpfilter.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-mixed-precedence.golden b/agent/xds/testdata/rbac/default-deny-mixed-precedence.golden index 4bc3c7378..3fc78b295 100644 --- a/agent/xds/testdata/rbac/default-deny-mixed-precedence.golden +++ b/agent/xds/testdata/rbac/default-deny-mixed-precedence.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-one-allow--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-one-allow--httpfilter.golden index a37549e5e..c4795f46e 100644 --- a/agent/xds/testdata/rbac/default-deny-one-allow--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-one-allow--httpfilter.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-one-allow.golden b/agent/xds/testdata/rbac/default-deny-one-allow.golden index 4bc3c7378..3fc78b295 100644 --- a/agent/xds/testdata/rbac/default-deny-one-allow.golden +++ b/agent/xds/testdata/rbac/default-deny-one-allow.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-path-allow--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-path-allow--httpfilter.golden index 323526f48..003fdb24b 100644 --- a/agent/xds/testdata/rbac/default-deny-path-allow--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-path-allow--httpfilter.golden @@ -22,7 +22,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-peered-kitchen-sink--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-peered-kitchen-sink--httpfilter.golden index 304e428ef..80d977dff 100644 --- a/agent/xds/testdata/rbac/default-deny-peered-kitchen-sink--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-peered-kitchen-sink--httpfilter.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-peered-kitchen-sink.golden b/agent/xds/testdata/rbac/default-deny-peered-kitchen-sink.golden index 2d9985759..eb4801594 100644 --- a/agent/xds/testdata/rbac/default-deny-peered-kitchen-sink.golden +++ b/agent/xds/testdata/rbac/default-deny-peered-kitchen-sink.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-service-wildcard-allow--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-service-wildcard-allow--httpfilter.golden index 365f970c5..a65162f24 100644 --- a/agent/xds/testdata/rbac/default-deny-service-wildcard-allow--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-service-wildcard-allow--httpfilter.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-service-wildcard-allow.golden b/agent/xds/testdata/rbac/default-deny-service-wildcard-allow.golden index fd8445265..3780a9079 100644 --- a/agent/xds/testdata/rbac/default-deny-service-wildcard-allow.golden +++ b/agent/xds/testdata/rbac/default-deny-service-wildcard-allow.golden @@ -18,7 +18,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/[^/]+$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/[^/]+$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-single-intention-with-kitchen-sink-perms--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-single-intention-with-kitchen-sink-perms--httpfilter.golden index 694f5858f..f23112071 100644 --- a/agent/xds/testdata/rbac/default-deny-single-intention-with-kitchen-sink-perms--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-single-intention-with-kitchen-sink-perms--httpfilter.golden @@ -226,7 +226,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } diff --git a/agent/xds/testdata/rbac/default-deny-two-path-deny-and-path-allow--httpfilter.golden b/agent/xds/testdata/rbac/default-deny-two-path-deny-and-path-allow--httpfilter.golden index aeadff72b..75534cb10 100644 --- a/agent/xds/testdata/rbac/default-deny-two-path-deny-and-path-allow--httpfilter.golden +++ b/agent/xds/testdata/rbac/default-deny-two-path-deny-and-path-allow--httpfilter.golden @@ -46,7 +46,7 @@ "googleRe2": { }, - "regex": "^spiffe://[^/]+/ns/default/dc/[^/]+/svc/web$" + "regex": "^spiffe://test.consul/ns/default/dc/[^/]+/svc/web$" } } } From 57cdbc63550c1501364173e18426a76ed399f72a Mon Sep 17 00:00:00 2001 From: Max Bowsher Date: Tue, 21 Jun 2022 04:37:08 +0100 Subject: [PATCH 084/149] Fix use of trailing dots on metric names in telemetry.mdx In the metric documentation, some metrics are written with an extra trailing dot character. Often this is just spurious. In some cases, it is an allusion to the metric name having various variable dot-separated parts (which really ought to be labels, but that's another issue). 1) Trim all the dots which are without meaning. 2) Explicitly show with placeholders in angle brackets, where there are variable parts. 3) Remove a `json` type indicator from a code block which was not JSON (rather Prometheus metrics) format, since my IDE was reporting the non-JSON syntax as an error. 4) Reformat each Markdown table which I had touched with other changes to have consistent column spacing in the source representation. As a result, this PR has a lot of whitespace changes, so please make use of GitHub's "ignore whitespace" checkbox in the PR diff viewer. Let me know if you would prefer: - to not accept the table spacing reformat in the same PR as other changes - to not accept the table spacing reformat ever - to have all tables in the file reformatted, not just ones touched with other changes --- website/content/docs/agent/telemetry.mdx | 451 ++++++++++++----------- 1 file changed, 226 insertions(+), 225 deletions(-) diff --git a/website/content/docs/agent/telemetry.mdx b/website/content/docs/agent/telemetry.mdx index 0435a42c8..8e8618f33 100644 --- a/website/content/docs/agent/telemetry.mdx +++ b/website/content/docs/agent/telemetry.mdx @@ -349,199 +349,199 @@ populated free list structure. This is a full list of metrics emitted by Consul. -| Metric | Description | Unit | Type | -| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------- | ------- | -| `consul.acl.blocked.{check,service}.deregistration` | Increments whenever a deregistration fails for an entity (check or service) is blocked by an ACL. | requests | counter | -| `consul.acl.blocked.{check,node,service}.registration` | Increments whenever a registration fails for an entity (check, node or service) is blocked by an ACL. | requests | counter | -| `consul.api.http` | Migrated from consul.http.. this samples how long it takes to service the given HTTP request for the given verb and path. Includes labels for `path` and `method`. `path` does not include details like service or key names, for these an underscore will be present as a placeholder (eg. path=`v1.kv._`) | ms | timer | -| `consul.client.rpc` | Increments whenever a Consul agent in client mode makes an RPC request to a Consul server. This gives a measure of how much a given agent is loading the Consul servers. Currently, this is only generated by agents in client mode, not Consul servers. | requests | counter | -| `consul.client.rpc.exceeded` | Increments whenever a Consul agent in client mode makes an RPC request to a Consul server gets rate limited by that agent's [`limits`](/docs/agent/config/config-files#limits) configuration. This gives an indication that there's an abusive application making too many requests on the agent, or that the rate limit needs to be increased. Currently, this only applies to agents in client mode, not Consul servers. | rejected requests | counter | -| `consul.client.rpc.failed` | Increments whenever a Consul agent in client mode makes an RPC request to a Consul server and fails. | requests | counter | -| `consul.client.api.catalog_register.` | Increments whenever a Consul agent receives a catalog register request. | requests | counter | -| `consul.client.api.success.catalog_register.` | Increments whenever a Consul agent successfully responds to a catalog register request. | requests | counter | -| `consul.client.rpc.error.catalog_register.` | Increments whenever a Consul agent receives an RPC error for a catalog register request. | errors | counter | -| `consul.client.api.catalog_deregister.` | Increments whenever a Consul agent receives a catalog deregister request. | requests | counter | -| `consul.client.api.success.catalog_deregister.` | Increments whenever a Consul agent successfully responds to a catalog deregister request. | requests | counter | -| `consul.client.rpc.error.catalog_deregister.` | Increments whenever a Consul agent receives an RPC error for a catalog deregister request. | errors | counter | -| `consul.client.api.catalog_datacenters.` | Increments whenever a Consul agent receives a request to list datacenters in the catalog. | requests | counter | -| `consul.client.api.success.catalog_datacenters.` | Increments whenever a Consul agent successfully responds to a request to list datacenters. | requests | counter | -| `consul.client.rpc.error.catalog_datacenters.` | Increments whenever a Consul agent receives an RPC error for a request to list datacenters. | errors | counter | -| `consul.client.api.catalog_nodes.` | Increments whenever a Consul agent receives a request to list nodes from the catalog. | requests | counter | -| `consul.client.api.success.catalog_nodes.` | Increments whenever a Consul agent successfully responds to a request to list nodes. | requests | counter | -| `consul.client.rpc.error.catalog_nodes.` | Increments whenever a Consul agent receives an RPC error for a request to list nodes. | errors | counter | -| `consul.client.api.catalog_services.` | Increments whenever a Consul agent receives a request to list services from the catalog. | requests | counter | -| `consul.client.api.success.catalog_services.` | Increments whenever a Consul agent successfully responds to a request to list services. | requests | counter | -| `consul.client.rpc.error.catalog_services.` | Increments whenever a Consul agent receives an RPC error for a request to list services. | errors | counter | -| `consul.client.api.catalog_service_nodes.` | Increments whenever a Consul agent receives a request to list nodes offering a service. | requests | counter | -| `consul.client.api.success.catalog_service_nodes.` | Increments whenever a Consul agent successfully responds to a request to list nodes offering a service. | requests | counter | -| `consul.client.api.error.catalog_service_nodes.` | Increments whenever a Consul agent receives an RPC error for request to list nodes offering a service. | requests | counter | -| `consul.client.rpc.error.catalog_service_nodes.` | Increments whenever a Consul agent receives an RPC error for a request to list nodes offering a service.   | errors | counter | -| `consul.client.api.catalog_node_services.` | Increments whenever a Consul agent receives a request to list services registered in a node.   | requests | counter | -| `consul.client.api.success.catalog_node_services.` | Increments whenever a Consul agent successfully responds to a request to list services in a node.   | requests | counter | -| `consul.client.rpc.error.catalog_node_services.` | Increments whenever a Consul agent receives an RPC error for a request to list services in a node.   | errors | counter | -| `consul.client.api.catalog_node_service_list` | Increments whenever a Consul agent receives a request to list a node's registered services. | requests | counter | -| `consul.client.rpc.error.catalog_node_service_list` | Increments whenever a Consul agent receives an RPC error for request to list a node's registered services. | errors | counter | -| `consul.client.api.success.catalog_node_service_list` | Increments whenever a Consul agent successfully responds to a request to list a node's registered services. | requests | counter | -| `consul.client.api.catalog_gateway_services.` | Increments whenever a Consul agent receives a request to list services associated with a gateway. | requests | counter | -| `consul.client.api.success.catalog_gateway_services.` | Increments whenever a Consul agent successfully responds to a request to list services associated with a gateway. | requests | counter | -| `consul.client.rpc.error.catalog_gateway_services.` | Increments whenever a Consul agent receives an RPC error for a request to list services associated with a gateway. | errors | counter | -| `consul.runtime.num_goroutines` | Tracks the number of running goroutines and is a general load pressure indicator. This may burst from time to time but should return to a steady state value. | number of goroutines | gauge | -| `consul.runtime.alloc_bytes` | Measures the number of bytes allocated by the Consul process. This may burst from time to time but should return to a steady state value. | bytes | gauge | -| `consul.runtime.heap_objects` | Measures the number of objects allocated on the heap and is a general memory pressure indicator. This may burst from time to time but should return to a steady state value. | number of objects | gauge | -| `consul.state.nodes` | Measures the current number of nodes registered with Consul. It is only emitted by Consul servers. Added in v1.9.0. | number of objects | gauge | -| `consul.state.services` | Measures the current number of unique services registered with Consul, based on service name. It is only emitted by Consul servers. Added in v1.9.0. | number of objects | gauge | -| `consul.state.service_instances` | Measures the current number of unique service instances registered with Consul. It is only emitted by Consul servers. Added in v1.9.0. | number of objects | gauge | -| `consul.state.kv_entries` | Measures the current number of unique KV entries written in Consul. It is only emitted by Consul servers. Added in v1.10.3. | number of objects | gauge | -| `consul.state.connect_instances` | Measures the current number of unique connect service instances registered with Consul labeled by Kind (e.g. connect-proxy, connect-native, etc). Added in v1.10.4 | number of objects | gauge | -| `consul.state.config_entries` | Measures the current number of configuration entries registered with Consul labeled by Kind (e.g. service-defaults, proxy-defaults, etc). See [Configuration Entries](/docs/connect/config-entries) for more information. Added in v1.10.4 | number of objects | gauge | -| `consul.members.clients` | Measures the current number of client agents registered with Consul. It is only emitted by Consul servers. Added in v1.9.6. | number of clients | gauge | -| `consul.members.servers` | Measures the current number of server agents registered with Consul. It is only emitted by Consul servers. Added in v1.9.6. | number of servers | gauge | -| `consul.dns.stale_queries` | Increments when an agent serves a query within the allowed stale threshold. | queries | counter | -| `consul.dns.ptr_query.` | Measures the time spent handling a reverse DNS query for the given node. | ms | timer | -| `consul.dns.domain_query.` | Measures the time spent handling a domain query for the given node. | ms | timer | -| `consul.http...` | DEPRECATED IN 1.9: Tracks how long it takes to service the given HTTP request for the given verb and path. Paths do not include details like service or key names, for these an underscore will be present as a placeholder (eg. `consul.http.GET.v1.kv._`) | ms | timer | -| `consul.system.licenseExpiration` | This measures the number of hours remaining on the agents license. | hours | gauge | -| `consul.version` | Represents the Consul version. | agents | gauge | +| Metric | Description | Unit | Type | +|--------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------|---------| +| `consul.acl.blocked.{check,service}.deregistration` | Increments whenever a deregistration fails for an entity (check or service) is blocked by an ACL. | requests | counter | +| `consul.acl.blocked.{check,node,service}.registration` | Increments whenever a registration fails for an entity (check, node or service) is blocked by an ACL. | requests | counter | +| `consul.api.http` | Migrated from `consul.http..`. Samples how long it takes to service the given HTTP request for the given verb and path. Includes labels for `path` and `method`. `path` does not include details like service or key names, for these an underscore will be present as a placeholder (eg. path=`v1.kv._`) | ms | timer | +| `consul.client.rpc` | Increments whenever a Consul agent in client mode makes an RPC request to a Consul server. This gives a measure of how much a given agent is loading the Consul servers. Currently, this is only generated by agents in client mode, not Consul servers. | requests | counter | +| `consul.client.rpc.exceeded` | Increments whenever a Consul agent in client mode makes an RPC request to a Consul server gets rate limited by that agent's [`limits`](/docs/agent/config/config-files#limits) configuration. This gives an indication that there's an abusive application making too many requests on the agent, or that the rate limit needs to be increased. Currently, this only applies to agents in client mode, not Consul servers. | rejected requests | counter | +| `consul.client.rpc.failed` | Increments whenever a Consul agent in client mode makes an RPC request to a Consul server and fails. | requests | counter | +| `consul.client.api.catalog_register` | Increments whenever a Consul agent receives a catalog register request. | requests | counter | +| `consul.client.api.success.catalog_register` | Increments whenever a Consul agent successfully responds to a catalog register request. | requests | counter | +| `consul.client.rpc.error.catalog_register` | Increments whenever a Consul agent receives an RPC error for a catalog register request. | errors | counter | +| `consul.client.api.catalog_deregister` | Increments whenever a Consul agent receives a catalog deregister request. | requests | counter | +| `consul.client.api.success.catalog_deregister` | Increments whenever a Consul agent successfully responds to a catalog deregister request. | requests | counter | +| `consul.client.rpc.error.catalog_deregister` | Increments whenever a Consul agent receives an RPC error for a catalog deregister request. | errors | counter | +| `consul.client.api.catalog_datacenters` | Increments whenever a Consul agent receives a request to list datacenters in the catalog. | requests | counter | +| `consul.client.api.success.catalog_datacenters` | Increments whenever a Consul agent successfully responds to a request to list datacenters. | requests | counter | +| `consul.client.rpc.error.catalog_datacenters` | Increments whenever a Consul agent receives an RPC error for a request to list datacenters. | errors | counter | +| `consul.client.api.catalog_nodes` | Increments whenever a Consul agent receives a request to list nodes from the catalog. | requests | counter | +| `consul.client.api.success.catalog_nodes` | Increments whenever a Consul agent successfully responds to a request to list nodes. | requests | counter | +| `consul.client.rpc.error.catalog_nodes` | Increments whenever a Consul agent receives an RPC error for a request to list nodes. | errors | counter | +| `consul.client.api.catalog_services` | Increments whenever a Consul agent receives a request to list services from the catalog. | requests | counter | +| `consul.client.api.success.catalog_services` | Increments whenever a Consul agent successfully responds to a request to list services. | requests | counter | +| `consul.client.rpc.error.catalog_services` | Increments whenever a Consul agent receives an RPC error for a request to list services. | errors | counter | +| `consul.client.api.catalog_service_nodes` | Increments whenever a Consul agent receives a request to list nodes offering a service. | requests | counter | +| `consul.client.api.success.catalog_service_nodes` | Increments whenever a Consul agent successfully responds to a request to list nodes offering a service. | requests | counter | +| `consul.client.api.error.catalog_service_nodes` | Increments whenever a Consul agent receives an RPC error for request to list nodes offering a service. | requests | counter | +| `consul.client.rpc.error.catalog_service_nodes` | Increments whenever a Consul agent receives an RPC error for a request to list nodes offering a service. | errors | counter | +| `consul.client.api.catalog_node_services` | Increments whenever a Consul agent receives a request to list services registered in a node. | requests | counter | +| `consul.client.api.success.catalog_node_services` | Increments whenever a Consul agent successfully responds to a request to list services in a node. | requests | counter | +| `consul.client.rpc.error.catalog_node_services` | Increments whenever a Consul agent receives an RPC error for a request to list services in a node. | errors | counter | +| `consul.client.api.catalog_node_service_list` | Increments whenever a Consul agent receives a request to list a node's registered services. | requests | counter | +| `consul.client.rpc.error.catalog_node_service_list` | Increments whenever a Consul agent receives an RPC error for request to list a node's registered services. | errors | counter | +| `consul.client.api.success.catalog_node_service_list` | Increments whenever a Consul agent successfully responds to a request to list a node's registered services. | requests | counter | +| `consul.client.api.catalog_gateway_services` | Increments whenever a Consul agent receives a request to list services associated with a gateway. | requests | counter | +| `consul.client.api.success.catalog_gateway_services` | Increments whenever a Consul agent successfully responds to a request to list services associated with a gateway. | requests | counter | +| `consul.client.rpc.error.catalog_gateway_services` | Increments whenever a Consul agent receives an RPC error for a request to list services associated with a gateway. | errors | counter | +| `consul.runtime.num_goroutines` | Tracks the number of running goroutines and is a general load pressure indicator. This may burst from time to time but should return to a steady state value. | number of goroutines | gauge | +| `consul.runtime.alloc_bytes` | Measures the number of bytes allocated by the Consul process. This may burst from time to time but should return to a steady state value. | bytes | gauge | +| `consul.runtime.heap_objects` | Measures the number of objects allocated on the heap and is a general memory pressure indicator. This may burst from time to time but should return to a steady state value. | number of objects | gauge | +| `consul.state.nodes` | Measures the current number of nodes registered with Consul. It is only emitted by Consul servers. Added in v1.9.0. | number of objects | gauge | +| `consul.state.services` | Measures the current number of unique services registered with Consul, based on service name. It is only emitted by Consul servers. Added in v1.9.0. | number of objects | gauge | +| `consul.state.service_instances` | Measures the current number of unique service instances registered with Consul. It is only emitted by Consul servers. Added in v1.9.0. | number of objects | gauge | +| `consul.state.kv_entries` | Measures the current number of unique KV entries written in Consul. It is only emitted by Consul servers. Added in v1.10.3. | number of objects | gauge | +| `consul.state.connect_instances` | Measures the current number of unique connect service instances registered with Consul labeled by Kind (e.g. connect-proxy, connect-native, etc). Added in v1.10.4 | number of objects | gauge | +| `consul.state.config_entries` | Measures the current number of configuration entries registered with Consul labeled by Kind (e.g. service-defaults, proxy-defaults, etc). See [Configuration Entries](/docs/connect/config-entries) for more information. Added in v1.10.4 | number of objects | gauge | +| `consul.members.clients` | Measures the current number of client agents registered with Consul. It is only emitted by Consul servers. Added in v1.9.6. | number of clients | gauge | +| `consul.members.servers` | Measures the current number of server agents registered with Consul. It is only emitted by Consul servers. Added in v1.9.6. | number of servers | gauge | +| `consul.dns.stale_queries` | Increments when an agent serves a query within the allowed stale threshold. | queries | counter | +| `consul.dns.ptr_query` | Measures the time spent handling a reverse DNS query for the given node. | ms | timer | +| `consul.dns.domain_query` | Measures the time spent handling a domain query for the given node. | ms | timer | +| `consul.http..` | DEPRECATED IN 1.9: Tracks how long it takes to service the given HTTP request for the given verb and path. Paths do not include details like service or key names, for these an underscore will be present as a placeholder (eg. `consul.http.GET.v1.kv._`) | ms | timer | +| `consul.system.licenseExpiration` | This measures the number of hours remaining on the agents license. | hours | gauge | +| `consul.version` | Represents the Consul version. | agents | gauge | ## Server Health These metrics are used to monitor the health of the Consul servers. -| Metric | Description | Unit | Type | -| --------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------- | ------- | -| `consul.acl.ResolveToken` | Measures the time it takes to resolve an ACL token. | ms | timer | -| `consul.acl.ResolveTokenToIdentity` | Measures the time it takes to resolve an ACL token to an Identity. This metric was removed in Consul 1.12. The time will now be reflected in `consul.acl.ResolveToken`. | ms | timer | -| `consul.acl.token.cache_hit` | Increments if Consul is able to resolve a token's identity, or a legacy token, from the cache. | cache read op | counter | -| `consul.acl.token.cache_miss` | Increments if Consul cannot resolve a token's identity, or a legacy token, from the cache. | cache read op | counter | -| `consul.cache.bypass` | Counts how many times a request bypassed the cache because no cache-key was provided. | counter | counter | -| `consul.cache.fetch_success` | Counts the number of successful fetches by the cache. | counter | counter | -| `consul.cache.fetch_error` | Counts the number of failed fetches by the cache. | counter | counter | -| `consul.cache.evict_expired` | Counts the number of expired entries that are evicted. | counter | counter | -| `consul.raft.applied_index` | Represents the raft applied index. | index | gauge | -| `consul.raft.apply` | Counts the number of Raft transactions occurring over the interval, which is a general indicator of the write load on the Consul servers. | raft transactions / interval | counter | -| `consul.raft.barrier` | Counts the number of times the agent has started the barrier i.e the number of times it has issued a blocking call, to ensure that the agent has all the pending operations that were queued, to be applied to the agent's FSM. | blocks / interval | counter | -| `consul.raft.boltdb.freelistBytes` | Represents the number of bytes necessary to encode the freelist metadata. When [`raft_boltdb.NoFreelistSync`](/docs/agent/options#NoFreelistSync) is set to `false` these metadata bytes must also be written to disk for each committed log. | bytes | gauge | -| `consul.raft.boltdb.freePageBytes` | Represents the number of bytes of free space within the raft.db file. | bytes | gauge | -| `consul.raft.boltdb.getLog` | Measures the amount of time spent reading logs from the db. | ms | timer | -| `consul.raft.boltdb.logBatchSize` | Measures the total size in bytes of logs being written to the db in a single batch. | bytes | sample | -| `consul.raft.boltdb.logsPerBatch` | Measures the number of logs being written per batch to the db. | logs | sample | -| `consul.raft.boltdb.logSize` | Measures the size of logs being written to the db. | bytes | sample | -| `consul.raft.boltdb.numFreePages` | Represents the number of free pages within the raft.db file. | pages | gauge | -| `consul.raft.boltdb.numPendingPages` | Represents the number of pending pages within the raft.db that will soon become free. | pages | gauge | -| `consul.raft.boltdb.openReadTxn` | Represents the number of open read transactions against the db | transactions | gauge | -| `consul.raft.boltdb.totalReadTxn` | Represents the total number of started read transactions against the db | transactions | gauge | -| `consul.raft.boltdb.storeLogs` | Measures the amount of time spent writing logs to the db. | ms | timer | -| `consul.raft.boltdb.txstats.cursorCount` | Counts the number of cursors created since Consul was started. | cursors | counter | -| `consul.raft.boltdb.txstats.nodeCount` | Counts the number of node allocations within the db since Consul was started. | allocations | counter | -| `consul.raft.boltdb.txstats.nodeDeref` | Counts the number of node dereferences in the db since Consul was started. | dereferences | counter | -| `consul.raft.boltdb.txstats.pageAlloc` | Represents the number of bytes allocated within the db since Consul was started. Note that this does not take into account space having been freed and reused. In that case, the value of this metric will still increase. | bytes | gauge | -| `consul.raft.boltdb.txstats.pageCount` | Represents the number of pages allocated since Consul was started. Note that this does not take into account space having been freed and reused. In that case, the value of this metric will still increase. | pages | gauge | -| `consul.raft.boltdb.txstats.rebalance` | Counts the number of node rebalances performed in the db since Consul was started. | rebalances | counter | -| `consul.raft.boltdb.txstats.rebalanceTime` | Measures the time spent rebalancing nodes in the db. | ms | timer | -| `consul.raft.boltdb.txstats.spill` | Counts the number of nodes spilled in the db since Consul was started. | spills | counter | -| `consul.raft.boltdb.txstats.spillTime` | Measures the time spent spilling nodes in the db. | ms | timer | -| `consul.raft.boltdb.txstats.split` | Counts the number of nodes split in the db since Consul was started. | splits | counter | -| `consul.raft.boltdb.txstats.write` | Counts the number of writes to the db since Consul was started. | writes | counter | -| `consul.raft.boltdb.txstats.writeTime` | Measures the amount of time spent performing writes to the db. | ms | timer | -| `consul.raft.boltdb.writeCapacity` | Theoretical write capacity in terms of the number of logs that can be written per second. Each sample outputs what the capacity would be if future batched log write operations were similar to this one. This similarity encompasses 4 things: batch size, byte size, disk performance and boltdb performance. While none of these will be static and its highly likely individual samples of this metric will vary, aggregating this metric over a larger time window should provide a decent picture into how this BoltDB store can perform | logs/second | sample | -| `consul.raft.commitNumLogs` | Measures the count of logs processed for application to the FSM in a single batch. | logs | gauge | -| `consul.raft.commitTime` | Measures the time it takes to commit a new entry to the Raft log on the leader. | ms | timer | -| `consul.raft.fsm.lastRestoreDuration` | Measures the time taken to restore the FSM from a snapshot on an agent restart or from the leader calling installSnapshot. This is a gauge that holds it's value since most servers only restore during restarts which are typically infrequent. | ms | gauge | -| `consul.raft.fsm.snapshot` | Measures the time taken by the FSM to record the current state for the snapshot. | ms | timer | -| `consul.raft.fsm.apply` | Measures the time to apply a log to the FSM. | ms | timer | -| `consul.raft.fsm.enqueue` | Measures the amount of time to enqueue a batch of logs for the FSM to apply. | ms | timer | -| `consul.raft.fsm.restore` | Measures the time taken by the FSM to restore its state from a snapshot. | ms | timer | -| `consul.raft.last_index` | Represents the raft applied index. | index | gauge | -| `consul.raft.leader.dispatchLog` | Measures the time it takes for the leader to write log entries to disk. | ms | timer | -| `consul.raft.leader.dispatchNumLogs` | Measures the number of logs committed to disk in a batch. | logs | gauge | -| `consul.raft.leader.lastContact` | Measures the time since the leader was last able to contact the follower nodes when checking its leader lease. It can be used as a measure for how stable the Raft timing is and how close the leader is to timing out its lease.The lease timeout is 500 ms times the [`raft_multiplier` configuration](/docs/agent/config/config-files#raft_multiplier), so this telemetry value should not be getting close to that configured value, otherwise the Raft timing is marginal and might need to be tuned, or more powerful servers might be needed. See the [Server Performance](/docs/install/performance) guide for more details. | ms | timer | +| Metric | Description | Unit | Type | +|-----------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------|---------| +| `consul.acl.ResolveToken` | Measures the time it takes to resolve an ACL token. | ms | timer | +| `consul.acl.ResolveTokenToIdentity` | Measures the time it takes to resolve an ACL token to an Identity. This metric was removed in Consul 1.12. The time will now be reflected in `consul.acl.ResolveToken`. | ms | timer | +| `consul.acl.token.cache_hit` | Increments if Consul is able to resolve a token's identity, or a legacy token, from the cache. | cache read op | counter | +| `consul.acl.token.cache_miss` | Increments if Consul cannot resolve a token's identity, or a legacy token, from the cache. | cache read op | counter | +| `consul.cache.bypass` | Counts how many times a request bypassed the cache because no cache-key was provided. | counter | counter | +| `consul.cache.fetch_success` | Counts the number of successful fetches by the cache. | counter | counter | +| `consul.cache.fetch_error` | Counts the number of failed fetches by the cache. | counter | counter | +| `consul.cache.evict_expired` | Counts the number of expired entries that are evicted. | counter | counter | +| `consul.raft.applied_index` | Represents the raft applied index. | index | gauge | +| `consul.raft.apply` | Counts the number of Raft transactions occurring over the interval, which is a general indicator of the write load on the Consul servers. | raft transactions / interval | counter | +| `consul.raft.barrier` | Counts the number of times the agent has started the barrier i.e the number of times it has issued a blocking call, to ensure that the agent has all the pending operations that were queued, to be applied to the agent's FSM. | blocks / interval | counter | +| `consul.raft.boltdb.freelistBytes` | Represents the number of bytes necessary to encode the freelist metadata. When [`raft_boltdb.NoFreelistSync`](/docs/agent/options#NoFreelistSync) is set to `false` these metadata bytes must also be written to disk for each committed log. | bytes | gauge | +| `consul.raft.boltdb.freePageBytes` | Represents the number of bytes of free space within the raft.db file. | bytes | gauge | +| `consul.raft.boltdb.getLog` | Measures the amount of time spent reading logs from the db. | ms | timer | +| `consul.raft.boltdb.logBatchSize` | Measures the total size in bytes of logs being written to the db in a single batch. | bytes | sample | +| `consul.raft.boltdb.logsPerBatch` | Measures the number of logs being written per batch to the db. | logs | sample | +| `consul.raft.boltdb.logSize` | Measures the size of logs being written to the db. | bytes | sample | +| `consul.raft.boltdb.numFreePages` | Represents the number of free pages within the raft.db file. | pages | gauge | +| `consul.raft.boltdb.numPendingPages` | Represents the number of pending pages within the raft.db that will soon become free. | pages | gauge | +| `consul.raft.boltdb.openReadTxn` | Represents the number of open read transactions against the db | transactions | gauge | +| `consul.raft.boltdb.totalReadTxn` | Represents the total number of started read transactions against the db | transactions | gauge | +| `consul.raft.boltdb.storeLogs` | Measures the amount of time spent writing logs to the db. | ms | timer | +| `consul.raft.boltdb.txstats.cursorCount` | Counts the number of cursors created since Consul was started. | cursors | counter | +| `consul.raft.boltdb.txstats.nodeCount` | Counts the number of node allocations within the db since Consul was started. | allocations | counter | +| `consul.raft.boltdb.txstats.nodeDeref` | Counts the number of node dereferences in the db since Consul was started. | dereferences | counter | +| `consul.raft.boltdb.txstats.pageAlloc` | Represents the number of bytes allocated within the db since Consul was started. Note that this does not take into account space having been freed and reused. In that case, the value of this metric will still increase. | bytes | gauge | +| `consul.raft.boltdb.txstats.pageCount` | Represents the number of pages allocated since Consul was started. Note that this does not take into account space having been freed and reused. In that case, the value of this metric will still increase. | pages | gauge | +| `consul.raft.boltdb.txstats.rebalance` | Counts the number of node rebalances performed in the db since Consul was started. | rebalances | counter | +| `consul.raft.boltdb.txstats.rebalanceTime` | Measures the time spent rebalancing nodes in the db. | ms | timer | +| `consul.raft.boltdb.txstats.spill` | Counts the number of nodes spilled in the db since Consul was started. | spills | counter | +| `consul.raft.boltdb.txstats.spillTime` | Measures the time spent spilling nodes in the db. | ms | timer | +| `consul.raft.boltdb.txstats.split` | Counts the number of nodes split in the db since Consul was started. | splits | counter | +| `consul.raft.boltdb.txstats.write` | Counts the number of writes to the db since Consul was started. | writes | counter | +| `consul.raft.boltdb.txstats.writeTime` | Measures the amount of time spent performing writes to the db. | ms | timer | +| `consul.raft.boltdb.writeCapacity` | Theoretical write capacity in terms of the number of logs that can be written per second. Each sample outputs what the capacity would be if future batched log write operations were similar to this one. This similarity encompasses 4 things: batch size, byte size, disk performance and boltdb performance. While none of these will be static and its highly likely individual samples of this metric will vary, aggregating this metric over a larger time window should provide a decent picture into how this BoltDB store can perform | logs/second | sample | +| `consul.raft.commitNumLogs` | Measures the count of logs processed for application to the FSM in a single batch. | logs | gauge | +| `consul.raft.commitTime` | Measures the time it takes to commit a new entry to the Raft log on the leader. | ms | timer | +| `consul.raft.fsm.lastRestoreDuration` | Measures the time taken to restore the FSM from a snapshot on an agent restart or from the leader calling installSnapshot. This is a gauge that holds it's value since most servers only restore during restarts which are typically infrequent. | ms | gauge | +| `consul.raft.fsm.snapshot` | Measures the time taken by the FSM to record the current state for the snapshot. | ms | timer | +| `consul.raft.fsm.apply` | Measures the time to apply a log to the FSM. | ms | timer | +| `consul.raft.fsm.enqueue` | Measures the amount of time to enqueue a batch of logs for the FSM to apply. | ms | timer | +| `consul.raft.fsm.restore` | Measures the time taken by the FSM to restore its state from a snapshot. | ms | timer | +| `consul.raft.last_index` | Represents the raft applied index. | index | gauge | +| `consul.raft.leader.dispatchLog` | Measures the time it takes for the leader to write log entries to disk. | ms | timer | +| `consul.raft.leader.dispatchNumLogs` | Measures the number of logs committed to disk in a batch. | logs | gauge | +| `consul.raft.leader.lastContact` | Measures the time since the leader was last able to contact the follower nodes when checking its leader lease. It can be used as a measure for how stable the Raft timing is and how close the leader is to timing out its lease.The lease timeout is 500 ms times the [`raft_multiplier` configuration](/docs/agent/config/config-files#raft_multiplier), so this telemetry value should not be getting close to that configured value, otherwise the Raft timing is marginal and might need to be tuned, or more powerful servers might be needed. See the [Server Performance](/docs/install/performance) guide for more details. | ms | timer | | `consul.raft.leader.oldestLogAge` | The number of milliseconds since the _oldest_ log in the leader's log store was written. This can be important for replication health where write rate is high and the snapshot is large as followers may be unable to recover from a restart if restoring takes longer than the minimum value for the current leader. Compare this with `consul.raft.fsm.lastRestoreDuration` and `consul.raft.rpc.installSnapshot` to monitor. In normal usage this gauge value will grow linearly over time until a snapshot completes on the leader and the log is truncated. Note: this metric won't be emitted until the leader writes a snapshot. After an upgrade to Consul 1.10.0 it won't be emitted until the oldest log was written after the upgrade. | ms | gauge | -| `consul.raft.replication.heartbeat` | Measures the time taken to invoke appendEntries on a peer, so that it doesn’t timeout on a periodic basis. | ms | timer | -| `consul.raft.replication.appendEntries` | Measures the time it takes to replicate log entries to followers. This is a general indicator of the load pressure on the Consul servers, as well as the performance of the communication between the servers. | ms | timer | -| `consul.raft.replication.appendEntries.rpc` | Measures the time taken by the append entries RFC, to replicate the log entries of a leader agent onto its follower agent(s) | ms | timer | -| `consul.raft.replication.appendEntries.logs` | Measures the number of logs replicated to an agent, to bring it up to speed with the leader's logs. | logs appended/ interval | counter | -| `consul.raft.restore` | Counts the number of times the restore operation has been performed by the agent. Here, restore refers to the action of raft consuming an external snapshot to restore its state. | operation invoked / interval | counter | -| `consul.raft.restoreUserSnapshot` | Measures the time taken by the agent to restore the FSM state from a user's snapshot | ms | timer | -| `consul.raft.rpc.appendEntries` | Measures the time taken to process an append entries RPC call from an agent. | ms | timer | -| `consul.raft.rpc.appendEntries.storeLogs` | Measures the time taken to add any outstanding logs for an agent, since the last appendEntries was invoked | ms | timer | -| `consul.raft.rpc.appendEntries.processLogs` | Measures the time taken to process the outstanding log entries of an agent. | ms | timer | -| `consul.raft.rpc.installSnapshot` | Measures the time taken to process the installSnapshot RPC call. This metric should only be seen on agents which are currently in the follower state. | ms | timer | -| `consul.raft.rpc.processHeartBeat` | Measures the time taken to process a heartbeat request. | ms | timer | -| `consul.raft.rpc.requestVote` | Measures the time taken to process the request vote RPC call. | ms | timer | -| `consul.raft.snapshot.create` | Measures the time taken to initialize the snapshot process. | ms | timer | -| `consul.raft.snapshot.persist` | Measures the time taken to dump the current snapshot taken by the Consul agent to the disk. | ms | timer | -| `consul.raft.snapshot.takeSnapshot` | Measures the total time involved in taking the current snapshot (creating one and persisting it) by the Consul agent. | ms | timer | -| `consul.serf.snapshot.appendLine` | Measures the time taken by the Consul agent to append an entry into the existing log. | ms | timer | -| `consul.serf.snapshot.compact` | Measures the time taken by the Consul agent to compact a log. This operation occurs only when the snapshot becomes large enough to justify the compaction . | ms | timer | -| `consul.raft.state.candidate` | Increments whenever a Consul server starts an election. If this increments without a leadership change occurring it could indicate that a single server is overloaded or is experiencing network connectivity issues. | election attempts / interval | counter | -| `consul.raft.state.leader` | Increments whenever a Consul server becomes a leader. If there are frequent leadership changes this may be indication that the servers are overloaded and aren't meeting the soft real-time requirements for Raft, or that there are networking problems between the servers. | leadership transitions / interval | counter | -| `consul.raft.state.follower` | Counts the number of times an agent has entered the follower mode. This happens when a new agent joins the cluster or after the end of a leader election. | follower state entered / interval | counter | -| `consul.raft.transition.heartbeat_timeout` | The number of times an agent has transitioned to the Candidate state, after receive no heartbeat messages from the last known leader. | timeouts / interval | counter | -| `consul.raft.verify_leader` | This metric doesn't have a direct correlation to the leader change. It just counts the number of times an agent checks if it is still the leader or not. For example, during every consistent read, the check is done. Depending on the load in the system, this metric count can be high as it is incremented each time a consistent read is completed. | checks / interval | Counter | -| `consul.rpc.accept_conn` | Increments when a server accepts an RPC connection. | connections | counter | -| `consul.catalog.register` | Measures the time it takes to complete a catalog register operation. | ms | timer | -| `consul.catalog.deregister` | Measures the time it takes to complete a catalog deregister operation. | ms | timer | -| `consul.server.isLeader` | Track if a server is a leader(1) or not(0) | 1 or 0 | gauge | -| `consul.fsm.register` | Measures the time it takes to apply a catalog register operation to the FSM. | ms | timer | -| `consul.fsm.deregister` | Measures the time it takes to apply a catalog deregister operation to the FSM. | ms | timer | -| `consul.fsm.session.` | Measures the time it takes to apply the given session operation to the FSM. | ms | timer | -| `consul.fsm.kvs.` | Measures the time it takes to apply the given KV operation to the FSM. | ms | timer | -| `consul.fsm.tombstone.` | Measures the time it takes to apply the given tombstone operation to the FSM. | ms | timer | -| `consul.fsm.coordinate.batch-update` | Measures the time it takes to apply the given batch coordinate update to the FSM. | ms | timer | -| `consul.fsm.prepared-query.` | Measures the time it takes to apply the given prepared query update operation to the FSM. | ms | timer | -| `consul.fsm.txn` | Measures the time it takes to apply the given transaction update to the FSM. | ms | timer | -| `consul.fsm.autopilot` | Measures the time it takes to apply the given autopilot update to the FSM. | ms | timer | -| `consul.fsm.persist` | Measures the time it takes to persist the FSM to a raft snapshot. | ms | timer | -| `consul.fsm.intention` | Measures the time it takes to apply an intention operation to the state store. | ms | timer | -| `consul.fsm.ca` | Measures the time it takes to apply CA configuration operations to the FSM. | ms | timer | -| `consul.fsm.ca.leaf` | Measures the time it takes to apply an operation while signing a leaf certificate. | ms | timer | -| `consul.fsm.acl.token` | Measures the time it takes to apply an ACL token operation to the FSM. | ms | timer | -| `consul.fsm.acl.policy` | Measures the time it takes to apply an ACL policy operation to the FSM. | ms | timer | -| `consul.fsm.acl.bindingrule` | Measures the time it takes to apply an ACL binding rule operation to the FSM. | ms | timer | -| `consul.fsm.acl.authmethod` | Measures the time it takes to apply an ACL authmethod operation to the FSM. | ms | timer | -| `consul.fsm.system_metadata` | Measures the time it takes to apply a system metadata operation to the FSM. | ms | timer | -| `consul.kvs.apply` | Measures the time it takes to complete an update to the KV store. | ms | timer | -| `consul.leader.barrier` | Measures the time spent waiting for the raft barrier upon gaining leadership. | ms | timer | -| `consul.leader.reconcile` | Measures the time spent updating the raft store from the serf member information. | ms | timer | -| `consul.leader.reconcileMember` | Measures the time spent updating the raft store for a single serf member's information. | ms | timer | -| `consul.leader.reapTombstones` | Measures the time spent clearing tombstones. | ms | timer | -| `consul.leader.replication.acl-policies.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of ACL policy replication was successful or 0 if there was an error. | healthy | gauge | -| `consul.leader.replication.acl-policies.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of ACL policies in the primary datacenter that have been successfully replicated. | index | gauge | -| `consul.leader.replication.acl-roles.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of ACL role replication was successful or 0 if there was an error. | healthy | gauge | -| `consul.leader.replication.acl-roles.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of ACL roles in the primary datacenter that have been successfully replicated. | index | gauge | -| `consul.leader.replication.acl-tokens.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of ACL token replication was successful or 0 if there was an error. | healthy | gauge | -| `consul.leader.replication.acl-tokens.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of ACL tokens in the primary datacenter that have been successfully replicated. | index | gauge | -| `consul.leader.replication.config-entries.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of config entry replication was successful or 0 if there was an error. | healthy | gauge | -| `consul.leader.replication.config-entries.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of config entries in the primary datacenter that have been successfully replicated. | index | gauge | -| `consul.leader.replication.federation-state.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of federation state replication was successful or 0 if there was an error. | healthy | gauge | -| `consul.leader.replication.federation-state.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of federation states in the primary datacenter that have been successfully replicated. | index | gauge | -| `consul.leader.replication.namespaces.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of namespace replication was successful or 0 if there was an error. | healthy | gauge | -| `consul.leader.replication.namespaces.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of namespaces in the primary datacenter that have been successfully replicated. | index | gauge | -| `consul.prepared-query.apply` | Measures the time it takes to apply a prepared query update. | ms | timer | -| `consul.prepared-query.explain` | Measures the time it takes to process a prepared query explain request. | ms | timer | -| `consul.prepared-query.execute` | Measures the time it takes to process a prepared query execute request. | ms | timer | -| `consul.prepared-query.execute_remote` | Measures the time it takes to process a prepared query execute request that was forwarded to another datacenter. | ms | timer | -| `consul.rpc.raft_handoff` | Increments when a server accepts a Raft-related RPC connection. | connections | counter | -| `consul.rpc.request_error` | Increments when a server returns an error from an RPC request. | errors | counter | -| `consul.rpc.request` | Increments when a server receives a Consul-related RPC request. | requests | counter | -| `consul.rpc.query` | Increments when a server receives a read RPC request, indicating the rate of new read queries. See consul.rpc.queries_blocking for the current number of in-flight blocking RPC calls. This metric changed in 1.7.0 to only increment on the the start of a query. The rate of queries will appear lower, but is more accurate. | queries | counter | -| `consul.rpc.queries_blocking` | The current number of in-flight blocking queries the server is handling. | queries | gauge | -| `consul.rpc.cross-dc` | Increments when a server sends a (potentially blocking) cross datacenter RPC query. | queries | counter | -| `consul.rpc.consistentRead` | Measures the time spent confirming that a consistent read can be performed. | ms | timer | -| `consul.session.apply` | Measures the time spent applying a session update. | ms | timer | -| `consul.session.renew` | Measures the time spent renewing a session. | ms | timer | -| `consul.session_ttl.invalidate` | Measures the time spent invalidating an expired session. | ms | timer | -| `consul.txn.apply` | Measures the time spent applying a transaction operation. | ms | timer | -| `consul.txn.read` | Measures the time spent returning a read transaction. | ms | timer | -| `consul.grpc.client.request.count` | Counts the number of gRPC requests made by the client agent to a Consul server. | requests | counter | -| `consul.grpc.client.connection.count` | Counts the number of new gRPC connections opened by the client agent to a Consul server. | connections | counter | -| `consul.grpc.client.connections` | Measures the number of active gRPC connections open from the client agent to any Consul servers. | connections | gauge | -| `consul.grpc.server.request.count` | Counts the number of gRPC requests received by the server. | requests | counter | -| `consul.grpc.server.connection.count` | Counts the number of new gRPC connections received by the server. | connections | counter | -| `consul.grpc.server.connections` | Measures the number of active gRPC connections open on the server. | connections | gauge | -| `consul.grpc.server.stream.count` | Counts the number of new gRPC streams received by the server. | streams | counter | -| `consul.grpc.server.streams` | Measures the number of active gRPC streams handled by the server. | streams | gauge | -| `consul.xds.server.streams` | Measures the number of active xDS streams handled by the server split by protocol version. | streams | gauge | +| `consul.raft.replication.heartbeat` | Measures the time taken to invoke appendEntries on a peer, so that it doesn’t timeout on a periodic basis. | ms | timer | +| `consul.raft.replication.appendEntries` | Measures the time it takes to replicate log entries to followers. This is a general indicator of the load pressure on the Consul servers, as well as the performance of the communication between the servers. | ms | timer | +| `consul.raft.replication.appendEntries.rpc` | Measures the time taken by the append entries RFC, to replicate the log entries of a leader agent onto its follower agent(s) | ms | timer | +| `consul.raft.replication.appendEntries.logs` | Measures the number of logs replicated to an agent, to bring it up to speed with the leader's logs. | logs appended/ interval | counter | +| `consul.raft.restore` | Counts the number of times the restore operation has been performed by the agent. Here, restore refers to the action of raft consuming an external snapshot to restore its state. | operation invoked / interval | counter | +| `consul.raft.restoreUserSnapshot` | Measures the time taken by the agent to restore the FSM state from a user's snapshot | ms | timer | +| `consul.raft.rpc.appendEntries` | Measures the time taken to process an append entries RPC call from an agent. | ms | timer | +| `consul.raft.rpc.appendEntries.storeLogs` | Measures the time taken to add any outstanding logs for an agent, since the last appendEntries was invoked | ms | timer | +| `consul.raft.rpc.appendEntries.processLogs` | Measures the time taken to process the outstanding log entries of an agent. | ms | timer | +| `consul.raft.rpc.installSnapshot` | Measures the time taken to process the installSnapshot RPC call. This metric should only be seen on agents which are currently in the follower state. | ms | timer | +| `consul.raft.rpc.processHeartBeat` | Measures the time taken to process a heartbeat request. | ms | timer | +| `consul.raft.rpc.requestVote` | Measures the time taken to process the request vote RPC call. | ms | timer | +| `consul.raft.snapshot.create` | Measures the time taken to initialize the snapshot process. | ms | timer | +| `consul.raft.snapshot.persist` | Measures the time taken to dump the current snapshot taken by the Consul agent to the disk. | ms | timer | +| `consul.raft.snapshot.takeSnapshot` | Measures the total time involved in taking the current snapshot (creating one and persisting it) by the Consul agent. | ms | timer | +| `consul.serf.snapshot.appendLine` | Measures the time taken by the Consul agent to append an entry into the existing log. | ms | timer | +| `consul.serf.snapshot.compact` | Measures the time taken by the Consul agent to compact a log. This operation occurs only when the snapshot becomes large enough to justify the compaction . | ms | timer | +| `consul.raft.state.candidate` | Increments whenever a Consul server starts an election. If this increments without a leadership change occurring it could indicate that a single server is overloaded or is experiencing network connectivity issues. | election attempts / interval | counter | +| `consul.raft.state.leader` | Increments whenever a Consul server becomes a leader. If there are frequent leadership changes this may be indication that the servers are overloaded and aren't meeting the soft real-time requirements for Raft, or that there are networking problems between the servers. | leadership transitions / interval | counter | +| `consul.raft.state.follower` | Counts the number of times an agent has entered the follower mode. This happens when a new agent joins the cluster or after the end of a leader election. | follower state entered / interval | counter | +| `consul.raft.transition.heartbeat_timeout` | The number of times an agent has transitioned to the Candidate state, after receive no heartbeat messages from the last known leader. | timeouts / interval | counter | +| `consul.raft.verify_leader` | This metric doesn't have a direct correlation to the leader change. It just counts the number of times an agent checks if it is still the leader or not. For example, during every consistent read, the check is done. Depending on the load in the system, this metric count can be high as it is incremented each time a consistent read is completed. | checks / interval | Counter | +| `consul.rpc.accept_conn` | Increments when a server accepts an RPC connection. | connections | counter | +| `consul.catalog.register` | Measures the time it takes to complete a catalog register operation. | ms | timer | +| `consul.catalog.deregister` | Measures the time it takes to complete a catalog deregister operation. | ms | timer | +| `consul.server.isLeader` | Track if a server is a leader(1) or not(0) | 1 or 0 | gauge | +| `consul.fsm.register` | Measures the time it takes to apply a catalog register operation to the FSM. | ms | timer | +| `consul.fsm.deregister` | Measures the time it takes to apply a catalog deregister operation to the FSM. | ms | timer | +| `consul.fsm.session` | Measures the time it takes to apply the given session operation to the FSM. | ms | timer | +| `consul.fsm.kvs` | Measures the time it takes to apply the given KV operation to the FSM. | ms | timer | +| `consul.fsm.tombstone` | Measures the time it takes to apply the given tombstone operation to the FSM. | ms | timer | +| `consul.fsm.coordinate.batch-update` | Measures the time it takes to apply the given batch coordinate update to the FSM. | ms | timer | +| `consul.fsm.prepared-query` | Measures the time it takes to apply the given prepared query update operation to the FSM. | ms | timer | +| `consul.fsm.txn` | Measures the time it takes to apply the given transaction update to the FSM. | ms | timer | +| `consul.fsm.autopilot` | Measures the time it takes to apply the given autopilot update to the FSM. | ms | timer | +| `consul.fsm.persist` | Measures the time it takes to persist the FSM to a raft snapshot. | ms | timer | +| `consul.fsm.intention` | Measures the time it takes to apply an intention operation to the state store. | ms | timer | +| `consul.fsm.ca` | Measures the time it takes to apply CA configuration operations to the FSM. | ms | timer | +| `consul.fsm.ca.leaf` | Measures the time it takes to apply an operation while signing a leaf certificate. | ms | timer | +| `consul.fsm.acl.token` | Measures the time it takes to apply an ACL token operation to the FSM. | ms | timer | +| `consul.fsm.acl.policy` | Measures the time it takes to apply an ACL policy operation to the FSM. | ms | timer | +| `consul.fsm.acl.bindingrule` | Measures the time it takes to apply an ACL binding rule operation to the FSM. | ms | timer | +| `consul.fsm.acl.authmethod` | Measures the time it takes to apply an ACL authmethod operation to the FSM. | ms | timer | +| `consul.fsm.system_metadata` | Measures the time it takes to apply a system metadata operation to the FSM. | ms | timer | +| `consul.kvs.apply` | Measures the time it takes to complete an update to the KV store. | ms | timer | +| `consul.leader.barrier` | Measures the time spent waiting for the raft barrier upon gaining leadership. | ms | timer | +| `consul.leader.reconcile` | Measures the time spent updating the raft store from the serf member information. | ms | timer | +| `consul.leader.reconcileMember` | Measures the time spent updating the raft store for a single serf member's information. | ms | timer | +| `consul.leader.reapTombstones` | Measures the time spent clearing tombstones. | ms | timer | +| `consul.leader.replication.acl-policies.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of ACL policy replication was successful or 0 if there was an error. | healthy | gauge | +| `consul.leader.replication.acl-policies.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of ACL policies in the primary datacenter that have been successfully replicated. | index | gauge | +| `consul.leader.replication.acl-roles.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of ACL role replication was successful or 0 if there was an error. | healthy | gauge | +| `consul.leader.replication.acl-roles.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of ACL roles in the primary datacenter that have been successfully replicated. | index | gauge | +| `consul.leader.replication.acl-tokens.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of ACL token replication was successful or 0 if there was an error. | healthy | gauge | +| `consul.leader.replication.acl-tokens.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of ACL tokens in the primary datacenter that have been successfully replicated. | index | gauge | +| `consul.leader.replication.config-entries.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of config entry replication was successful or 0 if there was an error. | healthy | gauge | +| `consul.leader.replication.config-entries.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of config entries in the primary datacenter that have been successfully replicated. | index | gauge | +| `consul.leader.replication.federation-state.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of federation state replication was successful or 0 if there was an error. | healthy | gauge | +| `consul.leader.replication.federation-state.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of federation states in the primary datacenter that have been successfully replicated. | index | gauge | +| `consul.leader.replication.namespaces.status` | This will only be emitted by the leader in a secondary datacenter. The value will be a 1 if the last round of namespace replication was successful or 0 if there was an error. | healthy | gauge | +| `consul.leader.replication.namespaces.index` | This will only be emitted by the leader in a secondary datacenter. Increments to the index of namespaces in the primary datacenter that have been successfully replicated. | index | gauge | +| `consul.prepared-query.apply` | Measures the time it takes to apply a prepared query update. | ms | timer | +| `consul.prepared-query.explain` | Measures the time it takes to process a prepared query explain request. | ms | timer | +| `consul.prepared-query.execute` | Measures the time it takes to process a prepared query execute request. | ms | timer | +| `consul.prepared-query.execute_remote` | Measures the time it takes to process a prepared query execute request that was forwarded to another datacenter. | ms | timer | +| `consul.rpc.raft_handoff` | Increments when a server accepts a Raft-related RPC connection. | connections | counter | +| `consul.rpc.request_error` | Increments when a server returns an error from an RPC request. | errors | counter | +| `consul.rpc.request` | Increments when a server receives a Consul-related RPC request. | requests | counter | +| `consul.rpc.query` | Increments when a server receives a read RPC request, indicating the rate of new read queries. See consul.rpc.queries_blocking for the current number of in-flight blocking RPC calls. This metric changed in 1.7.0 to only increment on the the start of a query. The rate of queries will appear lower, but is more accurate. | queries | counter | +| `consul.rpc.queries_blocking` | The current number of in-flight blocking queries the server is handling. | queries | gauge | +| `consul.rpc.cross-dc` | Increments when a server sends a (potentially blocking) cross datacenter RPC query. | queries | counter | +| `consul.rpc.consistentRead` | Measures the time spent confirming that a consistent read can be performed. | ms | timer | +| `consul.session.apply` | Measures the time spent applying a session update. | ms | timer | +| `consul.session.renew` | Measures the time spent renewing a session. | ms | timer | +| `consul.session_ttl.invalidate` | Measures the time spent invalidating an expired session. | ms | timer | +| `consul.txn.apply` | Measures the time spent applying a transaction operation. | ms | timer | +| `consul.txn.read` | Measures the time spent returning a read transaction. | ms | timer | +| `consul.grpc.client.request.count` | Counts the number of gRPC requests made by the client agent to a Consul server. | requests | counter | +| `consul.grpc.client.connection.count` | Counts the number of new gRPC connections opened by the client agent to a Consul server. | connections | counter | +| `consul.grpc.client.connections` | Measures the number of active gRPC connections open from the client agent to any Consul servers. | connections | gauge | +| `consul.grpc.server.request.count` | Counts the number of gRPC requests received by the server. | requests | counter | +| `consul.grpc.server.connection.count` | Counts the number of new gRPC connections received by the server. | connections | counter | +| `consul.grpc.server.connections` | Measures the number of active gRPC connections open on the server. | connections | gauge | +| `consul.grpc.server.stream.count` | Counts the number of new gRPC streams received by the server. | streams | counter | +| `consul.grpc.server.streams` | Measures the number of active gRPC streams handled by the server. | streams | gauge | +| `consul.xds.server.streams` | Measures the number of active xDS streams handled by the server split by protocol version. | streams | gauge | ## Server Workload @@ -592,7 +592,7 @@ Here is a Prometheus style example of an RPC metric and its labels: -```json +``` ... consul_rpc_server_call{errored="false",method="Catalog.ListNodes",request_type="read",rpc_type="net/rpc",quantile="0.5"} 255 ... @@ -606,43 +606,44 @@ Any metric in this section can be turned off with the [`prefix_filter`](/docs/ag These metrics give insight into the health of the cluster as a whole. -| Metric | Description | Unit | Type | -| ------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------- | ------- | -| `consul.memberlist.degraded.probe` | Counts the number of times the agent has performed failure detection on another agent at a slower probe rate. The agent uses its own health metric as an indicator to perform this action. (If its health score is low, means that the node is healthy, and vice versa.) | probes / interval | counter | -| `consul.memberlist.degraded.timeout` | Counts the number of times an agent was marked as a dead node, whilst not getting enough confirmations from a randomly selected list of agent nodes in an agent's membership. | occurrence / interval | counter | -| `consul.memberlist.msg.dead` | Counts the number of times an agent has marked another agent to be a dead node. | messages / interval | counter | -| `consul.memberlist.health.score` | Describes a node's perception of its own health based on how well it is meeting the soft real-time requirements of the protocol. This metric ranges from 0 to 8, where 0 indicates "totally healthy". This health score is used to scale the time between outgoing probes, and higher scores translate into longer probing intervals. For more details see section IV of the Lifeguard paper: https://arxiv.org/pdf/1707.00788.pdf | score | gauge | -| `consul.memberlist.msg.suspect` | Increments when an agent suspects another as failed when executing random probes as part of the gossip protocol. These can be an indicator of overloaded agents, network problems, or configuration errors where agents can not connect to each other on the [required ports](/docs/agent/config/config-files#ports). | suspect messages received / interval | counter | -| `consul.memberlist.tcp.accept` | Counts the number of times an agent has accepted an incoming TCP stream connection. | connections accepted / interval | counter | -| `consul.memberlist.udp.sent/received` | Measures the total number of bytes sent/received by an agent through the UDP protocol. | bytes sent or bytes received / interval | counter | -| `consul.memberlist.tcp.connect` | Counts the number of times an agent has initiated a push/pull sync with an other agent. | push/pull initiated / interval | counter | -| `consul.memberlist.tcp.sent` | Measures the total number of bytes sent by an agent through the TCP protocol | bytes sent / interval | counter | -| `consul.memberlist.gossip` | Measures the time taken for gossip messages to be broadcasted to a set of randomly selected nodes. | ms | timer | -| `consul.memberlist.msg_alive` | Counts the number of alive messages, that the agent has processed so far, based on the message information given by the network layer. | messages / Interval | counter | -| `consul.memberlist.msg_dead` | The number of dead messages that the agent has processed so far, based on the message information given by the network layer. | messages / Interval | counter | -| `consul.memberlist.msg_suspect` | The number of suspect messages that the agent has processed so far, based on the message information given by the network layer. | messages / Interval | counter | -| `consul.memberlist.probeNode` | Measures the time taken to perform a single round of failure detection on a select agent. | nodes / Interval | counter | -| `consul.memberlist.pushPullNode` | Measures the number of agents that have exchanged state with this agent. | nodes / Interval | counter | -| `consul.serf.member.failed` | Increments when an agent is marked dead. This can be an indicator of overloaded agents, network problems, or configuration errors where agents cannot connect to each other on the [required ports](/docs/agent/config/config-files#ports). | failures / interval | counter | -| `consul.serf.member.flap` | Available in Consul 0.7 and later, this increments when an agent is marked dead and then recovers within a short time period. This can be an indicator of overloaded agents, network problems, or configuration errors where agents cannot connect to each other on the [required ports](/docs/agent/config/config-files#ports). | flaps / interval | counter | -| `consul.serf.member.join` | Increments when an agent joins the cluster. If an agent flapped or failed this counter also increments when it re-joins. | joins / interval | counter | -| `consul.serf.member.left` | Increments when an agent leaves the cluster. | leaves / interval | counter | -| `consul.serf.events` | Increments when an agent processes an [event](/commands/event). Consul uses events internally so there may be additional events showing in telemetry. There are also a per-event counters emitted as `consul.serf.events.`. | events / interval | counter | -| `consul.serf.msgs.sent` | This metric is sample of the number of bytes of messages broadcast to the cluster. In a given time interval, the sum of this metric is the total number of bytes sent and the count is the number of messages sent. | message bytes / interval | counter | -| `consul.autopilot.failure_tolerance` | Tracks the number of voting servers that the cluster can lose while continuing to function. | servers | gauge | -| `consul.autopilot.healthy` | Tracks the overall health of the local server cluster. If all servers are considered healthy by Autopilot, this will be set to 1. If any are unhealthy, this will be 0. | boolean | gauge | -| `consul.session_ttl.active` | Tracks the active number of sessions being tracked. | sessions | gauge | -| `consul.catalog.service.query.` | Increments for each catalog query for the given service. | queries | counter | -| `consul.catalog.service.query-tag..` | Increments for each catalog query for the given service with the given tag. | queries | counter | -| `consul.catalog.service.query-tags..` | Increments for each catalog query for the given service with the given tags. | queries | counter | -| `consul.catalog.service.not-found.` | Increments for each catalog query where the given service could not be found. | queries | counter | -| `consul.catalog.connect.query.` | Increments for each connect-based catalog query for the given service. | queries | counter | -| `consul.catalog.connect.query-tag..` | Increments for each connect-based catalog query for the given service with the given tag. | queries | counter | -| `consul.catalog.connect.query-tags..` | Increments for each connect-based catalog query for the given service with the given tags. | queries | counter | -| `consul.catalog.connect.not-found.` | Increments for each connect-based catalog query where the given service could not be found. | queries | counter | -| `consul.mesh.active-root-ca.expiry` | The number of seconds until the root CA expires, updated every hour. | seconds | gauge | -| `consul.mesh.active-signing-ca.expiry`| The number of seconds until the signing CA expires, updated every hour. | seconds | gauge | -| `consul.agent.tls.cert.expiry` | The number of seconds until the Agent TLS certificate expires, updated every hour. | seconds | gauge | +| Metric | Description | Unit | Type | +|----------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------|---------| +| `consul.memberlist.degraded.probe` | Counts the number of times the agent has performed failure detection on another agent at a slower probe rate. The agent uses its own health metric as an indicator to perform this action. (If its health score is low, means that the node is healthy, and vice versa.) | probes / interval | counter | +| `consul.memberlist.degraded.timeout` | Counts the number of times an agent was marked as a dead node, whilst not getting enough confirmations from a randomly selected list of agent nodes in an agent's membership. | occurrence / interval | counter | +| `consul.memberlist.msg.dead` | Counts the number of times an agent has marked another agent to be a dead node. | messages / interval | counter | +| `consul.memberlist.health.score` | Describes a node's perception of its own health based on how well it is meeting the soft real-time requirements of the protocol. This metric ranges from 0 to 8, where 0 indicates "totally healthy". This health score is used to scale the time between outgoing probes, and higher scores translate into longer probing intervals. For more details see section IV of the Lifeguard paper: https://arxiv.org/pdf/1707.00788.pdf | score | gauge | +| `consul.memberlist.msg.suspect` | Increments when an agent suspects another as failed when executing random probes as part of the gossip protocol. These can be an indicator of overloaded agents, network problems, or configuration errors where agents can not connect to each other on the [required ports](/docs/agent/config/config-files#ports). | suspect messages received / interval | counter | +| `consul.memberlist.tcp.accept` | Counts the number of times an agent has accepted an incoming TCP stream connection. | connections accepted / interval | counter | +| `consul.memberlist.udp.sent/received` | Measures the total number of bytes sent/received by an agent through the UDP protocol. | bytes sent or bytes received / interval | counter | +| `consul.memberlist.tcp.connect` | Counts the number of times an agent has initiated a push/pull sync with an other agent. | push/pull initiated / interval | counter | +| `consul.memberlist.tcp.sent` | Measures the total number of bytes sent by an agent through the TCP protocol | bytes sent / interval | counter | +| `consul.memberlist.gossip` | Measures the time taken for gossip messages to be broadcasted to a set of randomly selected nodes. | ms | timer | +| `consul.memberlist.msg_alive` | Counts the number of alive messages, that the agent has processed so far, based on the message information given by the network layer. | messages / Interval | counter | +| `consul.memberlist.msg_dead` | The number of dead messages that the agent has processed so far, based on the message information given by the network layer. | messages / Interval | counter | +| `consul.memberlist.msg_suspect` | The number of suspect messages that the agent has processed so far, based on the message information given by the network layer. | messages / Interval | counter | +| `consul.memberlist.probeNode` | Measures the time taken to perform a single round of failure detection on a select agent. | nodes / Interval | counter | +| `consul.memberlist.pushPullNode` | Measures the number of agents that have exchanged state with this agent. | nodes / Interval | counter | +| `consul.serf.member.failed` | Increments when an agent is marked dead. This can be an indicator of overloaded agents, network problems, or configuration errors where agents cannot connect to each other on the [required ports](/docs/agent/config/config-files#ports). | failures / interval | counter | +| `consul.serf.member.flap` | Available in Consul 0.7 and later, this increments when an agent is marked dead and then recovers within a short time period. This can be an indicator of overloaded agents, network problems, or configuration errors where agents cannot connect to each other on the [required ports](/docs/agent/config/config-files#ports). | flaps / interval | counter | +| `consul.serf.member.join` | Increments when an agent joins the cluster. If an agent flapped or failed this counter also increments when it re-joins. | joins / interval | counter | +| `consul.serf.member.left` | Increments when an agent leaves the cluster. | leaves / interval | counter | +| `consul.serf.events` | Increments when an agent processes an [event](/commands/event). Consul uses events internally so there may be additional events showing in telemetry. There are also a per-event counters emitted as `consul.serf.events.`. | events / interval | counter | +| `consul.serf.events.` | Breakdown of `consul.serf.events` by type of event. | events / interval | counter | +| `consul.serf.msgs.sent` | This metric is sample of the number of bytes of messages broadcast to the cluster. In a given time interval, the sum of this metric is the total number of bytes sent and the count is the number of messages sent. | message bytes / interval | counter | +| `consul.autopilot.failure_tolerance` | Tracks the number of voting servers that the cluster can lose while continuing to function. | servers | gauge | +| `consul.autopilot.healthy` | Tracks the overall health of the local server cluster. If all servers are considered healthy by Autopilot, this will be set to 1. If any are unhealthy, this will be 0. | boolean | gauge | +| `consul.session_ttl.active` | Tracks the active number of sessions being tracked. | sessions | gauge | +| `consul.catalog.service.query` | Increments for each catalog query for the given service. | queries | counter | +| `consul.catalog.service.query-tag` | Increments for each catalog query for the given service with the given tag. | queries | counter | +| `consul.catalog.service.query-tags` | Increments for each catalog query for the given service with the given tags. | queries | counter | +| `consul.catalog.service.not-found` | Increments for each catalog query where the given service could not be found. | queries | counter | +| `consul.catalog.connect.query` | Increments for each connect-based catalog query for the given service. | queries | counter | +| `consul.catalog.connect.query-tag` | Increments for each connect-based catalog query for the given service with the given tag. | queries | counter | +| `consul.catalog.connect.query-tags` | Increments for each connect-based catalog query for the given service with the given tags. | queries | counter | +| `consul.catalog.connect.not-found` | Increments for each connect-based catalog query where the given service could not be found. | queries | counter | +| `consul.mesh.active-root-ca.expiry` | The number of seconds until the root CA expires, updated every hour. | seconds | gauge | +| `consul.mesh.active-signing-ca.expiry` | The number of seconds until the signing CA expires, updated every hour. | seconds | gauge | +| `consul.agent.tls.cert.expiry` | The number of seconds until the Agent TLS certificate expires, updated every hour. | seconds | gauge | ## Connect Built-in Proxy Metrics From 77d79042e235ad47bb9b8cca84a0091a708fc43a Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 10:42:52 -0400 Subject: [PATCH 085/149] Update website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx --- .../content/docs/release-notes/consul-api-gateway/v0_3_x.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx index 1d177d008..e8ecb2dcd 100644 --- a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx +++ b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx @@ -11,7 +11,7 @@ description: >- - **Improved High Availability and Scalability via Multiple Instances per Gateway:** This release introduces support for deploying multiple instances of a logical - gateway (i.e. a K8s `gateway` config object). You can use a external load + gateway (i.e. a K8s `gateway` config object). You can use an external load balancing service to distribute incoming traffic across the instances of a gateway. This provides higher availability of gateway services and also allows a single logical gateway to handle a volume of network traffic that is much From 62cc76d7f8b60cbf2544113d2a1c77f94e9b2894 Mon Sep 17 00:00:00 2001 From: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Date: Tue, 21 Jun 2022 09:55:10 -0500 Subject: [PATCH 086/149] Update website/content/docs/api-gateway/upgrade-specific-versions.mdx Co-authored-by: Nathan Coleman --- website/content/docs/api-gateway/upgrade-specific-versions.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index b6a475888..71afa6679 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -35,7 +35,7 @@ Ensure that the following requirements are met prior to upgrading: $ kubectl get deployment --namespace consul consul-api-gateway-controller --output=jsonpath= "{@.spec.template.spec.containers[?(@.name=='api-gateway-controller')].image}" ``` - You should receive the following response: + You should receive a response similar to the following: ```log "hashicorp/consul-api-gateway:0.2.1" From dd8134f78e572c68d46f3586ccd7836dc00a1fba Mon Sep 17 00:00:00 2001 From: sarahalsmiller <100602640+sarahalsmiller@users.noreply.github.com> Date: Tue, 21 Jun 2022 10:05:38 -0500 Subject: [PATCH 087/149] Update upgrade-specific-versions.mdx --- website/content/docs/api-gateway/upgrade-specific-versions.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 71afa6679..9c900a912 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -117,7 +117,7 @@ Ensure that the following requirements are met prior to upgrading: 1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `Gateways` and filter for secrets that require a `ReferencePolicy`. ```shell-session - $ kubectl get Gateway -o json -A | jq -r '.items[] | {gateway_name: .metadata.name, gateway_namespace: .metadata.namespace, kind: .kind, crossNamespaceSecrets: ( .metadata.namespace as $parentnamespace | .spec.listeners[] .tls.certificateRefs[] | select(.namespace != null and .namespace != $parentnamespace ) )} ' + $ kubectl get Gateway -o json -A | jq -r '.items[] | {gateway_name: .metadata.name, gateway_namespace: .metadata.namespace, kind: .kind, crossNamespaceSecrets: ( .metadata.namespace as $parentnamespace | .spec.listeners[] | select(has("tls")) | .tls.certificateRefs[] | select(.namespace != null and .namespace != $parentnamespace ) )} ' ``` From c0a0227c964256d41008c9cee0b1535307f29a1b Mon Sep 17 00:00:00 2001 From: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> Date: Tue, 21 Jun 2022 08:08:37 -0700 Subject: [PATCH 088/149] Apply suggestions from code review Co-authored-by: Jeff Boruszak <104028618+boruszak@users.noreply.github.com> Co-authored-by: Chris Thain <32781396+cthain@users.noreply.github.com> --- .../content/docs/ecs/manual/secure-configuration.mdx | 4 ++-- website/content/docs/ecs/terraform/install.mdx | 12 +++++++----- 2 files changed, 9 insertions(+), 7 deletions(-) diff --git a/website/content/docs/ecs/manual/secure-configuration.mdx b/website/content/docs/ecs/manual/secure-configuration.mdx index 02a4bd4a8..f9a0ac934 100644 --- a/website/content/docs/ecs/manual/secure-configuration.mdx +++ b/website/content/docs/ecs/manual/secure-configuration.mdx @@ -32,7 +32,7 @@ You must create two types of ACL tokens for Consul on ECS: * **Client tokens:** used by the `consul-client` containers to join the Consul cluster * **Service tokens:** used by sidecar containers for service registration and health syncing -This section describes how to manually create ACL tokens. You can install the ACL controller, however, to ease the burden of creating tokens. The ACL controller can automatically create ACL tokens for Consul on ECS. Refer to the [ACL Controller](/docs/manual/acl-controller) documentation for installation details. +This section describes how to manually create ACL tokens. Alternatively, you can install the ACL controller to ease the burden of creating tokens. The ACL controller can automatically create ACL tokens for Consul on ECS. For additional details, refer to [ACL Controller](/docs/manual/acl-controller). ### Define policies @@ -80,7 +80,7 @@ agent_prefix "" { ```hcl partition "" { ## If partitions enabled namespace "" { ## If namespaces enabled - service "" { + service "" { policy = "write" } node_prefix "" { diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index d52aeeaf4..99099d21b 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -162,6 +162,7 @@ module "my_mesh_gateway" { ecs_cluster_arn = "" subnets = [""] retry_join = ["
"] + tls = true consul_server_ca_cert_arn = "" gossip_key_secret_arn = "" } @@ -180,6 +181,7 @@ The following fields are required. Refer to the [module reference documentation] | `ecs_cluster_arn` | string | Specifies the ARN of the ECS cluster where the mesh gateway task should be launched. | | `subnets` | list of strings | Specifies the subnet IDs where the task will be launched. | | `retry_join` | list of strings | Defines a set of arguments to pass to the Consul agent [`-retry-join`](/docs/agent/config/cli-flags#_retry_join) flag. The arguments specify locations of the Consul servers in the local datacenter that Consul client agents can connect to. | +| `tls` | boolean | Set to `true` to enable TLS. | | `consul_server_ca_cert_arn` | string | Specifies the ARN of the Secrets Manager containing the Consul server CA certificate | | `gossip_key_secret_arn` | string | Specifies the ARN of the Secrets Manager containing the Consul's gossip encryption key. | @@ -215,7 +217,7 @@ The following examples illustrate how to configure the `gateway-task` for differ Mesh gateways need to be reachable over the WAN to route traffic between datacenters. Configure the following options in the `gateway-task` to enable ingress through the mesh gateway. -| Option | Type | Description | +| Input variable | Type | Description | | --- | --- | --- | | `lb_enabled` | Boolean | Set to `true` to automatically deploy and configure a network load balancer for ingress to the mesh gateway. | | `lb_vpc_id` | string | Specifies the VPC to launch the load balancer in. | @@ -421,15 +423,15 @@ module "web" { ## Configure the bind address To ensure that your application only receives traffic through the service mesh, -you must change the address that your application is listening on to only the loopback address. The loopback address is also called `localhost`, `lo`, and `127.0.0.1`. +you must change the address that your application listens on to the loopback address. The loopback address is also called `localhost`, `lo`, and `127.0.0.1`. Binding to the loopback address allows the sidecar proxy running in the same task to only make requests within the service mesh. -If your application is listening on all interfaces, e.g., `0.0.0.0`, then other +If your application is listening on all interfaces, such as `0.0.0.0`, then other applications can call it directly, bypassing its sidecar proxy. Changing the listening address is specific to the language and framework you're -using in your application. Regardless of which language/framework you're using, -it is a good practice to make the address configurable via environment variable. +using in your application. Regardless of which language or framework you're using, +it is a good practice to use the environment variable to configure the address. The following examples demonstrate how to bind the loopback address in golang and Django (Python): From 3a35b181f2a24f96ed825e5bad99c60d49773784 Mon Sep 17 00:00:00 2001 From: David Yu Date: Tue, 21 Jun 2022 08:14:43 -0700 Subject: [PATCH 089/149] docs: Lambda consul service mesh naming suggestions (#13506) * docs: Lambda consul service mesh naming suggestions Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- website/content/docs/lambda/index.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/content/docs/lambda/index.mdx b/website/content/docs/lambda/index.mdx index f95da5496..d8540285e 100644 --- a/website/content/docs/lambda/index.mdx +++ b/website/content/docs/lambda/index.mdx @@ -26,8 +26,8 @@ also be manually registered into Consul when using Lambda registrator is not pos See the [Registration page](/docs/lambda/registration) for more information about registring Lambda functions into Consul. -### Invoking Lambda Functions +### Invoking Lambda Functions from Consul Service Mesh Lambda functions can be invoked by any mesh service directly from connect proxies or through terminating gateways. The [Invocation page](/docs/lambda/invocation) -explains how to invoke Lambda functions from Consul connect services. +explains how to invoke Lambda functions from Consul service mesh services. From 2c1806774f2bce5156a3f9090f90360093785d0c Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 11:37:19 -0400 Subject: [PATCH 090/149] Link reference to Kubernetes Gateway API site --- .../docs/release-notes/consul-api-gateway/v0_1_x.mdx | 2 +- .../docs/release-notes/consul-api-gateway/v0_2_x.mdx | 2 +- .../docs/release-notes/consul-api-gateway/v0_3_x.mdx | 6 +++--- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_1_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_1_x.mdx index 46379d965..357fd7032 100644 --- a/website/content/docs/release-notes/consul-api-gateway/v0_1_x.mdx +++ b/website/content/docs/release-notes/consul-api-gateway/v0_1_x.mdx @@ -72,6 +72,6 @@ This release includes the following features and capabilities: ## Kubernetes Gateway API Specification -Supported version of the Gateway API spec: `v1alpha2`(v0.4.1) +Supported version of the [Gateway API](https://gateway-api.sigs.k8s.io/) spec: `v1alpha2`(v0.4.1) For more detailed information, please refer to the [product documentation](/docs/api-gateway). diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_2_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_2_x.mdx index 3643b7262..41014f2b9 100644 --- a/website/content/docs/release-notes/consul-api-gateway/v0_2_x.mdx +++ b/website/content/docs/release-notes/consul-api-gateway/v0_2_x.mdx @@ -33,7 +33,7 @@ description: >- ## Kubernetes Gateway API Specification -Supported version of the Gateway API spec: `v1alpha2`(v0.4.1) +Supported version of the [Gateway API](https://gateway-api.sigs.k8s.io/) spec: `v1alpha2`(v0.4.1) ## Upgrading diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx index e8ecb2dcd..996ed008b 100644 --- a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx +++ b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx @@ -21,7 +21,7 @@ description: >- gateway. Administrators can also set, in the `gatewayClassConfig`, the minimum, maximum and default number of gateway instances. These are applied per `gateway` to all `gateways` in that `gatewayClass`. The minimum and - maximum settings will override settings configured on any given `gateway`. + maximum settings will override settings configured on any given `gateway`. Note: There is a hard limit of eight (8) instances per `gateway` and it can not be overridden. This hard limit will probably be increased in the future. @@ -29,7 +29,7 @@ description: >- - **Cross Namespace Reference Policies for Certificates:** Reference policies is a security feature introduced in the 0.2 release. In this release, Reference Policies are now required for TLS certificates when a - `listener` has a `certificateRef` that is in a different `namespace`. + `listener` has a `certificateRef` that is in a different `namespace`. ## Supported Software @@ -44,7 +44,7 @@ description: >- ## Kubernetes Gateway API Specification -Supported version of the Gateway API spec: `v1alpha2`(v0.4.1) +Supported version of the [Gateway API](https://gateway-api.sigs.k8s.io/) spec: `v1alpha2`(v0.4.1) ## Upgrading From 7eedeeecfc7f8596c4b79165f070843ee012b29a Mon Sep 17 00:00:00 2001 From: Dhia Ayachi Date: Tue, 21 Jun 2022 12:20:00 -0400 Subject: [PATCH 091/149] update github.com/containerd/containerd to 1.5.13 (#13520) --- test/integration/consul-container/go.mod | 6 +++--- test/integration/consul-container/go.sum | 15 ++++++++++++--- 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/test/integration/consul-container/go.mod b/test/integration/consul-container/go.mod index b5a7b1738..4dea73bba 100644 --- a/test/integration/consul-container/go.mod +++ b/test/integration/consul-container/go.mod @@ -14,11 +14,11 @@ require ( require ( github.com/Azure/go-ansiterm v0.0.0-20210617225240-d185dfc1b5a1 // indirect github.com/Microsoft/go-winio v0.4.17 // indirect - github.com/Microsoft/hcsshim v0.8.23 // indirect + github.com/Microsoft/hcsshim v0.8.24 // indirect github.com/armon/go-metrics v0.3.10 // indirect github.com/cenkalti/backoff/v4 v4.1.2 // indirect - github.com/containerd/cgroups v1.0.1 // indirect - github.com/containerd/containerd v1.5.9 // indirect + github.com/containerd/cgroups v1.0.3 // indirect + github.com/containerd/containerd v1.5.13 // indirect github.com/davecgh/go-spew v1.1.1 // indirect github.com/docker/distribution v2.7.1+incompatible // indirect github.com/docker/go-connections v0.4.0 // indirect diff --git a/test/integration/consul-container/go.sum b/test/integration/consul-container/go.sum index 7d67cf3ca..4bc58fd5c 100644 --- a/test/integration/consul-container/go.sum +++ b/test/integration/consul-container/go.sum @@ -55,8 +55,9 @@ github.com/Microsoft/hcsshim v0.8.9/go.mod h1:5692vkUqntj1idxauYlpoINNKeqCiG6Sg3 github.com/Microsoft/hcsshim v0.8.14/go.mod h1:NtVKoYxQuTLx6gEq0L96c9Ju4JbRJ4nY2ow3VK6a9Lg= github.com/Microsoft/hcsshim v0.8.15/go.mod h1:x38A4YbHbdxJtc0sF6oIz+RG0npwSCAvn69iY6URG00= github.com/Microsoft/hcsshim v0.8.16/go.mod h1:o5/SZqmR7x9JNKsW3pu+nqHm0MF8vbA+VxGOoXdC600= -github.com/Microsoft/hcsshim v0.8.23 h1:47MSwtKGXet80aIn+7h4YI6fwPmwIghAnsx2aOUrG2M= github.com/Microsoft/hcsshim v0.8.23/go.mod h1:4zegtUJth7lAvFyc6cH2gGQ5B3OFQim01nnU2M8jKDg= +github.com/Microsoft/hcsshim v0.8.24 h1:jP+GMeRXIR1sH1kG4lJr9ShmSjVrua5jmFZDtfYGkn4= +github.com/Microsoft/hcsshim v0.8.24/go.mod h1:4zegtUJth7lAvFyc6cH2gGQ5B3OFQim01nnU2M8jKDg= github.com/Microsoft/hcsshim/test v0.0.0-20201218223536-d3e5debf77da/go.mod h1:5hlzMzRKMLyo42nCZ9oml8AdTlq/0cvIaBv6tK1RehU= github.com/Microsoft/hcsshim/test v0.0.0-20210227013316-43a75bb4edd3/go.mod h1:mw7qgWloBUl75W/gVH3cQszUg1+gUITj7D6NY7ywVnY= github.com/NYTimes/gziphandler v0.0.0-20170623195520-56545f4a5d46/go.mod h1:3wb06e3pkSAbeQ52E9H9iFoQsEEwGN64994WTCIhntQ= @@ -130,8 +131,9 @@ github.com/containerd/cgroups v0.0.0-20200531161412-0dbf7f05ba59/go.mod h1:pA0z1 github.com/containerd/cgroups v0.0.0-20200710171044-318312a37340/go.mod h1:s5q4SojHctfxANBDvMeIaIovkq29IP48TKAxnhYRxvo= github.com/containerd/cgroups v0.0.0-20200824123100-0b889c03f102/go.mod h1:s5q4SojHctfxANBDvMeIaIovkq29IP48TKAxnhYRxvo= github.com/containerd/cgroups v0.0.0-20210114181951-8a68de567b68/go.mod h1:ZJeTFisyysqgcCdecO57Dj79RfL0LNeGiFUqLYQRYLE= -github.com/containerd/cgroups v1.0.1 h1:iJnMvco9XGvKUvNQkv88bE4uJXxRQH18efbKo9w5vHQ= github.com/containerd/cgroups v1.0.1/go.mod h1:0SJrPIenamHDcZhEcJMNBB85rHcUsw4f25ZfBiPYRkU= +github.com/containerd/cgroups v1.0.3 h1:ADZftAkglvCiD44c77s5YmMqaP2pzVCFZvBmAlBdAP4= +github.com/containerd/cgroups v1.0.3/go.mod h1:/ofk34relqNjSGyqPrmEULrO4Sc8LJhvJmWbUCUKqj8= github.com/containerd/console v0.0.0-20180822173158-c12b1e7919c1/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw= github.com/containerd/console v0.0.0-20181022165439-0650fd9eeb50/go.mod h1:Tj/on1eG8kiEhd0+fhSDzsPAFESxzBBvdyEgyryXffw= github.com/containerd/console v0.0.0-20191206165004-02ecf6a7291e/go.mod h1:8Pf4gM6VEbTNRIT26AyyU7hxdQU3MvAvxVI0sc00XBE= @@ -150,8 +152,9 @@ github.com/containerd/containerd v1.5.0-beta.1/go.mod h1:5HfvG1V2FsKesEGQ17k5/T7 github.com/containerd/containerd v1.5.0-beta.3/go.mod h1:/wr9AVtEM7x9c+n0+stptlo/uBBoBORwEx6ardVcmKU= github.com/containerd/containerd v1.5.0-beta.4/go.mod h1:GmdgZd2zA2GYIBZ0w09ZvgqEq8EfBp/m3lcVZIvPHhI= github.com/containerd/containerd v1.5.0-rc.0/go.mod h1:V/IXoMqNGgBlabz3tHD2TWDoTJseu1FGOKuoA4nNb2s= -github.com/containerd/containerd v1.5.9 h1:rs6Xg1gtIxaeyG+Smsb/0xaSDu1VgFhOCKBXxMxbsF4= github.com/containerd/containerd v1.5.9/go.mod h1:fvQqCfadDGga5HZyn3j4+dx56qj2I9YwBrlSdalvJYQ= +github.com/containerd/containerd v1.5.13 h1:XqvKw9i4P7/mFrC3TSM7yV5cwFZ9avXe6M3YANKnzEE= +github.com/containerd/containerd v1.5.13/go.mod h1:3AlCrzKROjIuP3JALsY14n8YtntaUDBu7vek+rPN5Vc= github.com/containerd/continuity v0.0.0-20190426062206-aaeac12a7ffc/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y= github.com/containerd/continuity v0.0.0-20190815185530-f2a389ac0a02/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y= github.com/containerd/continuity v0.0.0-20191127005431-f65d91d395eb/go.mod h1:GL3xCUCBDV3CZiTSEKksMWbLE66hEyuu9qyDOOqM47Y= @@ -699,6 +702,7 @@ github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q= github.com/yuin/goldmark v1.1.27/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74= +github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k= github.com/yvasiyarov/go-metrics v0.0.0-20140926110328-57bccd1ccd43/go.mod h1:aX5oPXxHm3bOH+xeAttToC8pqch2ScQN/JoXYupl6xs= github.com/yvasiyarov/gorelic v0.0.0-20141212073537-a9bba5b9ab50/go.mod h1:NUSPSUX/bi6SeDMUh6brw0nXpxHnc96TguQh0+r/ssA= github.com/yvasiyarov/newrelic_platform_go v0.0.0-20140908184405-b21fdbd4370f/go.mod h1:GlGEuHIJweS1mbCqG+7vt2nvWLzLLnRHbXz5JKd/Qbg= @@ -715,6 +719,7 @@ go.opencensus.io v0.22.3/go.mod h1:yxeiOL68Rb0Xd1ddK5vPZ/oVn4vY4Ynel7k9FzqtOIw= go.opentelemetry.io/proto/otlp v0.7.0/go.mod h1:PqfVotwruBrMGOCsRd/89rSnXhoiJIqeYNgFYFoEGnI= go.uber.org/atomic v1.3.2/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE= +go.uber.org/goleak v1.1.12/go.mod h1:cwTWslyiVhfpKIDGSZEM2HlOvcqm+tG4zioyIeLoqMQ= go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0= go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q= golang.org/x/crypto v0.0.0-20171113213409-9f005a07e0d3/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4= @@ -761,6 +766,7 @@ golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzB golang.org/x/mod v0.1.1-0.20191107180719-034126e5016b/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg= golang.org/x/mod v0.2.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= +golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA= golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4= @@ -797,6 +803,7 @@ golang.org/x/net v0.0.0-20201021035429-f5854403a974/go.mod h1:sp8m0HH+o8qH0wwXwY golang.org/x/net v0.0.0-20201110031124-69a78807bb2b/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU= golang.org/x/net v0.0.0-20201224014010-6772e930b67b/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= golang.org/x/net v0.0.0-20210226172049-e18ecbb05110/go.mod h1:m0MpNAwzfU5UDzcl9v0D8zg8gWTRqZa9RBIspLL5mdg= +golang.org/x/net v0.0.0-20210405180319-a5a99cb37ef4/go.mod h1:p54w0d4576C0XHj96bSt6lcn1PtDYWL6XObtHCRCNQM= golang.org/x/net v0.0.0-20210410081132-afb366fc7cd1/go.mod h1:9tjilg8BloeKEkVJvy7fQ90B1CfIiPueXVOjqfkSzI8= golang.org/x/net v0.0.0-20211108170745-6635138e15ea/go.mod h1:9nx3DQGgdP8bBQD5qxJ1jj9UTztislL4KSBs9R2vV5Y= golang.org/x/net v0.0.0-20211216030914-fe4d6282115f h1:hEYJvxw1lSnWIl8X9ofsYMklzaDs90JI2az5YMd4fPM= @@ -886,6 +893,7 @@ golang.org/x/sys v0.0.0-20210324051608-47abb6519492/go.mod h1:h1NjWce9XRLGQEsW7w golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210423082822-04245dca01da/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= golang.org/x/sys v0.0.0-20210426230700-d19ff857e887/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs= +golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210615035016-665e8c7367d1/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20211025201205-69cdffdb9359/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= @@ -946,6 +954,7 @@ golang.org/x/tools v0.0.0-20200304193943-95d2e580d8eb/go.mod h1:o4KQGtdN14AW+yjs golang.org/x/tools v0.0.0-20200619180055-7c47624df98f/go.mod h1:EkVYQZoAsY45+roYkvgYkIh4xh/qjgUK9TdY2XT94GE= golang.org/x/tools v0.0.0-20210106214847-113979e3529a/go.mod h1:emZCQorbCU4vsT4fOWvOPXz4eW1wZW4PmDk9uLelYpA= golang.org/x/tools v0.1.0/go.mod h1:xkSsbof2nBLbhDlRMhhhyNLN/zl3eTqcnHD5viDpcZ0= +golang.org/x/tools v0.1.5/go.mod h1:o0xws9oXOQQZyjljx8fwUC0k7L1pTE6eaCbjGeHmOkk= golang.org/x/xerrors v0.0.0-20190717185122-a985d3407aa7/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= From b360ccc77dc24ae9ec1a5a40fc771c35bb7e0a72 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 12:20:24 -0400 Subject: [PATCH 092/149] Update website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx Co-authored-by: Mike Morris --- .../content/docs/release-notes/consul-api-gateway/v0_3_x.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx index 996ed008b..7e4654b6b 100644 --- a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx +++ b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx @@ -50,7 +50,7 @@ Supported version of the [Gateway API](https://gateway-api.sigs.k8s.io/) spec: ` For detailed information on upgrading, please refer to the [upgrade details page](/docs/api-gateway/upgrade-specific-versions) -## Change logs +## Changelog The changelogs for this major release version and any maintenance versions are listed below. From 8dc94aff32720159b0d9c3875a303f39fba08700 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Tue, 21 Jun 2022 09:27:04 -0700 Subject: [PATCH 093/149] removed terminating and ingress polices from secure manual installation --- .../docs/ecs/manual/secure-configuration.mdx | 45 ++++--------------- 1 file changed, 8 insertions(+), 37 deletions(-) diff --git a/website/content/docs/ecs/manual/secure-configuration.mdx b/website/content/docs/ecs/manual/secure-configuration.mdx index f9a0ac934..c9a856043 100644 --- a/website/content/docs/ecs/manual/secure-configuration.mdx +++ b/website/content/docs/ecs/manual/secure-configuration.mdx @@ -38,6 +38,8 @@ This section describes how to manually create ACL tokens. Alternatively, you can Configure the following ACL policy for the Consul client token: + + ```hcl node_prefix "" { policy = "write" @@ -47,19 +49,19 @@ service_prefix "" { } ``` + + The policy allows `node:write` for any node name, which is necessary because the Consul node names on ECS are not known until runtime. You can add the policy in Consul using the [`consul acl policy create`](/commands/acl/policy/create) command or the [`[PUT] /v1/acl/policy`](/api-docs/acl/policies#create-a-policy) API endpoint. -If you intend to create a gateway for connecting multiple Consul datacenters, you will need additional policies to specify the permission scope. +If you intend to create a gateway for connecting multiple Consul datacenters, you will need to configure a mesh gateway policy. If namespaces are enabled, the mesh gateway must run in the default namespace. -#### Mesh gateway policy - -Mesh gateways must run in the default namespace. + ```hcl namespace "default" { ## If namespaces enabled - service "" { + service "" { policy = "write" } } @@ -75,39 +77,8 @@ agent_prefix "" { policy = "read" } ``` -#### Terminating gateway policy - -```hcl -partition "" { ## If partitions enabled - namespace "" { ## If namespaces enabled - service "" { - policy = "write" - } - node_prefix "" { - policy = "read" - } - } -} -``` - -#### Ingress gateway policy - -```hcl -partition "" { ## If partitions enabled - namespace "" { ## If namespaces enabled - service "" { - policy = "write" - } - node_prefix "" { - policy = "read" - } - service_prefix "" { - policy = "read" - } - } -} -``` + ### Create service tokens From 6210fffb6a31404883bbaecfaaa2c0b080b287e5 Mon Sep 17 00:00:00 2001 From: Nitya Dhanushkodi Date: Tue, 21 Jun 2022 09:31:38 -0700 Subject: [PATCH 094/149] update from review comments --- website/content/docs/k8s/helm.mdx | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/website/content/docs/k8s/helm.mdx b/website/content/docs/k8s/helm.mdx index 9d3037ea3..4b411a3b5 100644 --- a/website/content/docs/k8s/helm.mdx +++ b/website/content/docs/k8s/helm.mdx @@ -63,9 +63,8 @@ Use these links to navigate to a particular top-level stanza. - `peering` ((#v-global-peering)) - [Experimental] Configures the Cluster Peering feature. Requires Consul v1.13+ and Consul-K8s v0.45+. - - `enabled` ((#v-global-peering-enabled)) (`boolean: false`) - If true, the Helm chart will enable Cluster Peering for the cluster. This will enable peering controllers and - allow use of the PeeringAcceptor and PeeringDialer CRDs to establish peerings for service mesh. - @type boolean + - `enabled` ((#v-global-peering-enabled)) (`boolean: false`) - If true, the Helm chart enables Cluster Peering for the cluster. This option enables peering controllers and + allows use of the PeeringAcceptor and PeeringDialer CRDs for establishing service mesh peerings. - `adminPartitions` ((#v-global-adminpartitions)) - Enabling `adminPartitions` allows creation of Admin Partitions in Kubernetes clusters. It additionally indicates that you are running Consul Enterprise v1.11+ with a valid Consul Enterprise From 6423e067789abce3b065a9bd2ad0c30d89f73a50 Mon Sep 17 00:00:00 2001 From: Jeff Apple <79924108+Jeff-Apple@users.noreply.github.com> Date: Tue, 21 Jun 2022 10:07:14 -0700 Subject: [PATCH 095/149] make "changelog" plural in the API GW 0.3 release notes. --- .../content/docs/release-notes/consul-api-gateway/v0_3_x.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx index 7e4654b6b..598808616 100644 --- a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx +++ b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx @@ -50,7 +50,7 @@ Supported version of the [Gateway API](https://gateway-api.sigs.k8s.io/) spec: ` For detailed information on upgrading, please refer to the [upgrade details page](/docs/api-gateway/upgrade-specific-versions) -## Changelog +## Changelogs The changelogs for this major release version and any maintenance versions are listed below. From c2421825f3479f180a3d2ed8b6fc39af13546b0c Mon Sep 17 00:00:00 2001 From: Matt Keeler Date: Tue, 21 Jun 2022 13:36:49 -0400 Subject: [PATCH 096/149] Add server local blocking queries and watches (#13438) Co-authored-by: Dan Upton --- agent/consul/watch/mock_StateStore_test.go | 40 ++ agent/consul/watch/server_local.go | 332 ++++++++++++++++ agent/consul/watch/server_local_test.go | 424 +++++++++++++++++++++ 3 files changed, 796 insertions(+) create mode 100644 agent/consul/watch/mock_StateStore_test.go create mode 100644 agent/consul/watch/server_local.go create mode 100644 agent/consul/watch/server_local_test.go diff --git a/agent/consul/watch/mock_StateStore_test.go b/agent/consul/watch/mock_StateStore_test.go new file mode 100644 index 000000000..08d58e2f0 --- /dev/null +++ b/agent/consul/watch/mock_StateStore_test.go @@ -0,0 +1,40 @@ +// Code generated by mockery v2.12.2. DO NOT EDIT. + +package watch + +import ( + testing "testing" + + mock "github.com/stretchr/testify/mock" +) + +// MockStateStore is an autogenerated mock type for the StateStore type +type MockStateStore struct { + mock.Mock +} + +// AbandonCh provides a mock function with given fields: +func (_m *MockStateStore) AbandonCh() <-chan struct{} { + ret := _m.Called() + + var r0 <-chan struct{} + if rf, ok := ret.Get(0).(func() <-chan struct{}); ok { + r0 = rf() + } else { + if ret.Get(0) != nil { + r0 = ret.Get(0).(<-chan struct{}) + } + } + + return r0 +} + +// NewMockStateStore creates a new instance of MockStateStore. It also registers the testing.TB interface on the mock and a cleanup function to assert the mocks expectations. +func NewMockStateStore(t testing.TB) *MockStateStore { + mock := &MockStateStore{} + mock.Mock.Test(t) + + t.Cleanup(func() { mock.AssertExpectations(t) }) + + return mock +} diff --git a/agent/consul/watch/server_local.go b/agent/consul/watch/server_local.go new file mode 100644 index 000000000..8085396fe --- /dev/null +++ b/agent/consul/watch/server_local.go @@ -0,0 +1,332 @@ +package watch + +import ( + "context" + "errors" + "fmt" + "time" + + "github.com/hashicorp/consul/lib/retry" + "github.com/hashicorp/go-memdb" + hashstructure_v2 "github.com/mitchellh/hashstructure/v2" +) + +var ( + ErrorNotFound = errors.New("no data found for query") + ErrorNotChanged = errors.New("data did not change for query") + + errNilContext = errors.New("cannot call ServerLocalNotify with a nil context") + errNilGetStore = errors.New("cannot call ServerLocalNotify without a callback to get a StateStore") + errNilQuery = errors.New("cannot call ServerLocalNotify without a callback to perform the query") + errNilNotify = errors.New("cannot call ServerLocalNotify without a callback to send notifications") +) + +//go:generate mockery --name StateStore --inpackage --testonly +type StateStore interface { + AbandonCh() <-chan struct{} +} + +const ( + defaultWaiterMinFailures uint = 1 + defaultWaiterMinWait = time.Second + defaultWaiterMaxWait = 60 * time.Second + defaultWaiterFactor = 2 * time.Second +) + +var ( + defaultWaiterJitter = retry.NewJitter(100) +) + +func defaultWaiter() *retry.Waiter { + return &retry.Waiter{ + MinFailures: defaultWaiterMinFailures, + MinWait: defaultWaiterMinWait, + MaxWait: defaultWaiterMaxWait, + Jitter: defaultWaiterJitter, + Factor: defaultWaiterFactor, + } +} + +// noopDone can be passed to serverLocalNotifyWithWaiter +func noopDone() {} + +// ServerLocalBlockingQuery performs a blocking query similar to the pre-existing blockingQuery +// method on the agent/consul.Server type. There are a few key differences. +// +// 1. This function makes use of Go 1.18 generics. The function is parameterized with two +// types. The first is the ResultType which can be anything. Having this be parameterized +// instead of using interface{} allows us to simplify the call sites so that no type +// coercion from interface{} to the real type is necessary. The second parameterized type +// is something that VERY loosely resembles a agent/consul/state.Store type. The StateStore +// interface in this package has a single method to get the stores abandon channel so we +// know when a snapshot restore is occurring and can act accordingly. We could have not +// parameterized this type and used a real *state.Store instead but then we would have +// concrete dependencies on the state package and it would make it a little harder to +// test this function. +// +// We could have also avoided the need to use a ResultType parameter by taking the route +// the original blockingQuery method did and to just assume all callers close around +// a pointer to their results and can modify it as necessary. That way of doing things +// feels a little gross so I have taken this one a different direction. The old way +// also gets especially gross with how we have to push concerns of spurious wakeup +// suppression down into every call site. +// +// 2. This method has no internal timeout and can potentially run forever until a state +// change is observed. If there is a desire to have a timeout, that should be built into +// the context.Context passed as the first argument. +// +// 3. This method bakes in some newer functionality around hashing of results to prevent sending +// back data when nothing has actually changed. With the old blockingQuery method this has to +// be done within the closure passed to the method which means the same bit of code is duplicated +// in many places. As this functionality isn't necessary in many scenarios whether to opt-in to +// that behavior is a argument to this function. +// +// Similar to the older method: +// +// 1. Errors returned from the query will be propagated back to the caller. +// +// The query function must follow these rules: +// +// 1. To access data it must use the passed in StoreType (which will be a state.Store when +// everything gets stiched together outside of unit tests). +// 2. It must return an index greater than the minIndex if the results returned by the query +// have changed. +// 3. Any channels added to the memdb.WatchSet must unblock when the results +// returned by the query have changed. +// +// To ensure optimal performance of the query, the query function should make a +// best-effort attempt to follow these guidelines: +// +// 1. Only return an index greater than the minIndex. +// 2. Any channels added to the memdb.WatchSet should only unblock when the +// results returned by the query have changed. This might be difficult +// to do when blocking on non-existent data. +// +func ServerLocalBlockingQuery[ResultType any, StoreType StateStore]( + ctx context.Context, + getStore func() StoreType, + minIndex uint64, + suppressSpuriousWakeup bool, + query func(memdb.WatchSet, StoreType) (uint64, ResultType, error), +) (uint64, ResultType, error) { + var ( + notFound bool + ranOnce bool + priorHash uint64 + ) + + var zeroResult ResultType + if getStore == nil { + return 0, zeroResult, fmt.Errorf("no getStore function was provided to ServerLocalBlockingQuery") + } + if query == nil { + return 0, zeroResult, fmt.Errorf("no query function was provided to ServerLocalBlockingQuery") + } + + for { + state := getStore() + + ws := memdb.NewWatchSet() + + // Adding the AbandonCh to the WatchSet allows us to detect when + // a snapshot restore happens that would otherwise not modify anything + // within the individual state store. If we didn't do this then we + // could end up blocking indefinitely. + ws.Add(state.AbandonCh()) + + index, result, err := query(ws, state) + + switch { + case errors.Is(err, ErrorNotFound): + // if minIndex is 0 then we should never block but we + // also should not propagate the error + if minIndex == 0 { + return index, result, nil + } + + // update the min index if the previous result was not found. This + // is an attempt to not return data unnecessarily when we end up + // watching the root of a memdb Radix tree because the data being + // watched doesn't exist yet. + if notFound { + minIndex = index + } + + notFound = true + case err != nil: + return index, result, err + } + + // when enabled we can prevent sending back data that hasn't changed. + if suppressSpuriousWakeup { + newHash, err := hashstructure_v2.Hash(result, hashstructure_v2.FormatV2, nil) + if err != nil { + return index, result, fmt.Errorf("error hashing data for spurious wakeup suppression: %w", err) + } + + // set minIndex to the returned index to prevent sending back identical data + if ranOnce && priorHash == newHash { + minIndex = index + } + ranOnce = true + priorHash = newHash + } + + // one final check if we should be considered unblocked and + // return the value. Some conditions in the switch above + // alter the minIndex and prevent this return if it would + // be desirable. One such case is when the actual data has + // not changed since the last round through the query and + // we would rather not do any further processing for unchanged + // data. This mostly protects against watches for data that + // doesn't exist from return the non-existant value constantly. + if index > minIndex { + return index, result, nil + } + + // Block until something changes. Because we have added the state + // stores AbandonCh to this watch set, a snapshot restore will + // cause things to unblock in addition to changes to the actual + // queried data. + if err := ws.WatchCtx(ctx); err != nil { + // exit if the context was cancelled + return index, result, nil + } + + select { + case <-state.AbandonCh(): + return index, result, nil + default: + } + } +} + +// ServerLocalNotify will watch for changes in the State Store using the provided +// query function and invoke the notify callback whenever the results of that query +// function have changed. This function will return an error if parameter validations +// fail but otherwise the background go routine to process the notifications will +// be spawned and nil will be returned. Just like ServerLocalBlockingQuery this makes +// use of Go Generics and for the same reasons as outlined in the documentation for +// that function. +func ServerLocalNotify[ResultType any, StoreType StateStore]( + ctx context.Context, + correlationID string, + getStore func() StoreType, + query func(memdb.WatchSet, StoreType) (uint64, ResultType, error), + notify func(ctx context.Context, correlationID string, result ResultType, err error), +) error { + return serverLocalNotify( + ctx, + correlationID, + getStore, + query, + notify, + // Public callers should not need to know when the internal go routines are finished. + // Being able to provide a done function to the internal version of this function is + // to allow our tests to be more determinstic and to eliminate arbitrary sleeps. + noopDone, + // Public callers do not get to override the error backoff configuration. Internally + // we want to allow for this to enable our unit tests to run much more quickly. + defaultWaiter(), + ) +} + +// serverLocalNotify is the internal version of ServerLocalNotify. It takes +// two additional arguments of the waiter to use and a function to call +// when the notification go routine has finished +func serverLocalNotify[ResultType any, StoreType StateStore]( + ctx context.Context, + correlationID string, + getStore func() StoreType, + query func(memdb.WatchSet, StoreType) (uint64, ResultType, error), + notify func(ctx context.Context, correlationID string, result ResultType, err error), + done func(), + waiter *retry.Waiter, +) error { + if ctx == nil { + return errNilContext + } + + if getStore == nil { + return errNilGetStore + } + + if query == nil { + return errNilQuery + } + + if notify == nil { + return errNilNotify + } + + go serverLocalNotifyRoutine( + ctx, + correlationID, + getStore, + query, + notify, + done, + waiter, + ) + return nil +} + +// serverLocalNotifyRoutine is the function intended to be run within a new +// go routine to process the updates. It will not check to ensure callbacks +// are non-nil nor perform other parameter validation. It is assumed that +// the in-package caller of this method will have already done that. It also +// takes the backoff waiter in as an argument so that unit tests within this +// package can override the default values that the exported ServerLocalNotify +// function would have set up. +func serverLocalNotifyRoutine[ResultType any, StoreType StateStore]( + ctx context.Context, + correlationID string, + getStore func() StoreType, + query func(memdb.WatchSet, StoreType) (uint64, ResultType, error), + notify func(ctx context.Context, correlationID string, result ResultType, err error), + done func(), + waiter *retry.Waiter, +) { + defer done() + + var minIndex uint64 + + for { + // Check if the context has been cancelled. Do not issue + // more queries if it has been cancelled. + if ctx.Err() != nil { + return + } + + // Perform the blocking query + index, result, err := ServerLocalBlockingQuery(ctx, getStore, minIndex, true, query) + + // Check if the context has been cancelled. If it has we should not send more + // notifications. + if ctx.Err() != nil { + return + } + + // Check the index to see if we should call notify + if minIndex == 0 || minIndex < index { + notify(ctx, correlationID, result, err) + minIndex = index + } + + // Handle errors with backoff. Badly behaved blocking calls that returned + // a zero index are considered as failures since we need to not get stuck + // in a busy loop. + if err == nil && index > 0 { + waiter.Reset() + } else { + if waiter.Wait(ctx) != nil { + return + } + } + + // ensure we don't use zero indexes + if err == nil && minIndex < 1 { + minIndex = 1 + } + } +} diff --git a/agent/consul/watch/server_local_test.go b/agent/consul/watch/server_local_test.go new file mode 100644 index 000000000..6fa440979 --- /dev/null +++ b/agent/consul/watch/server_local_test.go @@ -0,0 +1,424 @@ +package watch + +import ( + "context" + "fmt" + "testing" + "time" + + "github.com/hashicorp/consul/lib/retry" + "github.com/hashicorp/go-memdb" + mock "github.com/stretchr/testify/mock" + "github.com/stretchr/testify/require" +) + +type mockStoreProvider struct { + mock.Mock +} + +func newMockStoreProvider(t *testing.T) *mockStoreProvider { + t.Helper() + provider := &mockStoreProvider{} + t.Cleanup(func() { + provider.AssertExpectations(t) + }) + return provider +} + +func (m *mockStoreProvider) getStore() *MockStateStore { + return m.Called().Get(0).(*MockStateStore) +} + +type testResult struct { + value string +} + +func (m *mockStoreProvider) query(ws memdb.WatchSet, store *MockStateStore) (uint64, *testResult, error) { + ret := m.Called(ws, store) + + index := ret.Get(0).(uint64) + result := ret.Get(1).(*testResult) + err := ret.Error(2) + + return index, result, err +} + +func (m *mockStoreProvider) notify(ctx context.Context, correlationID string, result *testResult, err error) { + m.Called(ctx, correlationID, result, err) +} + +func TestServerLocalBlockingQuery_getStoreNotProvided(t *testing.T) { + _, _, err := ServerLocalBlockingQuery( + context.Background(), + nil, + 0, + true, + func(memdb.WatchSet, *MockStateStore) (uint64, struct{}, error) { + return 0, struct{}{}, nil + }, + ) + + require.Error(t, err) + require.Contains(t, err.Error(), "no getStore function was provided") +} + +func TestServerLocalBlockingQuery_queryNotProvided(t *testing.T) { + var query func(memdb.WatchSet, *MockStateStore) (uint64, struct{}, error) + _, _, err := ServerLocalBlockingQuery( + context.Background(), + func() *MockStateStore { return nil }, + 0, + true, + query, + ) + + require.Error(t, err) + require.Contains(t, err.Error(), "no query function was provided") +} + +func TestServerLocalBlockingQuery_NonBlocking(t *testing.T) { + abandonCh := make(chan struct{}) + t.Cleanup(func() { close(abandonCh) }) + + store := NewMockStateStore(t) + store.On("AbandonCh"). + Return(closeChan(abandonCh)). + Once() + + provider := newMockStoreProvider(t) + provider.On("getStore").Return(store).Once() + provider.On("query", mock.Anything, store). + Return(uint64(1), &testResult{value: "foo"}, nil). + Once() + + idx, result, err := ServerLocalBlockingQuery( + context.Background(), + provider.getStore, + 0, + true, + provider.query, + ) + require.NoError(t, err) + require.EqualValues(t, 1, idx) + require.Equal(t, &testResult{value: "foo"}, result) +} + +func TestServerLocalBlockingQuery_NotFound(t *testing.T) { + abandonCh := make(chan struct{}) + t.Cleanup(func() { close(abandonCh) }) + + store := NewMockStateStore(t) + store.On("AbandonCh"). + Return(closeChan(abandonCh)). + Once() + + provider := newMockStoreProvider(t) + provider.On("getStore"). + Return(store). + Once() + + var nilResult *testResult + provider.On("query", mock.Anything, store). + Return(uint64(1), nilResult, ErrorNotFound). + Once() + + idx, result, err := ServerLocalBlockingQuery( + context.Background(), + provider.getStore, + 0, + true, + provider.query, + ) + require.NoError(t, err) + require.EqualValues(t, 1, idx) + require.Nil(t, result) +} + +func TestServerLocalBlockingQuery_NotFoundBlocks(t *testing.T) { + abandonCh := make(chan struct{}) + t.Cleanup(func() { close(abandonCh) }) + + store := NewMockStateStore(t) + store.On("AbandonCh"). + Return(closeChan(abandonCh)). + Times(5) + + provider := newMockStoreProvider(t) + provider.On("getStore"). + Return(store). + Times(3) + + var nilResult *testResult + // Initial data returned is not found and has an index less than the original + // blocking index. This should not return data to the caller. + provider.On("query", mock.Anything, store). + Return(uint64(4), nilResult, ErrorNotFound). + Run(addReadyWatchSet). + Once() + // There is an update to the data but the value still doesn't exist. Therefore + // we should not return data to the caller. + provider.On("query", mock.Anything, store). + Return(uint64(6), nilResult, ErrorNotFound). + Run(addReadyWatchSet). + Once() + // Finally we have some real data and can return it to the caller. + provider.On("query", mock.Anything, store). + Return(uint64(7), &testResult{value: "foo"}, nil). + Once() + + idx, result, err := ServerLocalBlockingQuery( + context.Background(), + provider.getStore, + 5, + true, + provider.query, + ) + require.NoError(t, err) + require.EqualValues(t, 7, idx) + require.Equal(t, &testResult{value: "foo"}, result) +} + +func TestServerLocalBlockingQuery_Error(t *testing.T) { + abandonCh := make(chan struct{}) + t.Cleanup(func() { close(abandonCh) }) + + store := NewMockStateStore(t) + store.On("AbandonCh"). + Return(closeChan(abandonCh)). + Once() + + provider := newMockStoreProvider(t) + provider.On("getStore"). + Return(store). + Once() + + var nilResult *testResult + provider.On("query", mock.Anything, store). + Return(uint64(10), nilResult, fmt.Errorf("synthetic error")). + Once() + + idx, result, err := ServerLocalBlockingQuery( + context.Background(), + provider.getStore, + 4, + true, + provider.query, + ) + require.Error(t, err) + require.Contains(t, err.Error(), "synthetic error") + require.EqualValues(t, 10, idx) + require.Nil(t, result) +} + +func TestServerLocalBlockingQuery_ContextCancellation(t *testing.T) { + abandonCh := make(chan struct{}) + t.Cleanup(func() { close(abandonCh) }) + + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + + store := NewMockStateStore(t) + store.On("AbandonCh"). + Return(closeChan(abandonCh)). + Once() + + provider := newMockStoreProvider(t) + provider.On("getStore"). + Return(store). + Once() + provider.On("query", mock.Anything, store). + // Return an index that should not cause the blocking query to return. + Return(uint64(4), &testResult{value: "foo"}, nil). + Once(). + Run(func(_ mock.Arguments) { + // Cancel the context so that the memdb WatchCtx call will error. + cancel() + }) + + idx, result, err := ServerLocalBlockingQuery( + ctx, + provider.getStore, + 8, + true, + provider.query, + ) + // The internal cancellation error should not be propagated. + require.NoError(t, err) + require.EqualValues(t, 4, idx) + require.Equal(t, &testResult{value: "foo"}, result) +} + +func TestServerLocalBlockingQuery_StateAbandoned(t *testing.T) { + abandonCh := make(chan struct{}) + + store := NewMockStateStore(t) + store.On("AbandonCh"). + Return(closeChan(abandonCh)). + Twice() + + provider := newMockStoreProvider(t) + provider.On("getStore"). + Return(store). + Once() + provider.On("query", mock.Anything, store). + // Return an index that should not cause the blocking query to return. + Return(uint64(4), &testResult{value: "foo"}, nil). + Once(). + Run(func(_ mock.Arguments) { + // Cancel the context so that the memdb WatchCtx call will error. + close(abandonCh) + }) + + idx, result, err := ServerLocalBlockingQuery( + context.Background(), + provider.getStore, + 8, + true, + provider.query, + ) + // The internal cancellation error should not be propagated. + require.NoError(t, err) + require.EqualValues(t, 4, idx) + require.Equal(t, &testResult{value: "foo"}, result) +} + +func TestServerLocalNotify_Validations(t *testing.T) { + provider := newMockStoreProvider(t) + + type testCase struct { + ctx context.Context + getStore func() *MockStateStore + query func(memdb.WatchSet, *MockStateStore) (uint64, *testResult, error) + notify func(context.Context, string, *testResult, error) + err error + } + + cases := map[string]testCase{ + "nil-context": { + getStore: provider.getStore, + query: provider.query, + notify: provider.notify, + err: errNilContext, + }, + "nil-getStore": { + ctx: context.Background(), + query: provider.query, + notify: provider.notify, + err: errNilGetStore, + }, + "nil-query": { + ctx: context.Background(), + getStore: provider.getStore, + notify: provider.notify, + err: errNilQuery, + }, + "nil-notify": { + ctx: context.Background(), + getStore: provider.getStore, + query: provider.query, + err: errNilNotify, + }, + } + + for name, tcase := range cases { + t.Run(name, func(t *testing.T) { + err := ServerLocalNotify(tcase.ctx, "test", tcase.getStore, tcase.query, tcase.notify) + require.ErrorIs(t, err, tcase.err) + }) + } +} + +func TestServerLocalNotify(t *testing.T) { + notifyCtx, notifyCancel := context.WithCancel(context.Background()) + t.Cleanup(notifyCancel) + + abandonCh := make(chan struct{}) + + store := NewMockStateStore(t) + store.On("AbandonCh"). + Return(closeChan(abandonCh)). + Times(3) + + provider := newMockStoreProvider(t) + provider.On("getStore"). + Return(store). + Times(3) + provider.On("query", mock.Anything, store). + Return(uint64(4), &testResult{value: "foo"}, nil). + Once() + provider.On("notify", notifyCtx, t.Name(), &testResult{value: "foo"}, nil).Once() + provider.On("query", mock.Anything, store). + Return(uint64(6), &testResult{value: "bar"}, nil). + Once() + provider.On("notify", notifyCtx, t.Name(), &testResult{value: "bar"}, nil).Once() + provider.On("query", mock.Anything, store). + Return(uint64(7), &testResult{value: "baz"}, context.Canceled). + Run(func(mock.Arguments) { + notifyCancel() + }) + + doneCtx, routineDone := context.WithCancel(context.Background()) + err := serverLocalNotify(notifyCtx, t.Name(), provider.getStore, provider.query, provider.notify, routineDone, defaultWaiter()) + require.NoError(t, err) + + // Wait for the context cancellation which will happen when the "query" func is run the third time. The doneCtx gets "cancelled" + // by the backgrounded go routine when it is actually finished. We need to wait for this to ensure that all mocked calls have been + // made and that no extra calls get made. + <-doneCtx.Done() +} + +func TestServerLocalNotify_internal(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + t.Cleanup(cancel) + + abandonCh := make(chan struct{}) + + store := NewMockStateStore(t) + store.On("AbandonCh"). + Return(closeChan(abandonCh)). + Times(4) + + var nilResult *testResult + + provider := newMockStoreProvider(t) + provider.On("getStore"). + Return(store). + Times(4) + provider.On("query", mock.Anything, store). + Return(uint64(0), nilResult, fmt.Errorf("injected error")). + Times(3) + provider.On("notify", ctx, "test", nilResult, fmt.Errorf("injected error")). + Times(3) + provider.On("query", mock.Anything, store). + Return(uint64(7), &testResult{value: "foo"}, nil). + Once() + provider.On("notify", ctx, "test", &testResult{value: "foo"}, nil). + Once(). + Run(func(mock.Arguments) { + cancel() + }) + waiter := retry.Waiter{ + MinFailures: 1, + MinWait: time.Millisecond, + MaxWait: 50 * time.Millisecond, + Jitter: retry.NewJitter(100), + Factor: 2 * time.Millisecond, + } + + // all the mock expectations should ensure things are working properly + serverLocalNotifyRoutine(ctx, "test", provider.getStore, provider.query, provider.notify, noopDone, &waiter) +} + +func addReadyWatchSet(args mock.Arguments) { + ws := args.Get(0).(memdb.WatchSet) + ch := make(chan struct{}) + ws.Add(ch) + close(ch) +} + +// small convenience to make this more readable. The alternative in a few +// cases would be to do something like (<-chan struct{})(ch). I find that +// syntax very difficult to read. +func closeChan(ch chan struct{}) <-chan struct{} { + return ch +} From e7a7232a6b193a647e4f7e459334af5b288e78f2 Mon Sep 17 00:00:00 2001 From: "R.B. Boyer" <4903+rboyer@users.noreply.github.com> Date: Tue, 21 Jun 2022 13:04:08 -0500 Subject: [PATCH 097/149] state: peering ID assignment cannot happen inside of the state store (#13525) Move peering ID assignment outisde of the FSM, so that the ID is written to the raft log and the same ID is used by all voters, and after restarts. --- agent/consul/fsm/snapshot_oss_test.go | 1 + agent/consul/leader_peering_test.go | 20 ++- agent/consul/peering_backend.go | 11 ++ agent/consul/state/peering.go | 60 ++++----- agent/consul/state/peering_test.go | 115 ++++++++---------- agent/rpc/peering/service.go | 56 +++++++++ agent/rpc/peering/service_test.go | 16 +++ agent/rpc/peering/stream_test.go | 13 +- .../rpc/peering/subscription_manager_test.go | 1 + 9 files changed, 189 insertions(+), 104 deletions(-) diff --git a/agent/consul/fsm/snapshot_oss_test.go b/agent/consul/fsm/snapshot_oss_test.go index 558abf4be..29678d1d0 100644 --- a/agent/consul/fsm/snapshot_oss_test.go +++ b/agent/consul/fsm/snapshot_oss_test.go @@ -476,6 +476,7 @@ func TestFSM_SnapshotRestore_OSS(t *testing.T) { // Peerings require.NoError(t, fsm.state.PeeringWrite(31, &pbpeering.Peering{ + ID: "1fabcd52-1d46-49b0-b1d8-71559aee47f5", Name: "baz", })) diff --git a/agent/consul/leader_peering_test.go b/agent/consul/leader_peering_test.go index 3e2f6c8ff..169ca833f 100644 --- a/agent/consul/leader_peering_test.go +++ b/agent/consul/leader_peering_test.go @@ -7,13 +7,13 @@ import ( "testing" "time" - "github.com/hashicorp/consul/acl" - "github.com/hashicorp/consul/api" "github.com/stretchr/testify/require" "google.golang.org/grpc" + "github.com/hashicorp/consul/acl" "github.com/hashicorp/consul/agent/consul/state" "github.com/hashicorp/consul/agent/structs" + "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/proto/pbpeering" "github.com/hashicorp/consul/sdk/testutil/retry" "github.com/hashicorp/consul/testrpc" @@ -62,6 +62,10 @@ func TestLeader_PeeringSync_Lifecycle_ClientDeletion(t *testing.T) { _, found := s1.peeringService.StreamStatus(token.PeerID) require.False(t, found) + var ( + s2PeerID = "cc56f0b8-3885-4e78-8d7b-614a0c45712d" + ) + // Bring up s2 and store s1's token so that it attempts to dial. _, s2 := testServerWithConfig(t, func(c *Config) { c.NodeName = "s2.dc2" @@ -73,6 +77,7 @@ func TestLeader_PeeringSync_Lifecycle_ClientDeletion(t *testing.T) { // Simulate a peering initiation event by writing a peering with data from a peering token. // Eventually the leader in dc2 should dial and connect to the leader in dc1. p := &pbpeering.Peering{ + ID: s2PeerID, Name: "my-peer-s1", PeerID: token.PeerID, PeerCAPems: token.CA, @@ -92,6 +97,7 @@ func TestLeader_PeeringSync_Lifecycle_ClientDeletion(t *testing.T) { // Delete the peering to trigger the termination sequence. deleted := &pbpeering.Peering{ + ID: s2PeerID, Name: "my-peer-s1", DeletedAt: structs.TimeToProto(time.Now()), } @@ -151,6 +157,11 @@ func TestLeader_PeeringSync_Lifecycle_ServerDeletion(t *testing.T) { var token structs.PeeringToken require.NoError(t, json.Unmarshal(tokenJSON, &token)) + var ( + s1PeerID = token.PeerID + s2PeerID = "cc56f0b8-3885-4e78-8d7b-614a0c45712d" + ) + // Bring up s2 and store s1's token so that it attempts to dial. _, s2 := testServerWithConfig(t, func(c *Config) { c.NodeName = "s2.dc2" @@ -162,6 +173,7 @@ func TestLeader_PeeringSync_Lifecycle_ServerDeletion(t *testing.T) { // Simulate a peering initiation event by writing a peering with data from a peering token. // Eventually the leader in dc2 should dial and connect to the leader in dc1. p := &pbpeering.Peering{ + ID: s2PeerID, Name: "my-peer-s1", PeerID: token.PeerID, PeerCAPems: token.CA, @@ -181,6 +193,7 @@ func TestLeader_PeeringSync_Lifecycle_ServerDeletion(t *testing.T) { // Delete the peering from the server peer to trigger the termination sequence. deleted := &pbpeering.Peering{ + ID: s1PeerID, Name: "my-peer-s2", DeletedAt: structs.TimeToProto(time.Now()), } @@ -216,6 +229,7 @@ func TestLeader_Peering_DeferredDeletion(t *testing.T) { testrpc.WaitForLeader(t, s1.RPC, "dc1") var ( + peerID = "cc56f0b8-3885-4e78-8d7b-614a0c45712d" peerName = "my-peer-s2" defaultMeta = acl.DefaultEnterpriseMeta() lastIdx = uint64(0) @@ -224,6 +238,7 @@ func TestLeader_Peering_DeferredDeletion(t *testing.T) { // Simulate a peering initiation event by writing a peering to the state store. lastIdx++ require.NoError(t, s1.fsm.State().PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: peerID, Name: peerName, })) @@ -233,6 +248,7 @@ func TestLeader_Peering_DeferredDeletion(t *testing.T) { // Mark the peering for deletion to trigger the termination sequence. lastIdx++ require.NoError(t, s1.fsm.State().PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: peerID, Name: peerName, DeletedAt: structs.TimeToProto(time.Now()), })) diff --git a/agent/consul/peering_backend.go b/agent/consul/peering_backend.go index 0ba3463c8..047569f11 100644 --- a/agent/consul/peering_backend.go +++ b/agent/consul/peering_backend.go @@ -143,6 +143,17 @@ type peeringApply struct { srv *Server } +func (a *peeringApply) CheckPeeringUUID(id string) (bool, error) { + state := a.srv.fsm.State() + if _, existing, err := state.PeeringReadByID(nil, id); err != nil { + return false, err + } else if existing != nil { + return false, nil + } + + return true, nil +} + func (a *peeringApply) PeeringWrite(req *pbpeering.PeeringWriteRequest) error { _, err := a.srv.raftApplyProtobuf(structs.PeeringWriteType, req) return err diff --git a/agent/consul/state/peering.go b/agent/consul/state/peering.go index 3d115707d..6515055b2 100644 --- a/agent/consul/state/peering.go +++ b/agent/consul/state/peering.go @@ -1,12 +1,12 @@ package state import ( + "errors" "fmt" "strings" "github.com/golang/protobuf/proto" "github.com/hashicorp/go-memdb" - "github.com/hashicorp/go-uuid" "github.com/hashicorp/consul/acl" "github.com/hashicorp/consul/agent/structs" @@ -191,50 +191,47 @@ func (s *Store) peeringListTxn(ws memdb.WatchSet, tx ReadTxn, entMeta acl.Enterp return idx, result, nil } -func generatePeeringUUID(tx ReadTxn) (string, error) { - for { - uuid, err := uuid.GenerateUUID() - if err != nil { - return "", fmt.Errorf("failed to generate UUID: %w", err) - } - existing, err := peeringReadByIDTxn(tx, nil, uuid) - if err != nil { - return "", fmt.Errorf("failed to read peering: %w", err) - } - if existing == nil { - return uuid, nil - } - } -} - func (s *Store) PeeringWrite(idx uint64, p *pbpeering.Peering) error { tx := s.db.WriteTxn(idx) defer tx.Abort() - q := Query{ - Value: p.Name, - EnterpriseMeta: *structs.NodeEnterpriseMetaInPartition(p.Partition), + // Check that the ID and Name are set. + if p.ID == "" { + return errors.New("Missing Peering ID") } - existingRaw, err := tx.First(tablePeering, indexName, q) - if err != nil { - return fmt.Errorf("failed peering lookup: %w", err) + if p.Name == "" { + return errors.New("Missing Peering Name") } - existing, ok := existingRaw.(*pbpeering.Peering) - if existingRaw != nil && !ok { - return fmt.Errorf("invalid type %T", existingRaw) + // ensure the name is unique (cannot conflict with another peering with a different ID) + _, existing, err := peeringReadTxn(tx, nil, Query{ + Value: p.Name, + EnterpriseMeta: *structs.NodeEnterpriseMetaInPartition(p.Partition), + }) + if err != nil { + return err } if existing != nil { + if p.ID != existing.ID { + return fmt.Errorf("A peering already exists with the name %q and a different ID %q", p.Name, existing.ID) + } // Prevent modifications to Peering marked for deletion if !existing.IsActive() { return fmt.Errorf("cannot write to peering that is marked for deletion") } p.CreateIndex = existing.CreateIndex - p.ID = existing.ID - + p.ModifyIndex = idx } else { + idMatch, err := peeringReadByIDTxn(tx, nil, p.ID) + if err != nil { + return err + } + if idMatch != nil { + return fmt.Errorf("A peering already exists with the ID %q and a different name %q", p.Name, existing.ID) + } + if !p.IsActive() { return fmt.Errorf("cannot create a new peering marked for deletion") } @@ -242,13 +239,8 @@ func (s *Store) PeeringWrite(idx uint64, p *pbpeering.Peering) error { // TODO(peering): consider keeping PeeringState enum elsewhere? p.State = pbpeering.PeeringState_INITIAL p.CreateIndex = idx - - p.ID, err = generatePeeringUUID(tx) - if err != nil { - return fmt.Errorf("failed to generate peering id: %w", err) - } + p.ModifyIndex = idx } - p.ModifyIndex = idx if err := tx.Insert(tablePeering, p); err != nil { return fmt.Errorf("failed inserting peering: %w", err) diff --git a/agent/consul/state/peering_test.go b/agent/consul/state/peering_test.go index 4aba5c340..04389a8e9 100644 --- a/agent/consul/state/peering_test.go +++ b/agent/consul/state/peering_test.go @@ -1,13 +1,10 @@ package state import ( - "fmt" - "math/rand" "testing" "time" "github.com/hashicorp/go-memdb" - "github.com/hashicorp/go-uuid" "github.com/stretchr/testify/require" "github.com/hashicorp/consul/acl" @@ -17,6 +14,12 @@ import ( "github.com/hashicorp/consul/sdk/testutil" ) +const ( + testFooPeerID = "9e650110-ac74-4c5a-a6a8-9348b2bed4e9" + testBarPeerID = "5ebcff30-5509-4858-8142-a8e580f1863f" + testBazPeerID = "432feb2f-5476-4ae2-b33c-e43640ca0e86" +) + func insertTestPeerings(t *testing.T, s *Store) { t.Helper() @@ -26,7 +29,7 @@ func insertTestPeerings(t *testing.T, s *Store) { err := tx.Insert(tablePeering, &pbpeering.Peering{ Name: "foo", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), - ID: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + ID: testFooPeerID, State: pbpeering.PeeringState_INITIAL, CreateIndex: 1, ModifyIndex: 1, @@ -36,7 +39,7 @@ func insertTestPeerings(t *testing.T, s *Store) { err = tx.Insert(tablePeering, &pbpeering.Peering{ Name: "bar", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), - ID: "5ebcff30-5509-4858-8142-a8e580f1863f", + ID: testBarPeerID, State: pbpeering.PeeringState_FAILING, CreateIndex: 2, ModifyIndex: 2, @@ -97,16 +100,16 @@ func TestStateStore_PeeringReadByID(t *testing.T) { run := func(t *testing.T, tc testcase) { _, peering, err := s.PeeringReadByID(nil, tc.id) require.NoError(t, err) - require.Equal(t, tc.expect, peering) + prototest.AssertDeepEqual(t, tc.expect, peering) } tcs := []testcase{ { name: "get foo", - id: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + id: testFooPeerID, expect: &pbpeering.Peering{ Name: "foo", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), - ID: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + ID: testFooPeerID, State: pbpeering.PeeringState_INITIAL, CreateIndex: 1, ModifyIndex: 1, @@ -114,11 +117,11 @@ func TestStateStore_PeeringReadByID(t *testing.T) { }, { name: "get bar", - id: "5ebcff30-5509-4858-8142-a8e580f1863f", + id: testBarPeerID, expect: &pbpeering.Peering{ Name: "bar", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), - ID: "5ebcff30-5509-4858-8142-a8e580f1863f", + ID: testBarPeerID, State: pbpeering.PeeringState_FAILING, CreateIndex: 2, ModifyIndex: 2, @@ -149,7 +152,7 @@ func TestStateStore_PeeringRead(t *testing.T) { run := func(t *testing.T, tc testcase) { _, peering, err := s.PeeringRead(nil, tc.query) require.NoError(t, err) - require.Equal(t, tc.expect, peering) + prototest.AssertDeepEqual(t, tc.expect, peering) } tcs := []testcase{ { @@ -160,7 +163,7 @@ func TestStateStore_PeeringRead(t *testing.T) { expect: &pbpeering.Peering{ Name: "foo", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), - ID: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + ID: testFooPeerID, State: pbpeering.PeeringState_INITIAL, CreateIndex: 1, ModifyIndex: 1, @@ -189,6 +192,7 @@ func TestStore_Peering_Watch(t *testing.T) { // set up initial write err := s.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testFooPeerID, Name: "foo", }) require.NoError(t, err) @@ -210,6 +214,7 @@ func TestStore_Peering_Watch(t *testing.T) { lastIdx++ err := s.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testBarPeerID, Name: "bar", }) require.NoError(t, err) @@ -229,6 +234,7 @@ func TestStore_Peering_Watch(t *testing.T) { // unrelated write shouldn't fire watch lastIdx++ err := s.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testBarPeerID, Name: "bar", }) require.NoError(t, err) @@ -237,6 +243,7 @@ func TestStore_Peering_Watch(t *testing.T) { // foo write should fire watch lastIdx++ err = s.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testFooPeerID, Name: "foo", DeletedAt: structs.TimeToProto(time.Now()), }) @@ -261,6 +268,7 @@ func TestStore_Peering_Watch(t *testing.T) { // mark for deletion before actually deleting lastIdx++ err := s.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testBarPeerID, Name: "bar", DeletedAt: structs.TimeToProto(time.Now()), }) @@ -293,7 +301,7 @@ func TestStore_PeeringList(t *testing.T) { { Name: "foo", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), - ID: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + ID: testFooPeerID, State: pbpeering.PeeringState_INITIAL, CreateIndex: 1, ModifyIndex: 1, @@ -301,7 +309,7 @@ func TestStore_PeeringList(t *testing.T) { { Name: "bar", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), - ID: "5ebcff30-5509-4858-8142-a8e580f1863f", + ID: testBarPeerID, State: pbpeering.PeeringState_FAILING, CreateIndex: 2, ModifyIndex: 2, @@ -336,6 +344,7 @@ func TestStore_PeeringList_Watch(t *testing.T) { lastIdx++ // insert a peering err := s.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testFooPeerID, Name: "foo", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), }) @@ -357,6 +366,7 @@ func TestStore_PeeringList_Watch(t *testing.T) { // update peering lastIdx++ require.NoError(t, s.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testFooPeerID, Name: "foo", DeletedAt: structs.TimeToProto(time.Now()), Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), @@ -422,6 +432,7 @@ func TestStore_PeeringWrite(t *testing.T) { { name: "create baz", input: &pbpeering.Peering{ + ID: testBazPeerID, Name: "baz", Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), }, @@ -429,6 +440,7 @@ func TestStore_PeeringWrite(t *testing.T) { { name: "update baz", input: &pbpeering.Peering{ + ID: testBazPeerID, Name: "baz", State: pbpeering.PeeringState_FAILING, Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), @@ -437,6 +449,7 @@ func TestStore_PeeringWrite(t *testing.T) { { name: "mark baz for deletion", input: &pbpeering.Peering{ + ID: testBazPeerID, Name: "baz", State: pbpeering.PeeringState_TERMINATED, DeletedAt: structs.TimeToProto(time.Now()), @@ -446,6 +459,7 @@ func TestStore_PeeringWrite(t *testing.T) { { name: "cannot update peering marked for deletion", input: &pbpeering.Peering{ + ID: testBazPeerID, Name: "baz", // Attempt to add metadata Meta: map[string]string{ @@ -458,6 +472,7 @@ func TestStore_PeeringWrite(t *testing.T) { { name: "cannot create peering marked for deletion", input: &pbpeering.Peering{ + ID: testFooPeerID, Name: "foo", DeletedAt: structs.TimeToProto(time.Now()), Partition: structs.NodeEnterpriseMetaInDefaultPartition().PartitionOrEmpty(), @@ -472,54 +487,6 @@ func TestStore_PeeringWrite(t *testing.T) { } } -func TestStore_PeeringWrite_GenerateUUID(t *testing.T) { - rand.Seed(1) - - s := NewStateStore(nil) - - entMeta := structs.NodeEnterpriseMetaInDefaultPartition() - partition := entMeta.PartitionOrDefault() - - for i := 1; i < 11; i++ { - require.NoError(t, s.PeeringWrite(uint64(i), &pbpeering.Peering{ - Name: fmt.Sprintf("peering-%d", i), - Partition: partition, - })) - } - - idx, peerings, err := s.PeeringList(nil, *entMeta) - require.NoError(t, err) - require.Equal(t, uint64(10), idx) - require.Len(t, peerings, 10) - - // Ensure that all assigned UUIDs are unique. - uniq := make(map[string]struct{}) - for _, p := range peerings { - uniq[p.ID] = struct{}{} - } - require.Len(t, uniq, 10) - - // Ensure that the ID of an existing peering cannot be overwritten. - updated := &pbpeering.Peering{ - Name: peerings[0].Name, - Partition: peerings[0].Partition, - } - - // Attempt to overwrite ID. - updated.ID, err = uuid.GenerateUUID() - require.NoError(t, err) - require.NoError(t, s.PeeringWrite(11, updated)) - - q := Query{ - Value: updated.Name, - EnterpriseMeta: *entMeta, - } - idx, got, err := s.PeeringRead(nil, q) - require.NoError(t, err) - require.Equal(t, uint64(11), idx) - require.Equal(t, peerings[0].ID, got.ID) -} - func TestStore_PeeringDelete(t *testing.T) { s := NewStateStore(nil) insertTestPeerings(t, s) @@ -532,6 +499,7 @@ func TestStore_PeeringDelete(t *testing.T) { testutil.RunStep(t, "can delete after marking for deletion", func(t *testing.T) { require.NoError(t, s.PeeringWrite(11, &pbpeering.Peering{ + ID: testFooPeerID, Name: "foo", DeletedAt: structs.TimeToProto(time.Now()), })) @@ -550,7 +518,7 @@ func TestStore_PeeringTerminateByID(t *testing.T) { insertTestPeerings(t, s) // id corresponding to default/foo - id := "9e650110-ac74-4c5a-a6a8-9348b2bed4e9" + const id = testFooPeerID require.NoError(t, s.PeeringTerminateByID(10, id)) @@ -607,7 +575,7 @@ func TestStateStore_PeeringTrustBundleRead(t *testing.T) { run := func(t *testing.T, tc testcase) { _, ptb, err := s.PeeringTrustBundleRead(nil, tc.query) require.NoError(t, err) - require.Equal(t, tc.expect, ptb) + prototest.AssertDeepEqual(t, tc.expect, ptb) } entMeta := structs.NodeEnterpriseMetaInDefaultPartition() @@ -708,6 +676,7 @@ func TestStateStore_ExportedServicesForPeer(t *testing.T) { lastIdx++ require.NoError(t, s.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testUUID(), Name: "my-peering", })) @@ -1000,6 +969,9 @@ func TestStateStore_PeeringsForService(t *testing.T) { var lastIdx uint64 // Create peerings for _, tp := range tc.peerings { + if tp.peering.ID == "" { + tp.peering.ID = testUUID() + } lastIdx++ require.NoError(t, s.PeeringWrite(lastIdx, tp.peering)) @@ -1009,6 +981,7 @@ func TestStateStore_PeeringsForService(t *testing.T) { lastIdx++ copied := pbpeering.Peering{ + ID: tp.peering.ID, Name: tp.peering.Name, DeletedAt: structs.TimeToProto(time.Now()), } @@ -1247,6 +1220,11 @@ func TestStore_TrustBundleListByService(t *testing.T) { var lastIdx uint64 ws := memdb.NewWatchSet() + var ( + peerID1 = testUUID() + peerID2 = testUUID() + ) + testutil.RunStep(t, "no results on initial setup", func(t *testing.T) { idx, resp, err := store.TrustBundleListByService(ws, "foo", entMeta) require.NoError(t, err) @@ -1279,6 +1257,7 @@ func TestStore_TrustBundleListByService(t *testing.T) { testutil.RunStep(t, "creating peering does not yield trust bundles", func(t *testing.T) { lastIdx++ require.NoError(t, store.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: peerID1, Name: "peer1", })) @@ -1377,6 +1356,7 @@ func TestStore_TrustBundleListByService(t *testing.T) { testutil.RunStep(t, "bundles for other peers are ignored", func(t *testing.T) { lastIdx++ require.NoError(t, store.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: peerID2, Name: "peer2", })) @@ -1431,6 +1411,7 @@ func TestStore_TrustBundleListByService(t *testing.T) { testutil.RunStep(t, "deleting the peering excludes its trust bundle", func(t *testing.T) { lastIdx++ require.NoError(t, store.PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: peerID1, Name: "peer1", DeletedAt: structs.TimeToProto(time.Now()), })) @@ -1470,7 +1451,7 @@ func TestStateStore_Peering_ListDeleted(t *testing.T) { err := tx.Insert(tablePeering, &pbpeering.Peering{ Name: "foo", Partition: acl.DefaultPartitionName, - ID: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + ID: testFooPeerID, DeletedAt: structs.TimeToProto(time.Now()), CreateIndex: 1, ModifyIndex: 1, @@ -1480,7 +1461,7 @@ func TestStateStore_Peering_ListDeleted(t *testing.T) { err = tx.Insert(tablePeering, &pbpeering.Peering{ Name: "bar", Partition: acl.DefaultPartitionName, - ID: "5ebcff30-5509-4858-8142-a8e580f1863f", + ID: testBarPeerID, CreateIndex: 2, ModifyIndex: 2, }) @@ -1489,7 +1470,7 @@ func TestStateStore_Peering_ListDeleted(t *testing.T) { err = tx.Insert(tablePeering, &pbpeering.Peering{ Name: "baz", Partition: acl.DefaultPartitionName, - ID: "432feb2f-5476-4ae2-b33c-e43640ca0e86", + ID: testBazPeerID, DeletedAt: structs.TimeToProto(time.Now()), CreateIndex: 3, ModifyIndex: 3, diff --git a/agent/rpc/peering/service.go b/agent/rpc/peering/service.go index 6d8de85d3..5638702aa 100644 --- a/agent/rpc/peering/service.go +++ b/agent/rpc/peering/service.go @@ -24,6 +24,7 @@ import ( "github.com/hashicorp/consul/agent/consul/stream" "github.com/hashicorp/consul/agent/dns" "github.com/hashicorp/consul/agent/structs" + "github.com/hashicorp/consul/lib" "github.com/hashicorp/consul/proto/pbpeering" ) @@ -140,6 +141,7 @@ type Store interface { // Apply provides a write-only interface for persisting Peering data. type Apply interface { + CheckPeeringUUID(id string) (bool, error) PeeringWrite(req *pbpeering.PeeringWriteRequest) error PeeringTerminateByID(req *pbpeering.PeeringTerminateByIDRequest) error PeeringTrustBundleWrite(req *pbpeering.PeeringTrustBundleWriteRequest) error @@ -189,8 +191,16 @@ func (s *Service) GenerateToken( return nil, err } + canRetry := true +RETRY_ONCE: + id, err := s.getExistingOrCreateNewPeerID(req.PeerName, req.Partition) + if err != nil { + return nil, err + } + writeReq := pbpeering.PeeringWriteRequest{ Peering: &pbpeering.Peering{ + ID: id, Name: req.PeerName, // TODO(peering): Normalize from ACL token once this endpoint is guarded by ACLs. Partition: req.PartitionOrDefault(), @@ -198,6 +208,15 @@ func (s *Service) GenerateToken( }, } if err := s.Backend.Apply().PeeringWrite(&writeReq); err != nil { + // There's a possible race where two servers call Generate Token at the + // same time with the same peer name for the first time. They both + // generate an ID and try to insert and only one wins. This detects the + // collision and forces the loser to discard its generated ID and use + // the one from the other server. + if canRetry && strings.Contains(err.Error(), "A peering already exists with the name") { + canRetry = false + goto RETRY_ONCE + } return nil, fmt.Errorf("failed to write peering: %w", err) } @@ -270,6 +289,11 @@ func (s *Service) Establish( serverAddrs[i] = addr } + id, err := s.getExistingOrCreateNewPeerID(req.PeerName, req.Partition) + if err != nil { + return nil, err + } + // as soon as a peering is written with a list of ServerAddresses that is // non-empty, the leader routine will see the peering and attempt to // establish a connection with the remote peer. @@ -278,6 +302,7 @@ func (s *Service) Establish( // RemotePeerID(PeerID) but at this point the other peer does not. writeReq := &pbpeering.PeeringWriteRequest{ Peering: &pbpeering.Peering{ + ID: id, Name: req.PeerName, PeerCAPems: tok.CA, PeerServerAddresses: serverAddrs, @@ -368,6 +393,16 @@ func (s *Service) PeeringWrite(ctx context.Context, req *pbpeering.PeeringWriteR defer metrics.MeasureSince([]string{"peering", "write"}, time.Now()) // TODO(peering): ACL check request token + if req.Peering == nil { + return nil, fmt.Errorf("missing required peering body") + } + + id, err := s.getExistingOrCreateNewPeerID(req.Peering.Name, req.Peering.Partition) + if err != nil { + return nil, err + } + req.Peering.ID = id + // TODO(peering): handle blocking queries err = s.Backend.Apply().PeeringWrite(req) if err != nil { @@ -418,6 +453,7 @@ func (s *Service) PeeringDelete(ctx context.Context, req *pbpeering.PeeringDelet // We only need to include the name and partition for the peering to be identified. // All other data associated with the peering can be discarded because once marked // for deletion the peering is effectively gone. + ID: existing.ID, Name: req.Name, Partition: req.Partition, DeletedAt: structs.TimeToProto(time.Now().UTC()), @@ -837,6 +873,26 @@ func getTrustDomain(store Store, logger hclog.Logger) (string, error) { return connect.SpiffeIDSigningForCluster(cfg.ClusterID).Host(), nil } +func (s *Service) getExistingOrCreateNewPeerID(peerName, partition string) (string, error) { + q := state.Query{ + Value: strings.ToLower(peerName), + EnterpriseMeta: *structs.NodeEnterpriseMetaInPartition(partition), + } + _, peering, err := s.Backend.Store().PeeringRead(nil, q) + if err != nil { + return "", err + } + if peering != nil { + return peering.ID, nil + } + + id, err := lib.GenerateUUID(s.Backend.Apply().CheckPeeringUUID) + if err != nil { + return "", err + } + return id, nil +} + func (s *Service) StreamStatus(peer string) (resp StreamStatus, found bool) { return s.streams.streamStatus(peer) } diff --git a/agent/rpc/peering/service_test.go b/agent/rpc/peering/service_test.go index aba7973d0..af089f56c 100644 --- a/agent/rpc/peering/service_test.go +++ b/agent/rpc/peering/service_test.go @@ -30,6 +30,7 @@ import ( "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/agent/token" "github.com/hashicorp/consul/api" + "github.com/hashicorp/consul/lib" "github.com/hashicorp/consul/proto/pbpeering" "github.com/hashicorp/consul/proto/pbservice" "github.com/hashicorp/consul/proto/prototest" @@ -224,6 +225,7 @@ func TestPeeringService_Read(t *testing.T) { // insert peering directly to state store p := &pbpeering.Peering{ + ID: testUUID(t), Name: "foo", State: pbpeering.PeeringState_INITIAL, PeerCAPems: nil, @@ -279,6 +281,7 @@ func TestPeeringService_Delete(t *testing.T) { s := newTestServer(t, nil) p := &pbpeering.Peering{ + ID: testUUID(t), Name: "foo", State: pbpeering.PeeringState_INITIAL, PeerCAPems: nil, @@ -316,6 +319,7 @@ func TestPeeringService_List(t *testing.T) { // Note that the state store holds reference to the underlying // variables; do not modify them after writing. foo := &pbpeering.Peering{ + ID: testUUID(t), Name: "foo", State: pbpeering.PeeringState_INITIAL, PeerCAPems: nil, @@ -324,6 +328,7 @@ func TestPeeringService_List(t *testing.T) { } require.NoError(t, s.Server.FSM().State().PeeringWrite(10, foo)) bar := &pbpeering.Peering{ + ID: testUUID(t), Name: "bar", State: pbpeering.PeeringState_ACTIVE, PeerCAPems: nil, @@ -405,6 +410,7 @@ func TestPeeringService_TrustBundleListByService(t *testing.T) { lastIdx++ require.NoError(t, s.Server.FSM().State().PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testUUID(t), Name: "foo", State: pbpeering.PeeringState_INITIAL, PeerServerName: "test", @@ -413,6 +419,7 @@ func TestPeeringService_TrustBundleListByService(t *testing.T) { lastIdx++ require.NoError(t, s.Server.FSM().State().PeeringWrite(lastIdx, &pbpeering.Peering{ + ID: testUUID(t), Name: "bar", State: pbpeering.PeeringState_INITIAL, PeerServerName: "test-bar", @@ -513,6 +520,7 @@ func Test_StreamHandler_UpsertServices(t *testing.T) { ) require.NoError(t, s.Server.FSM().State().PeeringWrite(0, &pbpeering.Peering{ + ID: testUUID(t), Name: "my-peer", })) @@ -998,7 +1006,9 @@ func newDefaultDeps(t *testing.T, c *consul.Config) consul.Deps { } func setupTestPeering(t *testing.T, store *state.Store, name string, index uint64) string { + t.Helper() err := store.PeeringWrite(index, &pbpeering.Peering{ + ID: testUUID(t), Name: name, }) require.NoError(t, err) @@ -1009,3 +1019,9 @@ func setupTestPeering(t *testing.T, store *state.Store, name string, index uint6 return p.ID } + +func testUUID(t *testing.T) string { + v, err := lib.GenerateUUID(nil) + require.NoError(t, err) + return v +} diff --git a/agent/rpc/peering/stream_test.go b/agent/rpc/peering/stream_test.go index dc30fa686..9bc8eff4e 100644 --- a/agent/rpc/peering/stream_test.go +++ b/agent/rpc/peering/stream_test.go @@ -23,6 +23,7 @@ import ( "github.com/hashicorp/consul/agent/consul/state" "github.com/hashicorp/consul/agent/consul/stream" "github.com/hashicorp/consul/agent/structs" + "github.com/hashicorp/consul/lib" "github.com/hashicorp/consul/proto/pbcommon" "github.com/hashicorp/consul/proto/pbpeering" "github.com/hashicorp/consul/proto/pbservice" @@ -1030,6 +1031,10 @@ type testApplier struct { store *state.Store } +func (a *testApplier) CheckPeeringUUID(id string) (bool, error) { + panic("not implemented") +} + func (a *testApplier) PeeringWrite(req *pbpeering.PeeringWriteRequest) error { panic("not implemented") } @@ -1216,6 +1221,7 @@ func writeEstablishedPeering(t *testing.T, store *state.Store, idx uint64, peerN require.NoError(t, err) peering := pbpeering.Peering{ + ID: testUUID(t), Name: peerName, PeerID: remotePeerID, } @@ -2169,5 +2175,10 @@ func requireEqualInstances(t *testing.T, expect, got structs.CheckServiceNodes) require.Equal(t, expect[i].Checks[j].PartitionOrDefault(), got[i].Checks[j].PartitionOrDefault(), "partition mismatch") } } - +} + +func testUUID(t *testing.T) string { + v, err := lib.GenerateUUID(nil) + require.NoError(t, err) + return v } diff --git a/agent/rpc/peering/subscription_manager_test.go b/agent/rpc/peering/subscription_manager_test.go index a7c49090b..d556ff23e 100644 --- a/agent/rpc/peering/subscription_manager_test.go +++ b/agent/rpc/peering/subscription_manager_test.go @@ -589,6 +589,7 @@ func (b *testSubscriptionBackend) ensureCARoots(t *testing.T, roots ...*structs. func setupTestPeering(t *testing.T, store *state.Store, name string, index uint64) string { err := store.PeeringWrite(index, &pbpeering.Peering{ + ID: testUUID(t), Name: name, }) require.NoError(t, err) From d22986ad0dee273e219f02c5499b6b4056766374 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 14:06:53 -0400 Subject: [PATCH 098/149] Update version of consul Helm chart --- website/content/docs/api-gateway/consul-api-gateway-install.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index fb6cbdd71..ce942b2e4 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -44,7 +44,7 @@ Ensure that the environment you are deploying Consul API Gateway in meets the re 1. Install Consul API Gateway using the standard Consul Helm chart and specify the custom values file. Available versions of the [Consul Helm chart](https://github.com/hashicorp/consul-k8s/releases) can be found in GitHub releases. ```shell-session - $ helm install consul hashicorp/consul --version 0.43.0 --values values.yaml --create-namespace --namespace consul + $ helm install consul hashicorp/consul --version 0.45.0 --values values.yaml --create-namespace --namespace consul ``` ## Usage From 7ec371e8985249af6f308cf282d5c0220f2ef6b5 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 14:07:07 -0400 Subject: [PATCH 099/149] Fix whitespace inconsistencies --- .../docs/api-gateway/consul-api-gateway-install.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index ce942b2e4..1fdb05e71 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -168,12 +168,12 @@ The following table describes the allowed parameters for the `spec` array: | `consul.ports.http` | Specifies the port for Consul's HTTP server. | Integer | `8500` | | `consul.scheme` | Specifies the scheme to use for connecting to Consul. The supported values are `"http"` and `"https"`. | String | `"http"` | | `copyAnnotations.service` | List of annotations to copy to the gateway service. | Array | `["external-dns.alpha.kubernetes.io/hostname"]` | -| `deployment.defaultInstances` | Specifies the number of instances to deploy by default for each gateway. | Integer | 1 | -| `deployment.maxInstances` | Specifies the maximum allowed number of instances per gateway. | Integer | 8 | -| `deployment.minInstances` | Specifies the minimum allowed number of instances per gateway. | Integer | 1 | -| `image.consulAPIGateway` | The image to use for consul-api-gateway. View available image tags on [DockerHub](https://hub.docker.com/r/hashicorp/consul-api-gateway/tags). | String | `"hashicorp/consul-api-gateway:RELEASE_VERSION"` | -| `image.envoy` | Specifies the container image to use for Envoy. View available image tags on [DockerHub](https://hub.docker.com/r/envoyproxy/envoy/tags). | String | `"envoyproxy/envoy:RELEASE_VERSION"` | -| `logLevel` | Specifies the error reporting level for logs. You can specify the following values: `error`, `warning`, `info`, `debug`, `trace`. | String | `"info"` | +| `deployment.defaultInstances` | Specifies the number of instances to deploy by default for each gateway. | Integer | 1 | +| `deployment.maxInstances` | Specifies the maximum allowed number of instances per gateway. | Integer | 8 | +| `deployment.minInstances` | Specifies the minimum allowed number of instances per gateway. | Integer | 1 | +| `image.consulAPIGateway` | The image to use for consul-api-gateway. View available image tags on [DockerHub](https://hub.docker.com/r/hashicorp/consul-api-gateway/tags). | String | `"hashicorp/consul-api-gateway:RELEASE_VERSION"` | +| `image.envoy` | Specifies the container image to use for Envoy. View available image tags on [DockerHub](https://hub.docker.com/r/envoyproxy/envoy/tags). | String | `"envoyproxy/envoy:RELEASE_VERSION"` | +| `logLevel` | Specifies the error reporting level for logs. You can specify the following values: `error`, `warning`, `info`, `debug`, `trace`. | String | `"info"` | | `nodeSelector` | Specifies a set of parameters that constrain the nodes on which the pod can run. Defining nodes with the `nodeSelector` enables the pod to fit on a node. The selector must match a node's labels for the pod to be scheduled on that node. Refer to the [Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/assign-pod-node/) for additional information. | Object | N/A | | `serviceType` | Specifies the ingress methods for a service. The following values are supported:
`ClusterIP`
`NodePort`
`LoadBalancer`. | String | N/A | | `useHostPorts` | If set to `true`, then the Envoy container ports are mapped to host ports. | Boolean | `false` | From 7d5b541e5282a640c96531e0d9a5206a669d82d0 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 14:07:23 -0400 Subject: [PATCH 100/149] Use new release version in examples --- website/content/docs/api-gateway/consul-api-gateway-install.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 1fdb05e71..8dc6bdd5c 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -15,7 +15,7 @@ Ensure that the environment you are deploying Consul API Gateway in meets the re ## Installation --> **NOTE:** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.2.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.2.0` +-> **NOTE:** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.3.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.3.0` 1. Issue the following command to install the CRDs: From 3135513c41d98dc90342b7a9d2e5edc1600935c5 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 14:08:18 -0400 Subject: [PATCH 101/149] Update website/content/docs/api-gateway/consul-api-gateway-install.mdx Co-authored-by: Tu Nguyen --- .../content/docs/api-gateway/consul-api-gateway-install.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 8dc6bdd5c..dd9f92667 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -246,8 +246,8 @@ spec: For a listener's `certificateRef` to reference a secret in a different namespace, you must also create a [ReferencePolicy](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy). -The following example creates a `Gateway` named `example-gateway` in namespace `gateway-namespace`. This `Gateway` has a `certificateRef` in namespace `secret-namespace`. -The reference is allowed because the `ReferencePolicy`, named `reference-policy` in namespace `secret-namespace`, allows `Gateways` in `gateway-namespace` to reference `Secrets` in `secret-namespace`. +The following example creates a `Gateway` named `example-gateway` in `gateway-namespace`. This `Gateway` has a `certificateRef` in `secret-namespace`. +The reference is allowed because `reference-policy` in `secret-namespace` lets `Gateways` in `gateway-namespace` to reference `Secrets` in `secret-namespace`. From 1382cfacdcd7878e78e0745f99c5bd60c20c51e5 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 14:08:42 -0400 Subject: [PATCH 102/149] Update website/content/docs/api-gateway/consul-api-gateway-install.mdx --- website/content/docs/api-gateway/consul-api-gateway-install.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index dd9f92667..56775656d 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -247,7 +247,7 @@ spec: For a listener's `certificateRef` to reference a secret in a different namespace, you must also create a [ReferencePolicy](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy). The following example creates a `Gateway` named `example-gateway` in `gateway-namespace`. This `Gateway` has a `certificateRef` in `secret-namespace`. -The reference is allowed because `reference-policy` in `secret-namespace` lets `Gateways` in `gateway-namespace` to reference `Secrets` in `secret-namespace`. +The reference is allowed because `reference-policy` in `secret-namespace` lets `Gateways` in `gateway-namespace` reference `Secrets` in `secret-namespace`. From c150662a443a486d70a8174fcb8eecf8fe363749 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 14:59:40 -0400 Subject: [PATCH 103/149] Apply suggestions from code review Co-authored-by: Tu Nguyen --- .../docs/api-gateway/consul-api-gateway-install.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/content/docs/api-gateway/consul-api-gateway-install.mdx b/website/content/docs/api-gateway/consul-api-gateway-install.mdx index 56775656d..fb48565ba 100644 --- a/website/content/docs/api-gateway/consul-api-gateway-install.mdx +++ b/website/content/docs/api-gateway/consul-api-gateway-install.mdx @@ -317,18 +317,18 @@ Refer to the [Kubernetes Gateway API documentation](https://gateway-api.sigs.k8s #### Scaling -You can scale a logical gateway object to multiple instances with the [`kubectl scale`](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment) command. The object scales according to the bounds set in GatewayClassConfig. +You can scale a logical gateway object to multiple instances with the [`kubectl scale`](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#scaling-a-deployment) command. The object scales according to the bounds set in `GatewayClassConfig`. -``` +```shell-session $ kubectl get deployment --selector api-gateway.consul.hashicorp.com/name=example-gateway NAME READY UP-TO-DATE AVAILABLE example-gateway 1/1 1 1 ``` -``` +```shell-session $ kubectl scale deployment/example-gateway --replicas=3 deployment.apps/example-gateway scaled ``` -``` +```shell-session $ kubectl get deployment --selector api-gateway.consul.hashicorp.com/name=example-gateway NAME READY UP-TO-DATE AVAILABLE example-gateway 3/3 3 3 From 2422c8bf374565fb2ae456a9adcf820b5b9d15f2 Mon Sep 17 00:00:00 2001 From: David Yu Date: Tue, 21 Jun 2022 12:10:43 -0700 Subject: [PATCH 104/149] docs: Consul K8s compat matrix update for 0.45.0+ to include Consul 1.11 compatibility (#13528) --- .../content/docs/k8s/installation/compatibility.mdx | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/website/content/docs/k8s/installation/compatibility.mdx b/website/content/docs/k8s/installation/compatibility.mdx index cb2370f83..2d651401d 100644 --- a/website/content/docs/k8s/installation/compatibility.mdx +++ b/website/content/docs/k8s/installation/compatibility.mdx @@ -14,11 +14,11 @@ For every release of Consul on Kubernetes, a Helm chart, `consul-k8s-control-pla Starting with Consul Kubernetes 0.33.0, Consul Kubernetes versions all of its components (`consul-k8s` CLI, `consul-k8s-control-plane`, and Helm chart) with a single semantic version. -| Consul Version | Compatible consul-k8s Versions | -| -------------- | ------------------------------- | -| 1.12.x | 0.43.0 - latest | -| 1.11.x | 0.39.0 - 0.42.0, 0.44.0 | -| 1.10.x | 0.33.0 - 0.38.0 | +| Consul Version | Compatible consul-k8s Versions | +| -------------- | -------------------------------- | +| 1.12.x | 0.43.0 - latest | +| 1.11.x | 0.39.0 - 0.42.0, 0.44.0 - latest | +| 1.10.x | 0.33.0 - 0.38.0 | ### Prior to version 0.33.0 From f9d80e768ada1c3996fd426562ebadb886eb6e15 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 15:37:26 -0400 Subject: [PATCH 105/149] Apply suggestions from code review Co-authored-by: Tu Nguyen --- .../content/docs/api-gateway/upgrade-specific-versions.mdx | 4 ++-- .../content/docs/release-notes/consul-api-gateway/v0_3_x.mdx | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 9c900a912..c2d69b869 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -48,7 +48,7 @@ Ensure that the following requirements are met prior to upgrading: ``` - If you have any active `Gateways`, you will receive output similar to the following response. The output has been truncated to show only relevant fields: + If you have any active `Gateways`, you will receive output similar to the following response. The output has been truncated to show only relevant fields: ```yaml apiVersion: v1 @@ -392,7 +392,7 @@ Ensure that the following requirements are met prior to upgrading: ## Standard Upgrade --> **NOTE** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.2.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.2.0` +~> **Note:** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.2.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.2.0` ### Requirements diff --git a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx index 598808616..7d6bf7ba3 100644 --- a/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx +++ b/website/content/docs/release-notes/consul-api-gateway/v0_3_x.mdx @@ -23,7 +23,7 @@ description: >- per `gateway` to all `gateways` in that `gatewayClass`. The minimum and maximum settings will override settings configured on any given `gateway`. - Note: There is a hard limit of eight (8) instances per `gateway` and it can + ~> **Note:** There is a hard limit of eight (8) instances per `gateway` and it can not be overridden. This hard limit will probably be increased in the future. - **Cross Namespace Reference Policies for Certificates:** @@ -54,6 +54,6 @@ For detailed information on upgrading, please refer to the [upgrade details page The changelogs for this major release version and any maintenance versions are listed below. --> **Note**: These links will take you to the changelogs on the GitHub website. +~> **Note:** The following link will take you to the changelogs on the GitHub website. - [0.3.0](https://github.com/hashicorp/consul-api-gateway/releases/tag/v0.3.0) From be6fe117846d8574cc27c061060b4c1f98a47687 Mon Sep 17 00:00:00 2001 From: trujillo-adam Date: Tue, 21 Jun 2022 13:20:14 -0700 Subject: [PATCH 106/149] applied suggestions from review, udpates to TF secure configuration --- .../content/docs/ecs/terraform/install.mdx | 4 +- .../ecs/terraform/secure-configuration.mdx | 86 ++++++++++--------- 2 files changed, 48 insertions(+), 42 deletions(-) diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index 99099d21b..87bf6da0c 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -22,13 +22,13 @@ The following procedure describes the general workflow: 2. [Run Terraform](#running-terraform) to deploy the resources in AWS -If you want to operate Consul with ACLs enabled (recommended), follow the instructions in the [Secure Configuration](/docs/ecs/terraform/secure-configuration) documentation. ACLs provide network security for production-grade deployments. +If you want to operate Consul in production environments, follow the instructions in the [Secure Configuration](/docs/ecs/terraform/secure-configuration) documentation. The instructions describe how to enable ACLs and TLS and gossip encyption, which provide network security for production-grade deployments. ## Requirements * You should have some familiarity with using Terraform. Refer to the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform. * You should also be familiar with AWS ECS before following these instructions. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for details. -* If you intend to [use the `gateway-task` module to deploy mesh gateways](#configure-the-gateway-task-module), all Consul server and client agents in all datacenters must have TLS and gossip encryption enabled (refer to [Encryption](/docs/security/encryption) for instructions). +* If you intend to [use the `gateway-task` module to deploy mesh gateways](#configure-the-gateway-task-module), all Consul server and client agents in all datacenters must have TLS and gossip encryption enabled. Refer to the [Secure Configuration](/docs/ecs/terraform/secure-configuration) documentation for instructions. ## Create the task definition diff --git a/website/content/docs/ecs/terraform/secure-configuration.mdx b/website/content/docs/ecs/terraform/secure-configuration.mdx index 521141da8..3a7f60e74 100644 --- a/website/content/docs/ecs/terraform/secure-configuration.mdx +++ b/website/content/docs/ecs/terraform/secure-configuration.mdx @@ -7,63 +7,69 @@ description: >- # Secure Configuration -This topic describes how to enable Consul security features for your production workloads. The following overview describes the process: +This topic describes how to enable Consul security features for your production workloads. + +## Overview + +To enable security in your production workloads, you must deploy the [ACL controller](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller), which provisions tokens for other service mesh tasks. Refer to [Automatic ACL Token Provisioning](/docs/ecs/architecture#automatic-acl-token-provisioning) to learn more about the ACL controller. + +The controller cannot provision tokens for itself, so you must create the token for the ACL controller. The following steps describe the overall process of enabling security features for your production workloads: 1. Enable the security features on your Consul server cluster per the [Prerequisites](#prerequisites). -1. Deploy the ACL controller. -1. Deploy your services. +1. Create the ACL token for the ACL controller in the datacenter. +1. Create a Secrets Manager secret containing the ACL controller's token. +1. Create a Secrets Manager secret containing the Consul CA certificate. +1. Deploy the ACL controller +1. Deploy the other services on the mesh. ## Prerequisites -Implement the following configurations before proceeding: +Implement the following security features for your Consul server clusters before applying them to your workloads: 1. [TLS encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. 1. [Gossip encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. 1. [Access control lists (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. +## ACL controller -## Deploy the ACL controller +1. Create a policy that grants `acl:write` and `operator:write` access for the controller. Refer to the [ACL policies documentation](/docs/security/acl/policies) for instructions. +1. Create a token and link it to the ACL controller policy. Refer to the [ACL tokens documentation](/docs/security/acl/tokens) for instructions. +1. Create a Secrets Manager secret containing the ACL controller's token and a Secrets Manager secret containing the Consul CA cert. -Before deploying your service, you will need to deploy the [ACL controller](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller) so that it can provision the necessary tokens -for tasks on the service mesh. To learn more about the ACL Controller, please see [Automatic ACL Token Provisioning](/docs/ecs/architecture#automatic-acl-token-provisioning). + ```hcl + resource "aws_secretsmanager_secret" "bootstrap_token" { + name = "bootstrap-token" + } -To deploy the controller, you will first need to store an ACL token with `acl:write` and `operator:write` privileges, -and a CA certificate for the Consul server in AWS Secrets Manager. + resource "aws_secretsmanager_secret_version" "bootstrap_token" { + secret_id = aws_secretsmanager_secret.bootstrap_token.id + secret_string = "" + } -```hcl -resource "aws_secretsmanager_secret" "bootstrap_token" { - name = "bootstrap-token" -} + resource "aws_secretsmanager_secret" "ca_cert" { + name = "server-ca-cert" + } -resource "aws_secretsmanager_secret_version" "bootstrap_token" { - secret_id = aws_secretsmanager_secret.bootstrap_token.id - secret_string = "" -} + resource "aws_secretsmanager_secret_version" "ca_cert" { + secret_id = aws_secretsmanager_secret.ca_cert.id + secret_string = "" + } + ``` -resource "aws_secretsmanager_secret" "ca_cert" { - name = "server-ca-cert" -} +1. Use the [`acl-controller` terraform module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller?tab=inputs) to deploy the controller. You must provide the ARN's for the token and CA cert in the `consul_bootstrap_token_secret_arn` and `consul_server_ca_cert_arn` fields, respectively. -resource "aws_secretsmanager_secret_version" "ca_cert" { - secret_id = aws_secretsmanager_secret.ca_cert.id - secret_string = "" -} -``` - -Use the [`acl-controller` terraform module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller?tab=inputs) to deploy the controller: - -```hcl -module "acl_controller" { - source = "hashicorp/consul/aws-ecs//modules/acl-controller" - consul_bootstrap_token_secret_arn = aws_secretsmanager_secret.bootstrap_token.arn - consul_server_http_addr = "https://consul-server.example.com:8501" - consul_server_ca_cert_arn = aws_secretsmanager_secret.ca_cert.arn - ecs_cluster_arn = "arn:aws:ecs:my-region:111111111111:cluster/consul-ecs" - region = "my-region" - subnets = ["subnet-abcdef123456789"] - name_prefix = "consul-ecs" -} -``` + ```hcl + module "acl_controller" { + source = "hashicorp/consul/aws-ecs//modules/acl-controller" + consul_bootstrap_token_secret_arn = aws_secretsmanager_secret.bootstrap_token.arn + consul_server_http_addr = "https://consul-server.example.com:8501" + consul_server_ca_cert_arn = aws_secretsmanager_secret.ca_cert.arn + ecs_cluster_arn = "arn:aws:ecs:my-region:111111111111:cluster/consul-ecs" + region = "my-region" + subnets = ["subnet-abcdef123456789"] + name_prefix = "consul-ecs" + } + ``` The `name_prefix` parameter is used to prefix any secrets that the ACL controller will update in AWS Secrets Manager. The `name_prefix` parameter value must be unique for each ECS cluster where you deploy this controller. From acbbfac8c2a016a3c70eeabe30c1e433061cfe7c Mon Sep 17 00:00:00 2001 From: Sarah Alsmiller Date: Tue, 21 Jun 2022 15:56:57 -0500 Subject: [PATCH 107/149] edit secret to certificateRef --- .../api-gateway/upgrade-specific-versions.mdx | 92 ++++++++----------- 1 file changed, 37 insertions(+), 55 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index c2d69b869..d6abe63e4 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -12,7 +12,7 @@ This topic describes how to upgrade Consul API Gateway. ## v0.3.0 -Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with a `secret` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows `Gateways` from the gateway's namesapce to use `Secrets` in the secret's namespace. +Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with `listner` with a `certificateRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows `Gateways` from the gateway's namesapce to use `certificateRef` in the `certificateRef`'s namespace. ### Requirements @@ -41,7 +41,7 @@ Ensure that the following requirements are met prior to upgrading: "hashicorp/consul-api-gateway:0.2.1" ``` -1. Retrieve all gateways that have a secret in a different namespace. If you have installed the [`jq`](https://stedolan.github.io/jq/) utility, you can skip to [step 4](#jq-command-secrets). Otherwise, issue the following command to get all `Gateways` across all namespaces: +1. Retrieve all gateways that have a `certificateRef` in a different namespace. If you have installed the [`jq`](https://stedolan.github.io/jq/) utility, you can skip to [step 4](#jq-command-secrets). Otherwise, issue the following command to get all `Gateways` across all namespaces: ```shell-session $ kubectl get Gateway --output json --all-namespaces @@ -51,61 +51,39 @@ Ensure that the following requirements are met prior to upgrading: If you have any active `Gateways`, you will receive output similar to the following response. The output has been truncated to show only relevant fields: ```yaml - apiVersion: v1 - items: - - apiVersion: gateway.networking.k8s.io/v1alpha2 - kind: HTTPRoute - metadata: - name: example-http-route, - namespace: example-namespace, - ... - spec: - parentRefs: - - group: gateway.networking.k8s.io - kind: Gateway - name: gateway - namespace: gw-ns - rules: - - backendRefs: - - group: "" - kind: Service - name: web-backend - namespace: gateway-namespace - ... - ... - - apiVersion: gateway.networking.k8s.io/v1alpha2 - kind: TCPRoute - metadata: - name: example-tcp-route, - namespace: a-different-namespace, - ... - spec: - parentRefs: - - group: gateway.networking.k8s.io - kind: Gateway - name: gateway - namespace: gateway-namespace - rules: - - backendRefs: - - group: "" - kind: Service - name: web-backend - namespace: gateway-namespace - ... - ... + apiVersion: gateway.networking.k8s.io/v1alpha2 + kind: Gateway + metadata: + name: example-gateway + namespace: gateway-namespace + spec: + gatewayClassName: "consul-api-gateway" + listeners: + - name: https + port: 443 + protocol: HTTPS + allowedRoutes: + namespaces: + from: All + tls: + certificateRefs: + - group: "" + kind: Secret + name: example-certificate + namespace: certificate-namespace ``` -1. Inspect the `secret` entries for each of the routes. +1. Inspect the `certificateRefs` entries for each of the routes. - If a `namespace` field is not defined in the `secret` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `secret`. Otherwise, note the `namespace` field values for `secret` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace secret-to-gateway pair to prevent the route from breaking (see [step 5](#create-secret-reference-policy)). + If a `namespace` field is not defined in the `certificateRef` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `certificateRef`. Otherwise, note the `namespace` field values for `certificateRef` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace certificateRef-to-gateway pair to prevent the route from breaking (see [step 5](#create-secret-reference-policy)). After completing this step, you will have a list of all secrets similar to the following: ```yaml hideClipboard - example-secret: - - namespace: secret-namespace + example-certificate: + - namespace: certificate-namespace parentNamespace: gateway-namespace ``` @@ -133,8 +111,8 @@ Ensure that the following requirements are met prior to upgrading: "crossNamespaceSecrets": { "group": "", "kind": "Secret", - "name": "secret-name", - "namespace": "secret-namespace" + "name": "cexample-certificate", + "namespace": "certificate-namespace" } } ``` @@ -145,13 +123,13 @@ Ensure that the following requirements are met prior to upgrading: 1. Using the list of secrets you created earlier as a guide, create a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) to allow each gateway cross namespace secret access. - The `ReferencePolicy` explicitly allows each cross-namespace gateway to secret pair. The `ReferencePolicy` must be created in the same `namespace` as the `Secret`. + The `ReferencePolicy` explicitly allows each cross-namespace gateway to secret pair. The `ReferencePolicy` must be created in the same `namespace` as the `certificateRef`. Skip to the next step if you've already created a `ReferencePolicy`. - The following example `ReferencePolicy` enables `example-gateway` in `gateway-namespace` to utilize secrets in the `secret-namespace` namespace: + The following example `ReferencePolicy` enables `example-gateway` in `gateway-namespace` to utilize `certificateRefs` in the `certificate-namespace` namespace: @@ -160,7 +138,7 @@ Ensure that the following requirements are met prior to upgrading: kind: ReferencePolicy metadata: name: reference-policy - namespace: secret-namespace + namespace: certificate-namespace spec: from: - group: gateway.networking.k8s.io @@ -173,7 +151,7 @@ Ensure that the following requirements are met prior to upgrading: -1. If you have already created a `ReferencePolicy`, modify it to allow your route and save it as `referencepolicy.yaml`. Note that each `ReferencePolicy` only supports one `to` field and one `from` field (refer the [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencepolicy/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferencePolicy`s. +1. If you have already created a `ReferencePolicy`, modify it to allow your gateway to access your `certificateRef` and save it as `referencepolicy.yaml`. Note that each `ReferencePolicy` only supports one `to` field and one `from` field (refer the [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/api-types/referencepolicy/#api-design-decisions) documentation). As a result, you may need to create multiple `ReferencePolicy`s. 1. Issue the following command to apply it to your cluster: @@ -181,8 +159,9 @@ Ensure that the following requirements are met prior to upgrading: $ kubectl apply --filename referencepolicy.yaml ``` - Repeat this step as needed until each of your cross-namespace secrets have a corresponding `ReferencePolicy`. + Repeat this step as needed until each of your cross-namespace `certificateRefs` have a corresponding `ReferencePolicy`. + Proceed with the [standard-upgrade](#standard-upgrade). ## v0.2.0 @@ -390,6 +369,9 @@ Ensure that the following requirements are met prior to upgrading: Repeat this step as needed until each of your cross-namespace routes have a corresponding `ReferencePolicy`. + Proceed with the [standard-upgrade](#standard-upgrade). + + ## Standard Upgrade ~> **Note:** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.2.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.2.0` From 5d123e02baaf39cbd3b1c92583ea36b0954d1fa1 Mon Sep 17 00:00:00 2001 From: Sarah Alsmiller Date: Tue, 21 Jun 2022 16:21:29 -0500 Subject: [PATCH 108/149] render issue --- .../docs/api-gateway/upgrade-specific-versions.mdx | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index d6abe63e4..3d0080f74 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -47,7 +47,6 @@ Ensure that the following requirements are met prior to upgrading: $ kubectl get Gateway --output json --all-namespaces ``` - If you have any active `Gateways`, you will receive output similar to the following response. The output has been truncated to show only relevant fields: ```yaml @@ -90,14 +89,16 @@ Ensure that the following requirements are met prior to upgrading: Proceed with the [standard-upgrade](#standard-upgrade) if your list is empty. - + + 1. If you have installed [`jq`](https://stedolan.github.io/jq/), issue the following command to get all `Gateways` and filter for secrets that require a `ReferencePolicy`. ```shell-session - $ kubectl get Gateway -o json -A | jq -r '.items[] | {gateway_name: .metadata.name, gateway_namespace: .metadata.namespace, kind: .kind, crossNamespaceSecrets: ( .metadata.namespace as $parentnamespace | .spec.listeners[] | select(has("tls")) | .tls.certificateRefs[] | select(.namespace != null and .namespace != $parentnamespace ) )} ' - ``` + $ kubectl get Gateway -o json -A | jq -r '.items[] | {gateway_name: .metadata.name, gateway_namespace: .metadata.namespace, kind: .kind, crossNamespaceSecrets: ( .metadata.namespace as $parentnamespace | .spec.listeners[] | select(has("tls")) | .tls.certificateRefs[] | select(.namespace != null and .namespace != $parentnamespace ) )} ' + + ``` The output will resemble the following response if gateways that require a new `ReferencePolicy` are returned: From 83ef4a260e48f0868dd99fb3971a41e0e75f83cb Mon Sep 17 00:00:00 2001 From: Sarah Alsmiller Date: Tue, 21 Jun 2022 16:26:46 -0500 Subject: [PATCH 109/149] typo --- website/content/docs/api-gateway/upgrade-specific-versions.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 3d0080f74..bb439c15c 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -24,7 +24,7 @@ Ensure that the following requirements are met prior to upgrading: - You should have the following permission rights on your Kubernetes cluster: - `Gateway.read` - `ReferencePolicy.create` -- (Optional) The [jq](https://stedolan.github.io/jq/download/) command line processor for JSON can be installed, which will ease route retrieval during the upgrade process. +- (Optional) The [jq](https://stedolan.github.io/jq/download/) command line processor for JSON can be installed, which will ease gateway retrieval during the upgrade process. ### Procedure From bbcad1a23037fc69baf7393f11371c86c65d4d98 Mon Sep 17 00:00:00 2001 From: Sarah Alsmiller Date: Tue, 21 Jun 2022 16:33:11 -0500 Subject: [PATCH 110/149] clarified wording --- website/content/docs/api-gateway/upgrade-specific-versions.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index bb439c15c..6119b9602 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -74,7 +74,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Inspect the `certificateRefs` entries for each of the routes. - If a `namespace` field is not defined in the `certificateRef` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `certificateRef`. Otherwise, note the `namespace` field values for `certificateRef` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace certificateRef-to-gateway pair to prevent the route from breaking (see [step 5](#create-secret-reference-policy)). + If a `namespace` field is not defined in the `certificateRef` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `certificateRef`. Otherwise, note the `namespace` field values for `certificateRef` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace certificateRef-to-gateway pair. (see [step 5](#create-secret-reference-policy)). After completing this step, you will have a list of all secrets similar to the following: From 4c0b4d026c1fc81c7deb0d5db7b12c58e961fe34 Mon Sep 17 00:00:00 2001 From: David Yu Date: Tue, 21 Jun 2022 14:43:10 -0700 Subject: [PATCH 111/149] docs: move k8s compat matrix top level (#13534) --- .../content/docs/k8s/{installation => }/compatibility.mdx | 0 website/data/docs-nav-data.json | 8 ++++---- website/redirects.js | 7 ++++++- 3 files changed, 10 insertions(+), 5 deletions(-) rename website/content/docs/k8s/{installation => }/compatibility.mdx (100%) diff --git a/website/content/docs/k8s/installation/compatibility.mdx b/website/content/docs/k8s/compatibility.mdx similarity index 100% rename from website/content/docs/k8s/installation/compatibility.mdx rename to website/content/docs/k8s/compatibility.mdx diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index e917d90db..cec473df1 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -553,10 +553,6 @@ "path": "k8s/installation/vault/wan-federation" } ] - }, - { - "title": "Compatibility Matrix", - "path": "k8s/installation/compatibility" } ] }, @@ -674,6 +670,10 @@ } ] }, + { + "title": "Compatibility Matrix", + "path": "k8s/compatibility" + }, { "title": "Helm Chart Configuration", "path": "k8s/helm" diff --git a/website/redirects.js b/website/redirects.js index 59be38aad..c2f751df7 100644 --- a/website/redirects.js +++ b/website/redirects.js @@ -1109,7 +1109,12 @@ module.exports = [ }, { source: '/docs/k8s/upgrade/compatibility', - destination: '/docs/k8s/installation/compatibility', + destination: '/docs/k8s/compatibility', + permanent: true, + }, + { + source: '/docs/k8s/installation/compatibility', + destination: '/docs/k8s/compatibility', permanent: true, }, { From 1709de307a312a8b08740b1e5c4157ac133f42e0 Mon Sep 17 00:00:00 2001 From: Nathan Coleman Date: Tue, 21 Jun 2022 17:47:45 -0400 Subject: [PATCH 112/149] Update website/content/docs/api-gateway/upgrade-specific-versions.mdx --- website/content/docs/api-gateway/upgrade-specific-versions.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 6119b9602..562f640b2 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -12,7 +12,7 @@ This topic describes how to upgrade Consul API Gateway. ## v0.3.0 -Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with `listner` with a `certificateRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows `Gateways` from the gateway's namesapce to use `certificateRef` in the `certificateRef`'s namespace. +Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with `listeners` with a `certificateRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows `Gateways` from the gateway's namesapce to use `certificateRef` in the `certificateRef`'s namespace. ### Requirements From fa204fc0e0bdbb7d631084ed18f30cf9b81ad933 Mon Sep 17 00:00:00 2001 From: Sarah Alsmiller Date: Tue, 21 Jun 2022 16:48:49 -0500 Subject: [PATCH 113/149] removed references to the word breaking --- .../docs/api-gateway/upgrade-specific-versions.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index 6119b9602..ba47cd1ee 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -12,7 +12,7 @@ This topic describes how to upgrade Consul API Gateway. ## v0.3.0 -Consul API Gateway v0.3.0 introduces a breaking change for people upgrading from lower versions. Gateways with `listner` with a `certificateRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows `Gateways` from the gateway's namesapce to use `certificateRef` in the `certificateRef`'s namespace. +Consul API Gateway v0.3.0 introduces a change for people upgrading from lower versions. Gateways with `listner` with a `certificateRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows `Gateways` from the gateway's namesapce to use `certificateRef` in the `certificateRef`'s namespace. ### Requirements @@ -166,7 +166,7 @@ Ensure that the following requirements are met prior to upgrading: ## v0.2.0 -Consul API Gateway v0.2.0 introduces a breaking change for people upgrading from Consul API Gateway v0.1.0. Routes with a `backendRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. +Consul API Gateway v0.2.0 introduces a change for people upgrading from Consul API Gateway v0.1.0. Routes with a `backendRef` defined in a different namespace now require a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) that explicitly allows traffic from the route's namespace to the `backendRef`'s namespace. ### Requirements @@ -253,7 +253,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Inspect the `backendRefs` entries for each of the routes. - If a `namespace` field is not defined in the `backendRef` or if the namespace matches the namespace of the route, then no additional action is required for the `backendRef`. Otherwise, note the `group`, `kind`, `name`, and `namespace` field values for `backendRef` configurations that have a `namespace` defined that do not match the namespace of the parent route. You must also note the `kind` and `namespace` of the parent route. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace route-to-service pair to prevent the route from breaking (see [step 5](#create-reference-policy)). + If a `namespace` field is not defined in the `backendRef` or if the namespace matches the namespace of the route, then no additional action is required for the `backendRef`. Otherwise, note the `group`, `kind`, `name`, and `namespace` field values for `backendRef` configurations that have a `namespace` defined that do not match the namespace of the parent route. You must also note the `kind` and `namespace` of the parent route. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace route-to-service pair (see [step 5](#create-reference-policy)). After completing this step, you will have a list of all routes similar to the following: @@ -331,7 +331,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Using the list of routes you created earlier as a guide, create a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) to allow cross namespace traffic for each route service pair. - The `ReferencePolicy` explicitly allows each cross-namespace route to service pair to prevent the route from breaking. The `ReferencePolicy` must be created in the same `namespace` as the backend `Service`. + The `ReferencePolicy` explicitly allows each cross-namespace route to service pair. The `ReferencePolicy` must be created in the same `namespace` as the backend `Service`. Skip to the next step if you've already created a `ReferencePolicy`. **NOTE:** When you see `VERSION` in examples of commands or configuration settings, replace `VERSION` with the version number of the release you are installing, like `0.3.0`. If there is a lower case "v" in front of `VERSION` the version number needs to follow the "v" as is `v0.3.0` +-> **Version reference convention:** Replace `VERSION` in command and configuration examples with the Consul API Gateway version you are installing, such as `0.3.0`. In some instances, `VERSION` is prepended with a lowercase _v_. This indicates that you must include the `v` as is part of the version, for example `v0.3.0`. 1. Issue the following command to install the CRDs: @@ -247,7 +247,7 @@ spec: For a listener's `certificateRef` to reference a secret in a different namespace, you must also create a [ReferencePolicy](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy). The following example creates a `Gateway` named `example-gateway` in `gateway-namespace`. This `Gateway` has a `certificateRef` in `secret-namespace`. -The reference is allowed because `reference-policy` in `secret-namespace` lets `Gateways` in `gateway-namespace` reference `Secrets` in `secret-namespace`. +The listener can use the certificate because `reference-policy` in `secret-namespace` is configured to allow `Gateways` in `gateway-namespace` to reference `Secrets` in `secret-namespace`. diff --git a/website/content/docs/api-gateway/upgrade-specific-versions.mdx b/website/content/docs/api-gateway/upgrade-specific-versions.mdx index fddf4a9f7..f926a0403 100644 --- a/website/content/docs/api-gateway/upgrade-specific-versions.mdx +++ b/website/content/docs/api-gateway/upgrade-specific-versions.mdx @@ -41,7 +41,7 @@ Ensure that the following requirements are met prior to upgrading: "hashicorp/consul-api-gateway:0.2.1" ``` -1. Retrieve all gateways that have a `certificateRef` in a different namespace. If you have installed the [`jq`](https://stedolan.github.io/jq/) utility, you can skip to [step 4](#jq-command-secrets). Otherwise, issue the following command to get all `Gateways` across all namespaces: +1. Retrieve all gateways that have a `certificateRefs` in a different namespace. If you have installed the [`jq`](https://stedolan.github.io/jq/) utility, you can skip to [step 4](#jq-command-secrets). Otherwise, issue the following command to get all `Gateways` across all namespaces: ```shell-session $ kubectl get Gateway --output json --all-namespaces @@ -74,7 +74,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Inspect the `certificateRefs` entries for each of the routes. - If a `namespace` field is not defined in the `certificateRef` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `certificateRef`. Otherwise, note the `namespace` field values for `certificateRef` configurations that have a `namespace` defined that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace certificateRef-to-gateway pair. (see [step 5](#create-secret-reference-policy)). + If a `namespace` field is not defined in the `certificateRefs` or if the namespace matches the namespace of the parent `Gateway`, then no additional action is required for the `certificateRefs`. Otherwise, note the `namespace` field values for `certificateRefs` configurations with a `namespace` field that do not match the namespace of the parent `Gateway`. You must also note the `namespace` of the parent gateway. You will need these to create a `ReferencePolicy` that explicitly allows each cross-namespace certificateRefs-to-gateway pair. (see [step 5](#create-secret-reference-policy)). After completing this step, you will have a list of all secrets similar to the following: @@ -124,7 +124,7 @@ Ensure that the following requirements are met prior to upgrading: 1. Using the list of secrets you created earlier as a guide, create a [`ReferencePolicy`](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.ReferencePolicy) to allow each gateway cross namespace secret access. - The `ReferencePolicy` explicitly allows each cross-namespace gateway to secret pair. The `ReferencePolicy` must be created in the same `namespace` as the `certificateRef`. + The `ReferencePolicy` explicitly allows each cross-namespace gateway to secret pair. The `ReferencePolicy` must be created in the same `namespace` as the `certificateRefs`. Skip to the next step if you've already created a `ReferencePolicy`. **NOTE:** If you are using the ACL controller with multiple ECS clusters, each cluster requires -its own instance of the ACL controller, and each instance of the ACL controller should have a unique -name prefix. +| Secret | Sample Secret Name | Description | +| --------------------- | ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Consul server CA cert | `my-consul-ca-cert` | The Consul server CA Cert for the HTTPS interface. This is required if the Consul server uses a self-signed or internal CA. It is not required for Consul servers in HCP. | +| Bootstrap ACL Token | `my-consul-bootstrap-token` | A Consul ACL token with `acl:write` and `operator:write` permissions. | ## Task Definition You must create a task definition to deploy the ACL controller in your ECS cluster. -The ACL controller must run in the same ECS cluster hosting your service mesh application +The ACL controller must run in the same ECS cluster that hosts your service mesh application tasks. The following example shows how the task definition should be configured for the ACL controller. @@ -62,11 +42,7 @@ The following example shows how the task definition should be configured for the "name": "acl-controller", "image": "public.ecr.aws/hashicorp/consul-ecs:", "essential": true, - "command": [ - "acl-controller", - "-consul-client-secret-arn", "arn:aws:secretsmanager:us-west-2:000000000000:secret:-consul-client-token", - "-secret-name-prefix", "", - ], + "command": ["acl-controller", "-iam-role-path", "/consul-ecs/"], "secrets": [ { "name": "CONSUL_HTTP_TOKEN", @@ -97,21 +73,28 @@ You must include the following top-level fields. In the `containerDefinitions` list, include one container with the following fields. -| Field name | Type | Description | -| ----------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- | -| `name` | string | The container name, which should be `acl-controller` | -| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | -| `command` | list | Must be set as shown. The startup command for the ACL controller. | -| `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. | -| `secrets` | list | Must have `CONSUL_HTTP_TOKEN` set to the ACL bootstrap token and `CONSUL_CACERT_PEM` set to the Consul server CA certificate. | -| `environment` | string | Must set the `CONSUL_HTTP_ADDR` environment variable to the address of the HTTP API of your Consul servers. | +| Field name | Type | Description | +| ------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name, which should be `acl-controller` | +| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | +| `command` | list | Should be set as shown. The startup command for the ACL controller. | +| `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. | +| `secrets` | list | Should be set as shown. Configures the secrets the ECS service will retrieve and set as environment variables in the `acl-controller` container. | +| `environment` | string | Must be set as shown. Configures environment variables that the ECS service will set in the `acl-controller` container. Must set the `CONSUL_HTTP_ADDR` environment variable to the HTTP(S) address of the Consul servers. | -The following CLI options are required in the `command` field of the container definition. +The following CLI options are available in the `command` field of the container definition. -| Flag | Type | Description | -| --------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------- | -| `-consul-client-secret-arn` | string | The secret where the ACL controller will store the Consul client token. | -| `-secret-name-prefix` | string | The [secret name prefix](/docs/ecs/manual/acl-controller#secret-name-prefix) that you chose for this ACL controller. | +| Flag | Type | Description | +| ------------------ | ------- | --------------------------------------------------------------------------------------------- | +| `-iam-role-path` | string | Specifies the path to IAM roles trusted by the AWS IAM auth method created by the controller. | +| `-log-level` | string | The log level for the ACL controller. Can be set to `DEBUG` for additional detail. | + +The following describes the entries to include in the `secrets` list. + +| Name | Description | +| ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | +| `CONSUL_HTTP_TOKEN` | Must be set to the secret containing the bootstrap ACL token. | +| `CONSUL_CACERT_PEM` | If applicable, should be set to the secret containing the Consul server CA certificate. This must not be set when using Consul servers in HCP. | ## ECS Service @@ -162,29 +145,17 @@ secrets. "ecs:DescribeTasks" ], "Resource": ["*"] - }, - { - "Effect": "Allow", - "Action": [ - "secretsmanager:GetSecretValue", - "secretsmanager:UpdateSecret" - ], - "Resource": [ - "arn:aws:secretsmanager:us-west-2:000000000000:secret:-*" - ] } ] } ``` -The following are the required permissions. You will need to substitute `` with your chosen [name prefix](/docs/ecs/manual/acl-controller#secret-name-prefix). +The following are the required permissions. -| Action | Resource | Description | -| ------------------------------- | ----------------------------------------------------------------- | ---------------------------------------------------------------------------------- | -| `ecs:ListTasks` | `*` | Allow the ACL controller to watch for new tasks. | -| `ecs:DescribeTasks` | `*` | Allow the ACL controller to retrieve details for new tasks. | -| `secretsmanager:GetSecretValue` | `arn:aws:secretsmanager:us-west-2:000000000000:secret:-*` | Allow the ACL controller to read secrets with a name prefix. | -| `secretsmanager:UpdateSecret` | `arn:aws:secretsmanager:us-west-2:000000000000:secret:-*` | Allow the ACL controller to store Consul ACL tokens in secrets with a name prefix. | +| Action | Resource | Description | +| --------------------- | --------- | ------------------------------------------------------------ | +| `ecs:ListTasks` | `*` | Allow the ACL controller to watch for new tasks. | +| `ecs:DescribeTasks` | `*` | Allow the ACL controller to retrieve details for new tasks. | ### Execution Role Policy @@ -205,9 +176,15 @@ The following example shows the policy needed for the execution role. ], "Resource": [ "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-bootstrap-token", - "arn:aws:secretsmanager:us-west-2:000000000000:secret:-consul-client-token" + "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-ca-cert" ] } ] } ``` + +The following are the required permissions. + +| Action | Resource | Description | +| ------------------------------- | ------------------------------------------------------------- | ---------------------------------------------------------------------- | +| `secretsmanager:GetSecretValue` | `arn:aws:secretsmanager:us-west-2:000000000000:secret:` | Allow ECS to retrieve this secret and inject the secret into the task. | diff --git a/website/content/docs/ecs/manual/install.mdx b/website/content/docs/ecs/manual/install.mdx index 7e20b3ea0..aac1b416b 100644 --- a/website/content/docs/ecs/manual/install.mdx +++ b/website/content/docs/ecs/manual/install.mdx @@ -25,7 +25,7 @@ Configure a task definition that creates the containers: - An Envoy sidecar-proxy container - A Consul client container - A `consul-ecs-mesh-init` container for service mesh setup -* (Optional) A `consul-ecs-health-sync` container to sync ECS health checks into Consul +- (Optional) A `consul-ecs-health-sync` container to sync ECS health checks into Consul ## Top-level fields @@ -42,10 +42,10 @@ during task startup. "networkMode": "awsvpc", "volumes": [ { - "name": "consul_data", + "name": "consul_data" }, { - "name": "consul_binary", + "name": "consul_binary" } ], "containerDefinitions": [...] @@ -71,7 +71,7 @@ during task startup. ### Task Tags -The `tags` list must include the following if you are using the ACL controller in a [secure configuration](/docs/manual/secure-configuration). +The `tags` list must include the following if you are using the ACL controller in a [secure configuration](/docs/ecs/manual/secure-configuration). Without these tags, the ACL controller will be unable to provision a service token for the task. | Tag Key | Tag Value | Description | @@ -369,7 +369,7 @@ configuration to a shared volume. | `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | | `mountPoints` | list | Must be set as show above, so the `consul` and `consul-ecs` binaries can be shared among containers for task setup. | | `command` | list | Set to `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. | -| `environment` | list | This must include the [`CONSUL_ECS_CONFIG_JSON`](/docs/ecs/manual-installation#consul_ecs_config_json) variable. See below for details. | +| `environment` | list | This must include the [`CONSUL_ECS_CONFIG_JSON`](/docs/ecs/manual/install#consul_ecs_config_json) variable. See below for details. | ### `CONSUL_ECS_CONFIG_JSON` @@ -405,7 +405,7 @@ the `consul-ecs-health-sync` container. | Field name | Type | Description | | ---------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | | `bootstrapDir` | string | This is the path of a shared volume that is mounted to other containers, where `consul-ecs-mesh-init` will write out Envoy configuration. | -| `healthSyncContainers` | list | Used for [health status syncing](/docs/ecs/manual-installation#consul-ecs-health-sync-container) from ECS to Consul. See below for details. | +| `healthSyncContainers` | list | Used for [health status syncing](/docs/ecs/architecture#ecs-health-check-syncing) from ECS to Consul. See below for details. | | `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. The `destinationName` and `localBindPort` fields are required. | | `service.name` | string | The name used to register this service into the Consul service catalog. | | `service.port` | integer | The port your application listens on. Set to `0` if your application does not listen on any port. | @@ -551,6 +551,6 @@ and `consul-ecs-mesh-init` containers. # Next Steps -* Create the task definition using the [AWS Console](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html) or the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/ecs/register-task-definition.html), or another method of your choice. +* Create the task definition using the [AWS Console](https://docs.aws.amazon.com/AmazonECS/latest/userguide/create-task-definition-classic.html) or the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/ecs/register-task-definition.html), or another method of your choice. * Create an [ECS Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) to start tasks using the task definition. * Follow the [Secure Configuration](/docs/ecs/manual/secure-configuration) to get production-ready. diff --git a/website/content/docs/ecs/manual/secure-configuration.mdx b/website/content/docs/ecs/manual/secure-configuration.mdx index c9a856043..1bb800c00 100644 --- a/website/content/docs/ecs/manual/secure-configuration.mdx +++ b/website/content/docs/ecs/manual/secure-configuration.mdx @@ -7,7 +7,7 @@ description: >- # Secure Configuration -This topic describes how to enable Consul security features for your production workloads. +This topic describes how to enable Consul security features for your production workloads. ## Prerequisites @@ -23,20 +23,84 @@ You should be familiar with [specifying sensitive data](https://docs.aws.amazon. You should be familiar with configuring Consul's secure features, including how to create ACL tokens and policies. Refer to the [ACL system documentation](/docs/security/acl) and [Day 1: Security tutorials](https://learn.hashicorp.com/collections/consul/security) for an introduction and additional information. -## ACL Tokens +## Auth Method -Tokens are artifacts within the ACL system that authenticate users, services, and Consul agents. Tokens are linked to policies that specify the resources the token bearer has access to when making requests in the network. +Tokens are artifacts within the ACL system that authenticate users, services, and Consul agents. Tokens are linked to policies that specify the resources the token bearer has access to when making requests in the network. -You must create two types of ACL tokens for Consul on ECS: +Auth Methods are a Consul server component that performs authentication against a trusted external party to authorize the creation of ACL tokens. The [AWS IAM auth method](/docs/security/acl/auth-methods/aws-iam) is used to enable an ECS task to automatically obtain ACL tokens when the task starts up. + +There are two types of ACL tokens for Consul on ECS: * **Client tokens:** used by the `consul-client` containers to join the Consul cluster * **Service tokens:** used by sidecar containers for service registration and health syncing -This section describes how to manually create ACL tokens. Alternatively, you can install the ACL controller to ease the burden of creating tokens. The ACL controller can automatically create ACL tokens for Consul on ECS. For additional details, refer to [ACL Controller](/docs/manual/acl-controller). +This section describes how to manually configure the AWS IAM auth method for Consul on ECS. Alternatively, you can install the ACL controller to ease the burden of creating these resources. The ACL controller can automatically configure ACL resources for Consul on ECS. For additional details, refer to [ACL Controller](/docs/manual/acl-controller) and [Architecture](/docs/ecs/architecture). -### Define policies +### ECS Task Role Configuration -Configure the following ACL policy for the Consul client token: +The ECS [task role](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html) +is an IAM role associated with an ECS task. + +When an ECS task starts up, it runs a `consul login` command. The `consul login` command obtains +credentials for the task role role from AWS. It uses those credentials to sign the login request to +the AWS IAM auth method. This proves the ECS task's identity to the Consul servers. + +The task role must be configured with the following details to compatible with the AWS IAM auth +method. + +* An `iam:GetRole` permission to fetch itself. See + [IAM Policies](/docs/security/acl/auth-methods/aws-iam#iam-policies) for details. +* A `consul.hashicorp.com.service-name` tag on the task role which contains the Consul service name + for the application in this task. +* A consul.hashicorp.com.namespace tag on the task role + indicating the Consul Enterprise namespace where this service is registering. + +The following sections describe how to configure the auth method to enable task roles to +successfully authenticate to the AWS IAM auth method and obtain tokens with the necessary +permissions. + +### Auth Method for Client Tokens + +The following steps configure an instance of the auth method that creates client tokens for tasks. + +1. Create the auth method instance +2. Create the client policy and role +3. Create the binding rule + +#### Create Auth Method for Client Tokens + +The following Consul CLI command creates an instance of the auth method for client tokens. + +```shell +consul acl auth-method create \ + -type aws-iam \ + -name iam-ecs-client-token \ + -description="AWS IAM auth method for ECS client tokens" \ + -config '{ + "BoundIAMPrincipalArns": ["arn:aws:iam:::role/consul-ecs/*"], + "EnableIAMEntityDetails": true +}' +``` + +The following flags are required: + +| Flag | Type | Description | +| --------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------- | +| `-type` | string | Must be `aws-iam`. | +| `-name` | string | A name of your choice. Must be unique among all auth methods. | +| `-description` | string | A description of your choice. | +| `-config` | string | A JSON string containing the [configuration](/docs/security/acl/auth-methods/aws-iam#config-parameters) for the the auth method. | + +In the `-config` option, the following fields are required: + +| Field | Type | Description | +| ------------------------- | ------- | --------------------------------------------------------------------------------------------------------- | +| `BoundIAMPrincipalArns` | list | The list of trusted IAM roles. We recommend using a wildcard to trust IAM roles at a particular path. | +| `EnableIAMEntityDetails` | boolean | Must be true so that the auth method can retrieve IAM role details, such as the role path and role tags. | + +#### Create Client Policy and Role + +Configure the following ACL policy for Consul client tokens: @@ -55,52 +119,167 @@ The policy allows `node:write` for any node name, which is necessary because the You can add the policy in Consul using the [`consul acl policy create`](/commands/acl/policy/create) command or the [`[PUT] /v1/acl/policy`](/api-docs/acl/policies#create-a-policy) API endpoint. -If you intend to create a gateway for connecting multiple Consul datacenters, you will need to configure a mesh gateway policy. If namespaces are enabled, the mesh gateway must run in the default namespace. +After the policy is created, create a Consul role associated with the policy by using the [`consul acl role create`](/commands/acl/role/create) command or the [`[PUT] /v1/acl/role`](/api-docs/acl/roles#create-a-role) API endpoint. - +The following example shows how to use the Consul CLI to create the client policy and role. + +```shell +consul acl policy create -name consul-ecs-client-policy -rules @client-token-policy.hcl + +consul acl role create -name consul-ecs-client-role -policy-name consul-ecs-client-policy +``` + +#### Create Binding Rule for Client Tokens + +The following creates a binding rule for the auth method for client tokens. The binding rule +associates the client role with each token created by a successful login to this auth +method instance. + +```shell +consul acl binding-rule create -method iam-ecs-client-token \ + -description 'Bind a role for Consul clients on ECS' \ + -bind-type role \ + -bind-name consul-ecs-client-role +``` + +### Auth Method for Service Tokens + + +The following steps configure an instance of the auth method that creates service tokens for tasks. + +* Create the auth method instance +* Create the binding rule + +#### Create Auth Method for Service Tokens + +The following uses the Consul CLI to create an instance of the auth method +for service tokens. This configures the auth method to associate a service identity +to each token created during login to this auth method instance. + +```shell +consul acl auth-method create \ + -type aws-iam \ + -name iam-ecs-service-token \ + -description="AWS IAM auth method for ECS service tokens" \ + -config '{ + "BoundIAMPrincipalArns": ["arn:aws:iam:::role/consul-ecs/*"], + "EnableIAMEntityDetails": true, + "IAMEntityTags": [ + "consul.hashicorp.com.service-name" + ] +}' +``` + +The following flags are required: + +| Flag | Type | Description | +| --------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------- | +| `-type` | string | Must be `aws-iam`. | +| `-name` | string | A name of your choice. Must be unique among all auth methods. | +| `-description` | string | A description of your choice. | +| `-config` | string | A JSON string containing the [configuration](/docs/security/acl/auth-methods/aws-iam#config-parameters) for the the auth method. | + +In the `-config` option, the following fields are required: + +| Field | Type | Description | +| --- | --- | --- | +| `BoundIAMPrincipalArns` | list | The list of trusted IAM roles. We recommend using a wildcard to trust IAM roles at a particular path. | +| `EnableIAMEntityDetails` | boolean | Must be true so that the auth method can retrieve IAM role details, such as the role path and role tags. | +| `IAMEntityTags` | list | The list of IAM role tags to make available to binding rules. Must include the service name tag as shown. | + +The following binding rule is used to associate a service identity with each token created by +successful login to this instance of the auth method. The service identity name is taken from the +`consul.hashicorp.com.service-name` tag from the authenticaing IAM role identity. + +#### Create Binding Rule + +```shell +consul acl binding-rule create \ + -method iam-ecs-service-token \ + -description 'Bind a service identity from IAM role tags for ECS service tokens' \ + -bind-type service \ + -bind-name '${entity_tags.consul.hashicorp.com.service-name}' +``` + +### Configuration for Consul Enterprise + +When using Consul Enterprise namespaces and admin partitions, pass the `-partition ` +option to the Consul CLI when creating Consul ACL roles, policies, auth methods, and binding rules, +in order to create these resources in a particular partition. + +The following shows how to create the ACL policy for client tokens. This ensures permissions for the +client token are scoped to a particular partition. + + ```hcl -namespace "default" { ## If namespaces enabled - service "" { - policy = "write" - } -} -namespace_prefix "" { ## If namespaces enabled +partition "" { + node_prefix "" { + policy = "write" + } + namespace_prefix "" { service_prefix "" { - policy = "read" + policy = "read" } - node_prefix "" { - policy = "read" - } -} -agent_prefix "" { - policy = "read" + } } ``` -### Create service tokens +The following commands show how to use the Consul CLI to create the policy and role. -Create the Consul client token and the service tokens after adding the necessary policies. Service tokens should be associated with a service identity. The service identity includes `service:write` permissions for the service and sidecar proxy. +```shell +consul acl policy create -partition \ + -name consul-ecs-client-policy \ + -rules @client-token-policy-ent.hcl -You can create tokens using the [`consul acl token create`](/commands/acl/token/create) command or the [`[PUT] /v1/acl/token`](/api-docs/acl/tokens#create-a-token) API endpoint. -The following example shows how to use the Consul CLI to create a service token for a service named `example-client-app`: - -```shell-session -$ consul acl token create -service-identity=example-client-app ... +consul acl role create \ + -partition \ + -name consul-ecs-client-role \ + -policy-name consul-ecs-client-policy ``` -You need to create one service token for each registered Consul service in ECS. When you add new services to the service mesh, you must create new tokens for each service. +The auth method for *service tokens* requires the following additional configuration to include a +namespace binding rule. This ensures the service tokens are created in the right namespace during +login. (The namespace binding rule must not be configured on the instance of the auth method +instance for *client tokens*.) + + + +```shell +consul acl auth-method create \ + -partition \ + -type aws-iam \ + -name iam-ecs-service-token \ + -description="AWS IAM auth method for ECS service tokens" \ + -namespace-rule-selector 'entity_tags["consul.hashicorp.com.namespace"] != ""' \ + -namespace-rule-bind-namespace '${entity_tags.consul.hashicorp.com.namespace}' \ + -config '{ + "BoundIAMPrincipalArns": ["arn:aws:iam:::role/consul-ecs/*"], + "EnableIAMEntityDetails": true, + "IAMEntityTags": [ + "consul.hashicorp.com.service-name", + "consul.hashicorp.com.namespace" + ] +}' +``` + + + +| Field | Type | Description | +| -------------------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `-partition` | string | The Consul Enterprise admin partition in which the auth method is created. | +| `-namespace-rule-selector` | string | When this expression evaluates to true during login, the `-namespace-rule-bind-namespace` is applied. As shown, it evaluates to true when the `consul.hashicorp.com.namespace` tag is non-empty on the task IAM role. | +| `-namespace-rule-bind-namespace` | string | This expression is evaluted to determine the namespace where the token is created during login. As shown, it uses the namespace from the `consul.hashicorp.com.namespace` tag on the task IAM role. | +| `IAMEntityTags` | list | Must include `consul.hashicorp.com.namespace` to enable use of this tag in binding rules. | ## Secret storage You should securely store the following secrets in order to make them available to ECS tasks. -1. Consul Server CA certificate +1. Consul Server CA certificates. More than one may be required for different Consul protocols. 2. Consul gossip encryption key -3. Consul client ACL token -4. Consul service ACL tokens (one per service) These secrets can be securely stored and passed to ECS tasks using either of the following AWS secret services: @@ -110,20 +289,18 @@ These secrets can be securely stored and passed to ECS tasks using either of the Once the secrets are stored they can be referenced using their ARN. The following shows example secret ARNs when using AWS Secrets Manager: -| Secret | Sample Secret ARN | -| ---------------------- | ---------------------------------------------------------------------------------- | -| Consul Server CA Cert | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-ca-cert` | -| Gossip encryption key | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-gossip-key` | -| Client token | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-client-token` | -| Service token | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-example-client-app-token` | +| Secret | Sample Secret ARN | +| ---------------------- | ---------------------------------------------------------------------------------- | +| Consul Server CA Cert for RPC | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-ca-cert` | +| Consul Server CA Cert for HTTPS | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-https-ca-cert` | +| Gossip encryption key | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-gossip-key` | ## Configure `consul-client` The following secrets must be passed to the `consul-client` container: -* Consul server CA certificate +* Consul server CA certificates * Gossip encryption key -* Consul client ACL token The following example shows how to include these secrets in the task definition. The `secrets` list specifies environment variable `name`s that will be set to the secret values for this container. @@ -137,16 +314,16 @@ ECS automatically fetches the secret values specified in the `valueFrom` fields "image": "public.ecr.aws/hashicorp/consul:", "secrets": [ { - "name": "CONSUL_CACERT", + "name": "CONSUL_CACERT_PEM", "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-ca-cert" }, + { + "name": "CONSUL_HTTPS_CACERT_PEM", + "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-https-ca-cert" + }, { "name": "CONSUL_GOSSIP_ENCRYPTION_KEY", "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-gossip-key" - }, - { - "name": "AGENT_TOKEN", - "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-client-token" } ] }, @@ -155,19 +332,76 @@ ECS automatically fetches the secret values specified in the `valueFrom` fields } ``` -Next, update Consul configuration options to pass the secrets to the Consul client. +Next, update the Consul client startup script with the `consul login` command and additional Consul configuration +options for a secure configuration. The following is an example of the *additional* content to include in the `consul-client` startup script. Refer to the [install page](/docs/ecs/manual/install#consul-client-container) for the remainder of the startup script and how to pass this script to the container. - + ```shell ... -# Write the CA Cert to a file -echo "$CONSUL_CACERT" > /tmp/consul-ca-cert.pem +# Obtain details from the task metadata +ECS_TASK_META=$(curl -s $ECS_CONTAINER_METADATA_URI_V4/task) +TASK_REGION=$(echo "$ECS_TASK_META" | jq -r .TaskARN | cut -d ':' -f 4) +TASK_ID=$(echo "$ECS_TASK_META" | jq -r .TaskARN | cut -d '/' -f 3) +CLUSTER_ARN=$(echo "$ECS_TASK_META" | jq -r .TaskARN | sed -E 's|:task/([^/]+).*|:cluster/\1|') + +# Write the CA certs to a files in the shared volume +echo "$CONSUL_CACERT_PEM" > /consul/consul-ca-cert.pem +echo "$CONSUL_HTTPS_CACERT_PEM" > /consul/consul-https-ca-cert.pem + +consul_login() { + echo "Logging into auth method" + consul login \ + -http-addr "" \ + -ca-file /consul/consul-https-ca-cert.pem \ + -partition "" \ + -type aws-iam \ + -method iam-ecs-client-token \ + -meta "consul.hashicorp.com/task-id=$TASK_ID" \ + -meta "consul.hashicorp.com/cluster=$CLUSTER_ARN" \ + -aws-region "$TASK_REGION" \ + -aws-auto-bearer-token -aws-include-entity \ + -token-sink-file /consul/client-token +} + +read_token_stale() { + consul acl token read -http-addr ${ consul_http_addr } \ + -ca-file /consul/consul-https-ca-cert.pem \ + -stale -self -token-file /consul/client-token \ + > /dev/null +} + +# Retry in order to login successfully. +while ! consul_login; do + sleep 2 +done + +# Allow the health-sync container to read this token for consul logout. +chmod 0644 /consul/client-token + +# Wait for raft replication to hopefully occur. Without this, an "ACL not found" may be cached for a while. +COUNT=20 +while [ "$COUNT" -gt 0 ]; do + echo "Checking that the ACL token exists when reading it in the stale consistency mode ($COUNT attempts remaining)" + if read_token_stale; then + echo "Successfully read ACL token from the server" + break + fi + sleep 0.1 + COUNT=$((COUNT - 1)) +done +if [ "$COUNT" -eq 0 ]; then + echo "Unable to read ACL token from a Consul server; please check that your server cluster is healthy" + exit 1 +fi + +# This is interpolated into the agent-defaults.hcl +export AGENT_TOKEN=$(cat /consul/client-token) # Write the Consul agent configuration file. cat << EOF > /consul/agent-defaults.hcl @@ -183,7 +417,7 @@ auto_encrypt = { } tls { defaults { - ca_file = "/tmp/consul-ca-cert.pem" + ca_file = "/consul/consul-ca-cert.pem" verify_outgoing = true } } @@ -198,46 +432,75 @@ acl { } } +partition = "" + EOF ``` +The following describes the additional steps added to the startup script: + +* Fetch additional details from the task metadata: the AWS region, task id, and cluster arn. + These details are necessary for the `consul login` command used to obtain a Consul client token. +* Write CA certificates to files for Consul CLI and Consul client +* Run the `consul login` command in a retry loop. +* Wait for Raft replication to hopefully occur for this token. +* Configure the Consul client config file with additional fields necessary for secure configuration. + +The following flags are passed to the `consul login` command: + +| Field name | Type | Description | +| ------------------------------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------ | +| [`-http-addr`](/commands/login#http-addr) | string | HTTP(S) address of the Consul server. | +| [`-ca-file`](/commands/login#ca-file) | string | Path of the CA cert for Consul's HTTPS interface. Not required when using Consul servers on HCP. | +| `-partition` | string | The Consul Enterprise admin partition the auth method belongs to. | +| [`-type`](/commands/login#type) | string | The auth method type. Must be `aws-iam`. | +| [`-method`](/commands/login#type) | string | The auth method name. Must be the name of the auth method for ECS client tokens. | +| [`-meta`](/commands/login#meta) | string | Metadata to set in description of the created token. Should include the task id and cluster as shown. | +| `-aws-region` | string | The AWS region where the task is running. | +| `-aws-auto-bearer-token` | | Must be set to login to the AWS IAM auth method. | +| `-aws-include-entity` | | Must be set to enable IAM role details. | + The following table describes the additional fields that must be included in the Consul client configuration file. -| Field name | Type | Description | -| -------------------------------------------------------------------------------| ------- | ------------------------------------------------------------------------------------ | -| [`encrypt`](/docs/agent/config/cli-flags#_encrypt) | string | Specifies the gossip encryption key | -| [`tls.defaults.ca_file`](/docs/agent/config/config-files#tls_defaults_ca_file) | string | Specifies the Consul server CA cert for TLS verification. | -| [`acl.enabled`](/docs/agent/config/config-files#acl_enabled) | boolean | Enable ACLs for this agent. | -| [`acl.tokens.agent`](/docs/agent/config/config-files#acl_tokens_agent) | string | Specifies the Consul client token which authorizes this agent with Consul servers. | +| Field name | Type | Description | +| ------------------------------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------ | +| [`encrypt`](/docs/agent/config/cli-flags#_encrypt) | string | Gossip encryption key | +| [`tls.defaults.ca_file`](/docs/agent/config/config-files#tls_defaults_ca_file) | string | Consul server CA cert for TLS verification. | +| [`acl.enabled`](/docs/agent/config/config-files#acl_enabled) | boolean | Enable ACLs for this agent. | +| [`acl.tokens.agent`](/docs/agent/config/config-files#acl_tokens_agent) | string | Consul client token which authorizes this agent with Consul servers. | +| [`partition`](/docs/agent/config/config-files#partition-1) | string | The Consul Enterprise admin partition this agent belongs to. | ## Configure `consul-ecs-mesh-init` and `consul-ecs-health-sync` -Both `consul-ecs-mesh-init` and `consul-ecs-health-sync` containers need to be configured with -the service ACL token. This allows these containers to make HTTP API requests to the local -Consul client for service registration and health syncing. +The following *additional* options should be set in the [`CONSUL_ECS_CONFIG_JSON`](/docs/ecs/manual/install#consul_ecs_config_json) environment variable. When these options are specified, the `consul-ecs mesh-init` command will run the `consul login` command to obtain a service token from the Consul AWS IAM Auth method. The `consul-ecs health-sync` command is responsible for running a `consul logout` command for both the service and client tokens when the task shuts down. -The following shows how to set the `CONSUL_HTTP_TOKEN` variable to the service token for the `example-client-app` service, -if the token is stored in AWS Secrets Manager. - - + ```json { - "containerDefinitions": [ - { - "secrets": [ - { - "name": "CONSUL_HTTP_TOKEN", - "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-example-client-app-token" - } - ] - }, - ... - ], + "consulHTTPAddr": "", + "consulCACertFile": "/consul/consul-https-ca-cert.pem", + "consulLogin": { + "enabled": true, + "method": "iam-ecs-service-token", + "extraLoginFlags": ["-partition", ""] + }, + "bootstrapDir": "/consul", + "healthSyncContainers": [], ... } ``` + +The following table explains the additional fields for the `CONSUL_ECS_CONFIG_JSON`: + +| Field name | Type | Description | +| ----------------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `consulHTTPAddr` | string | HTTP(S) address for the Consul server. | +| `consulCACertFile` | string | Path of the CA cert file for Consul's HTTPS interface. Not required for Consul servers in HCP. | +| `consulLogin.enabled` | boolean | Must be set to true to log in to the auth method. | +| `consulLogin.method` | string | Must be set to the name of the auth method instance for service tokens. | +| `consulLogin.extraLoginFlags` | list | Additional flags passed to the `consul login` command. This shows how to pass the Consul Enterprise admin partition to the `consul login` command. | diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index c523cc0b8..2d520e418 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -7,7 +7,7 @@ description: >- # Installation with Terraform -This topic describes how to use HashiCorp’s Terraform modules to launch your application in AWS ECS as part of Consul service mesh. If you do not use Terraform, refer to the [Manual Installation](/docs/ecs/manual-installation) page to install Consul on ECS without Terraform. +This topic describes how to use HashiCorp’s Terraform modules to launch your application in AWS ECS as part of Consul service mesh. If you do not use Terraform, refer to the [Manual Installation](/docs/ecs/manual/install) page to install Consul on ECS without Terraform. This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the guides in the [Getting Started](/docs/ecs#getting-started) section for complete and runnable examples. @@ -28,15 +28,15 @@ If you want to operate Consul in production environments, follow the instruction * You should have some familiarity with using Terraform. Refer to the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform. * You should also be familiar with AWS ECS before following these instructions. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for details. -* If you intend to [use the `gateway-task` module to deploy mesh gateways](#configure-the-gateway-task-module), all Consul server and client agents in all datacenters must have TLS and gossip encryption enabled. Refer to the [Secure Configuration](/docs/ecs/terraform/secure-configuration) documentation for instructions. +* If you intend to [use the `gateway-task` module to deploy mesh gateways](#configure-the-gateway-task-module), all Consul server and client agents in all datacenters must have TLS and gossip encryption enabled. Refer to the [Secure Configuration](/docs/ecs/terraform/secure-configuration) documentation for instructions. ## Create the task definition -To run an application in ECS with Consul service mesh, you must create an ECS task definition. The task definition includes your application containers and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container. +To run an application in ECS with Consul service mesh, you must create an ECS task definition. The task definition includes your application containers and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container. -Create a Terraform configuration file and include the `mesh-task` module. The module automatically adds the necessary sidecar containers. +Create a Terraform configuration file and include the `mesh-task` module. The module automatically adds the necessary sidecar containers. -If you intend to peer the service mesh to multiple Consul datacenters or partitions, you can also include the `gateway-task` module. The module enables connectivity between datacenters across service meshes. +If you intend to peer the service mesh to multiple Consul datacenters or partitions, you can also include the `gateway-task` module. The module enables connectivity between datacenters across service meshes. ### Configure the mesh task module @@ -88,7 +88,7 @@ The following fields are required. Refer to the [module reference documentation] | `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. | | `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. | | `port` | integer | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | -| `retry_join` | list | This is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | +| `retry_join` | list | This is the [`retry_join`](/docs/agent/options#retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | ### Configure an ECS service for the mesh task module [ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) are one of the most common @@ -126,7 +126,7 @@ The example shows a partially configured ECS service to highlight significant fi | `propagate_tags` | string | This option must be set to `TASK_DEFINITION` so that tags added by `mesh-task` to the task definition are copied to tasks. | After including the ECS service in your Terraform configuration, run `terraform apply` -from your project directory to create the ECS service resource. The ECS service +from your project directory to create the ECS service resource. The ECS service then starts your application in a task. The task automatically registers itself into the Consul service catalog during startup. @@ -411,7 +411,7 @@ module "web" { ## Configure the bind address To ensure that your application only receives traffic through the service mesh, -you must change the address that your application listens on to the loopback address. The loopback address is also called `localhost`, `lo`, and `127.0.0.1`. +you must change the address that your application listens on to the loopback address. The loopback address is also called `localhost`, `lo`, and `127.0.0.1`. Binding to the loopback address allows the sidecar proxy running in the same task to only make requests within the service mesh. If your application is listening on all interfaces, such as `0.0.0.0`, then other diff --git a/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx b/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx index 7281a7a11..ff6f52dd2 100644 --- a/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx +++ b/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx @@ -97,7 +97,7 @@ module "my_task" { The main differences are: -- You must remove the `execution_role_arn` and `task_role_arn` fields. The `mesh-task` module will create the task and execution roles. +- You should remove the `execution_role_arn` and `task_role_arn` fields. The `mesh-task` module creates the task and execution roles by default. If you need to use existing IAM role(s), set the `task_role` and `execution_role` fields to pass in existing roles. - You must set the `port` field to the port that your application listens on. If your application has no listening port, set `outbound_only = true` and remove the `port` field. - You must add the `retry_join` field. This specifies the location of your Consul servers so that your task can join the mesh. @@ -111,5 +111,5 @@ resource. Now that your task(s) are migrated to the `mesh-task` module, -- Start at the [ECS Service section](/docs/ecs/terraform/install#ecs-service) of the Installation Guide to continue installing Consul on ECS. +- Start at the [ECS Service section](/docs/ecs/terraform/install#configure-an-ecs-service-for-the-mesh-task-module) of the Installation Guide to continue installing Consul on ECS. - Refer to the [`mesh-task` reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs) for all available inputs to your mesh tasks. diff --git a/website/content/docs/ecs/terraform/secure-configuration.mdx b/website/content/docs/ecs/terraform/secure-configuration.mdx index 3a7f60e74..5d61b18a7 100644 --- a/website/content/docs/ecs/terraform/secure-configuration.mdx +++ b/website/content/docs/ecs/terraform/secure-configuration.mdx @@ -7,11 +7,11 @@ description: >- # Secure Configuration -This topic describes how to enable Consul security features for your production workloads. +This topic describes how to enable Consul security features for your production workloads. ## Overview -To enable security in your production workloads, you must deploy the [ACL controller](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller), which provisions tokens for other service mesh tasks. Refer to [Automatic ACL Token Provisioning](/docs/ecs/architecture#automatic-acl-token-provisioning) to learn more about the ACL controller. +To enable security in your production workloads, you must deploy the [ACL controller](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller), which provisions tokens for other service mesh tasks. Refer to [Architecture](/docs/ecs/architecture#acl-controller) to learn more about the ACL controller. The controller cannot provision tokens for itself, so you must create the token for the ACL controller. The following steps describe the overall process of enabling security features for your production workloads: @@ -24,16 +24,36 @@ The controller cannot provision tokens for itself, so you must create the token ## Prerequisites -Implement the following security features for your Consul server clusters before applying them to your workloads: +Implement the following security features for your Consul server clusters before applying them to your workloads: 1. [TLS encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. 1. [Gossip encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. 1. [Access control lists (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. +## Auth Method + +Consul on ECS uses the [AWS IAM Auth Method](/docs/ecs/architecture#aws-iam-auth-method) to enable +tasks to automatically obtain Consul ACL tokens during startup. + +With the Terraform modules for Consul on ECS, the auth method is supported by default when ACLs are +enabled. The ACL controller sets up the auth method on the Consul servers. The `mesh-task` module +configures the ECS task definition to be compatible with the auth method. + +A unique task IAM role is required for each ECS task family. Task IAM roles must not be shared by +different task families. This is because the task family represents only one Consul service and the +task IAM role must encode the Consul service name. + +By default, the `mesh-task` module will create and configure the task IAM role for you. + +-> **NOTE**: When passing an existing IAM role to the `mesh-task` module using the `task_role` input +variable, you must configure the IAM role as described in [ECS Task Role +Configuration](/docs/ecs/manual/secure-configuration#ecs-task-role-configuration) to be compatible with +the AWS IAM auth method. + ## ACL controller -1. Create a policy that grants `acl:write` and `operator:write` access for the controller. Refer to the [ACL policies documentation](/docs/security/acl/policies) for instructions. -1. Create a token and link it to the ACL controller policy. Refer to the [ACL tokens documentation](/docs/security/acl/tokens) for instructions. +1. Create a policy that grants `acl:write` and `operator:write` access for the controller. Refer to the [ACL policies documentation](/docs/security/acl/acl-policies) for instructions. +1. Create a token and link it to the ACL controller policy. Refer to the [ACL tokens documentation](/docs/security/acl/acl-tokens) for instructions. 1. Create a Secrets Manager secret containing the ACL controller's token and a Secrets Manager secret containing the Consul CA cert. ```hcl @@ -58,27 +78,45 @@ Implement the following security features for your Consul server clusters before 1. Use the [`acl-controller` terraform module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller?tab=inputs) to deploy the controller. You must provide the ARN's for the token and CA cert in the `consul_bootstrap_token_secret_arn` and `consul_server_ca_cert_arn` fields, respectively. - ```hcl - module "acl_controller" { - source = "hashicorp/consul/aws-ecs//modules/acl-controller" - consul_bootstrap_token_secret_arn = aws_secretsmanager_secret.bootstrap_token.arn - consul_server_http_addr = "https://consul-server.example.com:8501" - consul_server_ca_cert_arn = aws_secretsmanager_secret.ca_cert.arn - ecs_cluster_arn = "arn:aws:ecs:my-region:111111111111:cluster/consul-ecs" - region = "my-region" - subnets = ["subnet-abcdef123456789"] - name_prefix = "consul-ecs" - } - ``` + ```hcl + module "acl_controller" { + source = "hashicorp/consul/aws-ecs//modules/acl-controller" + version = "" -The `name_prefix` parameter is used to prefix any secrets that the ACL controller will -update in AWS Secrets Manager. The `name_prefix` parameter value must be unique for each ECS cluster where you deploy this controller. + consul_bootstrap_token_secret_arn = aws_secretsmanager_secret.bootstrap_token.arn + consul_server_http_addr = "https://consul-server.example.com:8501" + consul_server_ca_cert_arn = aws_secretsmanager_secret.ca_cert.arn + ecs_cluster_arn = "arn:aws:ecs:us-east-1:111111111111:cluster/consul-ecs" + region = "us-east-1" + subnets = ["subnet-abcdef123456789"] + name_prefix = "consul-ecs" + } + ``` + +The following table describes the required input variables for the `acl-controller` module. + +| Input Variable | Type | Description | +| ----------------------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | +| `source` | string | The location of the `acl-controller` module source. | +| `version` | string | The version of the `acl-controller` module. | +| `consul_bootstrap_token_secret_arn` | string | The Secrets Manager secret containing an ACL token for the ACL controller. The ACL token must have `acl:write` and `operator:write` permissions. | +| `consul_server_http_addr` | string | The HTTP(S) address of the Consul servers. | +| `consul_server_ca_cert_arn` | string | (Optional) The Secrets Manager secret containing the CA cert for HTTPS communication with Consul servers. Required if the server's certificate is self-signed or signed by an internal CA. This is not required for Consul servers in HCP. | +| `ecs_cluster_arn` | string | The ECS cluster where the ACL controller will be deployed. | +| `region` | string | The AWS region where the AWS resources will be created. | +| `subnets` | list | The AWS VPC subnet ids where the ACL controller will be deployed. | +| `name_prefix` | string | AWS resources created by the `acl-controller` module will include this prefix in the resource name. | + + +If you are using Consul Enterprise, see Admin Partitions and Namespaces for +additional configuration required to support Consul Enterprise on ECS. + ## Deploy your services Follow the instructions described in [Create a task definition](/docs/ecs/terraform/install#create-the-task-definition) to create the basic configuration for the task module. Add the following additional configurations to make the configuration production-ready. -### Create an AWS Secrets Manager secret +### Create an AWS Secrets Manager secret The secret stores the gossip encryption key that the Consul clients use. @@ -103,21 +141,32 @@ should be the same as the `name_prefix` you provide to the ACL controller module ```hcl module "my_task" { - source = "hashicorp/consul/aws-ecs//modules/mesh-task" - family = "my_task" + source = "hashicorp/consul/aws-ecs//modules/mesh-task" + version = "" ... - tls = true - consul_server_ca_cert_arn = aws_secretsmanager_secret.ca_cert.arn - gossip_key_secret_arn = aws_secretsmanager_secret.gossip_key.arn + tls = true + consul_server_ca_cert_arn = aws_secretsmanager_secret.ca_cert.arn + gossip_key_secret_arn = aws_secretsmanager_secret.gossip_key.arn - acls = true - consul_client_token_secret_arn = module.acl_controller.client_token_secret_arn - acl_secret_name_prefix = "consul-ecs" + acls = true + consul_http_addr = "https://consul-server.example.com:8501" + consul_https_ca_cert_arn = aws_secretsmanager_secret.ca_cert.arn } ``` +The following table explains the `mesh-task` input variables relevant to a secure configuration: + +| Input Variable | Type | Description | +| ----------------------- | -------- | ------------------------------------------------------------------------------------------------------------------- | +| `tls` | boolean | If true, TLS is enabled for RPC communication with the Consul servers. | +| `consul_server_ca_cert_arn` | string | The Secrets Manager secret containing the CA certificate for RPC communication with the Consul servers when TLS is enabled. | +| `gossip_key_secret_arn` | string | The Secrets Manager secret containing Consul's gossip encryption key. | +| `acls` | boolean | If true, ACLs are enabled. | +| `consul_http_addr` | string | The Consul server address. Required when `acls = true` in order to log in to Consul's AWS IAM auth method to obtain ACL tokens. | +| `consul_https_ca_cert_arn` | string | (optional) The Secrets Manager secret containing the CA cert for HTTPS communication with Consul servers. Required if the server's certificate is self-signed or signed by an internal CA. This is not required for Consul servers in HCP. | + Complete the following steps described in the Installation with Terraform chapter to deploy and connect your services: 1. [Run Terraform](/docs/ecs/terraform/install#run-terraform) diff --git a/website/public/img/consul-ecs-arch.png b/website/public/img/consul-ecs-arch.png index ae43ab9b4..7f6285894 100644 --- a/website/public/img/consul-ecs-arch.png +++ b/website/public/img/consul-ecs-arch.png @@ -1,3 +1,3 @@ version https://git-lfs.github.com/spec/v1 -oid sha256:96298e2c4297f01f39c68e1b6c5f79e136efc4a297265a294a0ee44edd1f210f -size 77203 +oid sha256:2ab7c3840e55e3de875e1822d5f43b4a03882ad404c38bd38b2ab52bd2b365f2 +size 156481 From 49856e8cec80052dcb00a2e3ef87d62c66fafdbd Mon Sep 17 00:00:00 2001 From: Frank DiRocco <160608+fdr2@users.noreply.github.com> Date: Fri, 24 Jun 2022 01:10:44 -0400 Subject: [PATCH 141/149] update terraform module location for consul aws modules (#13522) Co-authored-by: Paul Glass --- website/content/docs/ecs/enterprise.mdx | 4 ++-- website/content/docs/ecs/terraform/migrate-existing-tasks.mdx | 2 +- website/content/docs/ecs/terraform/secure-configuration.mdx | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/website/content/docs/ecs/enterprise.mdx b/website/content/docs/ecs/enterprise.mdx index 40344f499..9350fc6db 100644 --- a/website/content/docs/ecs/enterprise.mdx +++ b/website/content/docs/ecs/enterprise.mdx @@ -84,7 +84,7 @@ is not provided when `consul_partitions_enabled = true`, will default to the `de ```hcl module "acl_controller" { - source = "hashicorp/consul/aws-ecs//modules/acl-controller" + source = "hashicorp/consul/aws//modules/acl-controller" ... @@ -109,7 +109,7 @@ The following example demonstrates how to create a `mesh-task` assigned to the a ```hcl module "my_task" { - source = "hashicorp/consul/aws-ecs//modules/mesh-task" + source = "hashicorp/consul/aws//modules/mesh-task" family = "my_task" ... diff --git a/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx b/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx index ff6f52dd2..574e1fda0 100644 --- a/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx +++ b/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx @@ -68,7 +68,7 @@ The `mesh-task` module is used as follows: ```hcl module "my_task" { - source = "hashicorp/consul/aws-ecs//modules/mesh-task" + source = "hashicorp/consul/aws//modules/mesh-task" version = "" family = "my_task" diff --git a/website/content/docs/ecs/terraform/secure-configuration.mdx b/website/content/docs/ecs/terraform/secure-configuration.mdx index 5d61b18a7..24e3e165c 100644 --- a/website/content/docs/ecs/terraform/secure-configuration.mdx +++ b/website/content/docs/ecs/terraform/secure-configuration.mdx @@ -80,7 +80,7 @@ the AWS IAM auth method. ```hcl module "acl_controller" { - source = "hashicorp/consul/aws-ecs//modules/acl-controller" + source = "hashicorp/consul/aws//modules/acl-controller" version = "" consul_bootstrap_token_secret_arn = aws_secretsmanager_secret.bootstrap_token.arn @@ -141,7 +141,7 @@ should be the same as the `name_prefix` you provide to the ACL controller module ```hcl module "my_task" { - source = "hashicorp/consul/aws-ecs//modules/mesh-task" + source = "hashicorp/consul/aws//modules/mesh-task" version = "" ... From 91b8bf4b5547b1f8d0095fb80d94706f39a24b2c Mon Sep 17 00:00:00 2001 From: Matt Keeler Date: Fri, 24 Jun 2022 09:58:31 -0400 Subject: [PATCH 142/149] Clarify the wording of the peering limitations in the preview (#13590) --- website/content/docs/connect/cluster-peering/index.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/content/docs/connect/cluster-peering/index.mdx b/website/content/docs/connect/cluster-peering/index.mdx index 73980ceb2..1d2366632 100644 --- a/website/content/docs/connect/cluster-peering/index.mdx +++ b/website/content/docs/connect/cluster-peering/index.mdx @@ -40,7 +40,7 @@ Not all features and functionality are available in the technical preview releas - Consul ACLs must be disabled or the ACL `default_policy` must be set to `allow`. - Mesh gateways for _server to server traffic_ are not available. However, mesh gateways for _service to service traffic_ between clusters are available. -- Services exported to peered clusters must not be configured as HTTP. +- Services exported to peered clusters must be configured to use the TCP protcol (not HTTP, HTTP 2 and gRPC). - Support for dynamic routing such as splits, custom routes, or redirects is not available. - The `consul intention CLI` command is not supported. To manage intentions that specify services in peered clusters, use [configuration entries](/docs/connect/config-entries/service-intentions). - [L7 permissions](/docs/connect/l7-traffic) are not supported. From 988919a5813596180e8df43ef844b927cb911091 Mon Sep 17 00:00:00 2001 From: "R.B. Boyer" <4903+rboyer@users.noreply.github.com> Date: Fri, 24 Jun 2022 10:26:17 -0500 Subject: [PATCH 143/149] tests: ensure integration tests show logs from the containers to help debugging (#13593) --- GNUmakefile | 4 +- test/integration/consul-container/go.mod | 1 + test/integration/consul-container/go.sum | 1 + .../libs/node/consul-container.go | 133 +++++++++++++----- .../consul-container/libs/node/log.go | 23 +++ .../consul-container/libs/node/node.go | 19 ++- .../consul-container/metrics/leader_test.go | 4 +- .../upgrade/healthcheck_test.go | 13 +- 8 files changed, 143 insertions(+), 55 deletions(-) create mode 100644 test/integration/consul-container/libs/node/log.go diff --git a/GNUmakefile b/GNUmakefile index 197a3186d..7fa86d234 100644 --- a/GNUmakefile +++ b/GNUmakefile @@ -333,12 +333,12 @@ ifeq ("$(GOTAGS)","") @docker tag consul-dev:latest consul:local @docker run --rm -t consul:local consul version @cd ./test/integration/consul-container && \ - go test -v -timeout=30m ./upgrade --target-version local --latest-version latest + go test -v -timeout=30m ./... --target-version local --latest-version latest else @docker tag consul-dev:latest hashicorp/consul-enterprise:local @docker run --rm -t hashicorp/consul-enterprise:local consul version @cd ./test/integration/consul-container && \ - go test -v -timeout=30m ./upgrade --tags $(GOTAGS) --target-version local --latest-version latest + go test -v -timeout=30m ./... --tags $(GOTAGS) --target-version local --latest-version latest endif .PHONY: test-metrics-integ diff --git a/test/integration/consul-container/go.mod b/test/integration/consul-container/go.mod index 4dea73bba..182ff2e3a 100644 --- a/test/integration/consul-container/go.mod +++ b/test/integration/consul-container/go.mod @@ -7,6 +7,7 @@ require ( github.com/hashicorp/consul/api v1.11.0 github.com/hashicorp/consul/sdk v0.8.0 github.com/hashicorp/go-uuid v1.0.2 + github.com/hashicorp/hcl v1.0.0 github.com/stretchr/testify v1.7.0 github.com/testcontainers/testcontainers-go v0.13.0 ) diff --git a/test/integration/consul-container/go.sum b/test/integration/consul-container/go.sum index 4bc58fd5c..1ec317a43 100644 --- a/test/integration/consul-container/go.sum +++ b/test/integration/consul-container/go.sum @@ -427,6 +427,7 @@ github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8= github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc= github.com/hashicorp/golang-lru v0.5.4/go.mod h1:iADmTwqILo4mZ8BN3D2Q6+9jd8WM5uGBxy+E8yxSoD4= +github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4= github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ= github.com/hashicorp/logutils v1.0.0/go.mod h1:QIAnNjmIWmVIIkWDTG1z5v++HQmx9WQRO+LraFDTW64= github.com/hashicorp/mdns v1.0.4/go.mod h1:mtBihi+LeNXGtG8L9dX59gAEa12BDtBQSp4v/YAJqrc= diff --git a/test/integration/consul-container/libs/node/consul-container.go b/test/integration/consul-container/libs/node/consul-container.go index 598724170..caa8a7409 100644 --- a/test/integration/consul-container/libs/node/consul-container.go +++ b/test/integration/consul-container/libs/node/consul-container.go @@ -9,6 +9,7 @@ import ( "github.com/docker/docker/pkg/ioutils" "github.com/hashicorp/consul/api" + "github.com/hashicorp/hcl" "github.com/testcontainers/testcontainers-go" "github.com/testcontainers/testcontainers-go/wait" @@ -48,7 +49,6 @@ func newConsulContainerWithReq(ctx context.Context, req testcontainers.Container // NewConsulContainer starts a Consul node in a container with the given config. func NewConsulContainer(ctx context.Context, config Config) (Node, error) { - license, err := readLicense() if err != nil { return nil, err @@ -64,30 +64,29 @@ func NewConsulContainer(ctx context.Context, config Config) (Node, error) { return nil, err } + pc, err := readSomeConfigFileFields(config.HCL) + if err != nil { + return nil, err + } + configFile, err := createConfigFile(config.HCL) if err != nil { return nil, err } - skipReaper := isRYUKDisabled() - req := testcontainers.ContainerRequest{ - Image: consulImage + ":" + config.Version, - ExposedPorts: []string{"8500/tcp"}, - WaitingFor: wait.ForLog(bootLogLine).WithStartupTimeout(10 * time.Second), - AutoRemove: false, - Name: name, - Mounts: testcontainers.ContainerMounts{ - testcontainers.ContainerMount{Source: testcontainers.DockerBindMountSource{HostPath: configFile}, Target: "/consul/config/config.hcl"}, - testcontainers.ContainerMount{Source: testcontainers.DockerBindMountSource{HostPath: tmpDirData}, Target: "/consul/data"}, - }, - Cmd: config.Cmd, - SkipReaper: skipReaper, - Env: map[string]string{"CONSUL_LICENSE": license}, - } + + req := newContainerRequest(config, name, configFile, tmpDirData, license) container, err := newConsulContainerWithReq(ctx, req) if err != nil { return nil, err } + if err := container.StartLogProducer(ctx); err != nil { + return nil, err + } + container.FollowOutput(&NodeLogConsumer{ + Prefix: pc.NodeName, + }) + localIP, err := container.Host(ctx) if err != nil { return nil, err @@ -104,21 +103,42 @@ func NewConsulContainer(ctx context.Context, config Config) (Node, error) { } uri := fmt.Sprintf("http://%s:%s", localIP, mappedPort.Port()) - c := new(consulContainerNode) - c.config = config - c.container = container - c.ip = ip - c.port = mappedPort.Int() apiConfig := api.DefaultConfig() apiConfig.Address = uri - c.client, err = api.NewClient(apiConfig) - c.ctx = ctx - c.req = req - c.dataDir = tmpDirData + apiClient, err := api.NewClient(apiConfig) if err != nil { return nil, err } - return c, nil + + return &consulContainerNode{ + config: config, + container: container, + ip: ip, + port: mappedPort.Int(), + client: apiClient, + ctx: ctx, + req: req, + dataDir: tmpDirData, + }, nil +} + +func newContainerRequest(config Config, name, configFile, dataDir, license string) testcontainers.ContainerRequest { + skipReaper := isRYUKDisabled() + + return testcontainers.ContainerRequest{ + Image: consulImage + ":" + config.Version, + ExposedPorts: []string{"8500/tcp"}, + WaitingFor: wait.ForLog(bootLogLine).WithStartupTimeout(10 * time.Second), + AutoRemove: false, + Name: name, + Mounts: []testcontainers.ContainerMount{ + {Source: testcontainers.DockerBindMountSource{HostPath: configFile}, Target: "/consul/config/config.hcl"}, + {Source: testcontainers.DockerBindMountSource{HostPath: dataDir}, Target: "/consul/data"}, + }, + Cmd: config.Cmd, + SkipReaper: skipReaper, + Env: map[string]string{"CONSUL_LICENSE": license}, + } } // GetClient returns an API client that can be used to communicate with the Node. @@ -132,25 +152,44 @@ func (c *consulContainerNode) GetAddr() (string, int) { } func (c *consulContainerNode) Upgrade(ctx context.Context, config Config) error { + pc, err := readSomeConfigFileFields(config.HCL) + if err != nil { + return err + } + file, err := createConfigFile(config.HCL) if err != nil { return err } - c.req.Cmd = config.Cmd - c.req.Mounts = testcontainers.ContainerMounts{ - testcontainers.ContainerMount{Source: testcontainers.DockerBindMountSource{HostPath: file}, Target: "/consul/config/config.hcl"}, - testcontainers.ContainerMount{Source: testcontainers.DockerBindMountSource{HostPath: c.dataDir}, Target: "/consul/data"}, - } - c.req.Image = consulImage + ":" + config.Version - err = c.container.Terminate(ctx) - if err != nil { + + req2 := newContainerRequest( + config, + c.req.Name, + file, + c.dataDir, + "", + ) + req2.Env = c.req.Env // copy license + + _ = c.container.StopLogProducer() + if err := c.container.Terminate(ctx); err != nil { return err } + + c.req = req2 + container, err := newConsulContainerWithReq(ctx, c.req) if err != nil { return err } + if err := container.StartLogProducer(ctx); err != nil { + return err + } + container.FollowOutput(&NodeLogConsumer{ + Prefix: pc.NodeName, + }) + c.container = container localIP, err := container.Host(ctx) @@ -185,7 +224,19 @@ func (c *consulContainerNode) Upgrade(ctx context.Context, config Config) error // Terminate attempts to terminate the container. On failure, an error will be // returned and the reaper process (RYUK) will handle cleanup. func (c *consulContainerNode) Terminate() error { - return c.container.Terminate(c.ctx) + if c.container == nil { + return nil + } + + err := c.container.StopLogProducer() + + if err1 := c.container.Terminate(c.ctx); err == nil { + err = err1 + } + + c.container = nil + + return err } // isRYUKDisabled returns whether the reaper process (RYUK) has been disabled @@ -236,3 +287,15 @@ func createConfigFile(HCL string) (string, error) { } return configFile, nil } + +type parsedConfig struct { + NodeName string `hcl:"node_name"` +} + +func readSomeConfigFileFields(HCL string) (parsedConfig, error) { + var pc parsedConfig + if err := hcl.Decode(&pc, HCL); err != nil { + return pc, fmt.Errorf("Failed to parse config file: %w", err) + } + return pc, nil +} diff --git a/test/integration/consul-container/libs/node/log.go b/test/integration/consul-container/libs/node/log.go new file mode 100644 index 000000000..58b63f2ea --- /dev/null +++ b/test/integration/consul-container/libs/node/log.go @@ -0,0 +1,23 @@ +package node + +import ( + "fmt" + "os" + + "github.com/testcontainers/testcontainers-go" +) + +type NodeLogConsumer struct { + Prefix string +} + +var _ testcontainers.LogConsumer = (*NodeLogConsumer)(nil) + +func (c *NodeLogConsumer) Accept(log testcontainers.Log) { + switch log.LogType { + case "STDOUT": + fmt.Fprint(os.Stdout, c.Prefix+" ~~ "+string(log.Content)) + case "STDERR": + fmt.Fprint(os.Stderr, c.Prefix+" ~~ "+string(log.Content)) + } +} diff --git a/test/integration/consul-container/libs/node/node.go b/test/integration/consul-container/libs/node/node.go index 69e5c7e50..6f02a446c 100644 --- a/test/integration/consul-container/libs/node/node.go +++ b/test/integration/consul-container/libs/node/node.go @@ -2,19 +2,18 @@ package node import ( "context" + "github.com/hashicorp/consul/api" ) -type ( - // Node represent a Consul node abstraction - Node interface { - Terminate() error - GetClient() *api.Client - GetAddr() (string, int) - GetConfig() Config - Upgrade(ctx context.Context, config Config) error - } -) +// Node represent a Consul node abstraction +type Node interface { + Terminate() error + GetClient() *api.Client + GetAddr() (string, int) + GetConfig() Config + Upgrade(ctx context.Context, config Config) error +} // Config is a set of configurations required to create a Node type Config struct { diff --git a/test/integration/consul-container/metrics/leader_test.go b/test/integration/consul-container/metrics/leader_test.go index 9bea12592..08cf0e392 100644 --- a/test/integration/consul-container/metrics/leader_test.go +++ b/test/integration/consul-container/metrics/leader_test.go @@ -23,7 +23,7 @@ func TestLeadershipMetrics(t *testing.T) { configs = append(configs, node.Config{ HCL: `node_name="` + utils.RandName("consul-server") + `" - log_level="TRACE" + log_level="DEBUG" server=true telemetry { statsite_address = "127.0.0.1:2180" @@ -37,7 +37,7 @@ func TestLeadershipMetrics(t *testing.T) { configs = append(configs, node.Config{ HCL: `node_name="` + utils.RandName("consul-server") + `" - log_level="TRACE" + log_level="DEBUG" bootstrap_expect=3 server=true`, Cmd: []string{"agent", "-client=0.0.0.0"}, diff --git a/test/integration/consul-container/upgrade/healthcheck_test.go b/test/integration/consul-container/upgrade/healthcheck_test.go index 73c8a807b..7cc88ad6e 100644 --- a/test/integration/consul-container/upgrade/healthcheck_test.go +++ b/test/integration/consul-container/upgrade/healthcheck_test.go @@ -9,6 +9,7 @@ import ( "github.com/stretchr/testify/require" "github.com/hashicorp/consul/api" + libcluster "github.com/hashicorp/consul/integration/consul-container/libs/cluster" "github.com/hashicorp/consul/integration/consul-container/libs/node" "github.com/hashicorp/consul/integration/consul-container/libs/utils" @@ -74,7 +75,7 @@ func TestMixedServersMajorityLatestGAClient(t *testing.T) { configs = append(configs, node.Config{ HCL: `node_name="` + utils.RandName("consul-server") + `" - log_level="TRACE" + log_level="DEBUG" server=true`, Cmd: []string{"agent", "-client=0.0.0.0"}, Version: *utils.TargetImage, @@ -84,7 +85,7 @@ func TestMixedServersMajorityLatestGAClient(t *testing.T) { configs = append(configs, node.Config{ HCL: `node_name="` + utils.RandName("consul-server") + `" - log_level="TRACE" + log_level="DEBUG" bootstrap_expect=3 server=true`, Cmd: []string{"agent", "-client=0.0.0.0"}, @@ -151,7 +152,7 @@ func TestMixedServersMajorityTargetGAClient(t *testing.T) { configs = append(configs, node.Config{ HCL: `node_name="` + utils.RandName("consul-server") + `" - log_level="TRACE" + log_level="DEBUG" bootstrap_expect=3 server=true`, Cmd: []string{"agent", "-client=0.0.0.0"}, @@ -162,7 +163,7 @@ func TestMixedServersMajorityTargetGAClient(t *testing.T) { configs = append(configs, node.Config{ HCL: `node_name="` + utils.RandName("consul-server") + `" - log_level="TRACE" + log_level="DEBUG" server=true`, Cmd: []string{"agent", "-client=0.0.0.0"}, Version: *utils.LatestImage, @@ -227,7 +228,7 @@ func clientsCreate(t *testing.T, numClients int, version string, serfKey string) node.Config{ HCL: fmt.Sprintf(` node_name = %q - log_level = "TRACE" + log_level = "DEBUG" encrypt = %q`, utils.RandName("consul-client"), serfKey), Cmd: []string{"agent", "-client=0.0.0.0"}, Version: version, @@ -255,7 +256,7 @@ func serversCluster(t *testing.T, numServers int, version string) *libcluster.Cl for i := 0; i < numServers; i++ { configs = append(configs, node.Config{ HCL: `node_name="` + utils.RandName("consul-server") + `" - log_level="TRACE" + log_level="DEBUG" bootstrap_expect=3 server=true`, Cmd: []string{"agent", "-client=0.0.0.0"}, From a5f9994128dff8149b0aabc6c0423aad5c24bb0f Mon Sep 17 00:00:00 2001 From: "Chris S. Kim" Date: Fri, 24 Jun 2022 14:38:39 -0400 Subject: [PATCH 144/149] Add new index for PeeredServiceName and ServiceVirtualIP (#13582) For TProxy we will be leveraging the VirtualIP table, which needs to become peer-aware --- agent/consul/catalog_endpoint.go | 5 ++- agent/consul/fsm/snapshot_oss_test.go | 12 ++++-- agent/consul/leader_test.go | 13 +++--- agent/consul/state/catalog.go | 50 +++++++++++----------- agent/consul/state/catalog_oss.go | 12 ++++++ agent/consul/state/catalog_oss_test.go | 42 ++++++++++++++++--- agent/consul/state/catalog_schema.go | 14 +++++-- agent/consul/state/catalog_test.go | 57 +++++++++++++++++++++----- 8 files changed, 151 insertions(+), 54 deletions(-) diff --git a/agent/consul/catalog_endpoint.go b/agent/consul/catalog_endpoint.go index 9fe11800b..6508ba220 100644 --- a/agent/consul/catalog_endpoint.go +++ b/agent/consul/catalog_endpoint.go @@ -8,7 +8,7 @@ import ( "github.com/armon/go-metrics" "github.com/armon/go-metrics/prometheus" - bexpr "github.com/hashicorp/go-bexpr" + "github.com/hashicorp/go-bexpr" "github.com/hashicorp/go-hclog" "github.com/hashicorp/go-memdb" "github.com/hashicorp/go-uuid" @@ -1036,6 +1036,7 @@ func (c *Catalog) VirtualIPForService(args *structs.ServiceSpecificRequest, repl } state := c.srv.fsm.State() - *reply, err = state.VirtualIPForService(structs.NewServiceName(args.ServiceName, &args.EnterpriseMeta)) + psn := structs.PeeredServiceName{Peer: args.PeerName, ServiceName: structs.NewServiceName(args.ServiceName, &args.EnterpriseMeta)} + *reply, err = state.VirtualIPForService(psn) return err } diff --git a/agent/consul/fsm/snapshot_oss_test.go b/agent/consul/fsm/snapshot_oss_test.go index 29678d1d0..bb81d1627 100644 --- a/agent/consul/fsm/snapshot_oss_test.go +++ b/agent/consul/fsm/snapshot_oss_test.go @@ -451,7 +451,8 @@ func TestFSM_SnapshotRestore_OSS(t *testing.T) { Port: 8000, Connect: connectConf, }) - vip, err := fsm.state.VirtualIPForService(structs.NewServiceName("frontend", nil)) + psn := structs.PeeredServiceName{ServiceName: structs.NewServiceName("frontend", nil)} + vip, err := fsm.state.VirtualIPForService(psn) require.NoError(t, err) require.Equal(t, vip, "240.0.0.1") @@ -462,7 +463,8 @@ func TestFSM_SnapshotRestore_OSS(t *testing.T) { Port: 9000, Connect: connectConf, }) - vip, err = fsm.state.VirtualIPForService(structs.NewServiceName("backend", nil)) + psn = structs.PeeredServiceName{ServiceName: structs.NewServiceName("backend", nil)} + vip, err = fsm.state.VirtualIPForService(psn) require.NoError(t, err) require.Equal(t, vip, "240.0.0.2") @@ -592,10 +594,12 @@ func TestFSM_SnapshotRestore_OSS(t *testing.T) { require.Equal(t, uint64(25), checks[0].ModifyIndex) // Verify virtual IPs are consistent. - vip, err = fsm2.state.VirtualIPForService(structs.NewServiceName("frontend", nil)) + psn = structs.PeeredServiceName{ServiceName: structs.NewServiceName("frontend", nil)} + vip, err = fsm2.state.VirtualIPForService(psn) require.NoError(t, err) require.Equal(t, vip, "240.0.0.1") - vip, err = fsm2.state.VirtualIPForService(structs.NewServiceName("backend", nil)) + psn = structs.PeeredServiceName{ServiceName: structs.NewServiceName("backend", nil)} + vip, err = fsm2.state.VirtualIPForService(psn) require.NoError(t, err) require.Equal(t, vip, "240.0.0.2") diff --git a/agent/consul/leader_test.go b/agent/consul/leader_test.go index 7b04518fc..b041d2f92 100644 --- a/agent/consul/leader_test.go +++ b/agent/consul/leader_test.go @@ -2258,7 +2258,8 @@ func TestLeader_EnableVirtualIPs(t *testing.T) { }) require.NoError(t, err) - vip, err := state.VirtualIPForService(structs.NewServiceName("api", nil)) + psn := structs.PeeredServiceName{ServiceName: structs.NewServiceName("api", nil)} + vip, err := state.VirtualIPForService(psn) require.NoError(t, err) require.Equal(t, "", vip) @@ -2287,7 +2288,8 @@ func TestLeader_EnableVirtualIPs(t *testing.T) { // Make sure the service referenced in the terminating gateway config doesn't have // a virtual IP yet. - vip, err = state.VirtualIPForService(structs.NewServiceName("bar", nil)) + psn = structs.PeeredServiceName{ServiceName: structs.NewServiceName("bar", nil)} + vip, err = state.VirtualIPForService(psn) require.NoError(t, err) require.Equal(t, "", vip) @@ -2316,8 +2318,8 @@ func TestLeader_EnableVirtualIPs(t *testing.T) { }, }) require.NoError(t, err) - - vip, err = state.VirtualIPForService(structs.NewServiceName("api", nil)) + psn = structs.PeeredServiceName{ServiceName: structs.NewServiceName("api", nil)} + vip, err = state.VirtualIPForService(psn) require.NoError(t, err) require.Equal(t, "240.0.0.1", vip) @@ -2345,7 +2347,8 @@ func TestLeader_EnableVirtualIPs(t *testing.T) { // Make sure the baz service (only referenced in the config entry so far) // has a virtual IP. - vip, err = state.VirtualIPForService(structs.NewServiceName("baz", nil)) + psn = structs.PeeredServiceName{ServiceName: structs.NewServiceName("baz", nil)} + vip, err = state.VirtualIPForService(psn) require.NoError(t, err) require.Equal(t, "240.0.0.2", vip) } diff --git a/agent/consul/state/catalog.go b/agent/consul/state/catalog.go index 06990011e..2777b2fd1 100644 --- a/agent/consul/state/catalog.go +++ b/agent/consul/state/catalog.go @@ -7,7 +7,7 @@ import ( "reflect" "strings" - memdb "github.com/hashicorp/go-memdb" + "github.com/hashicorp/go-memdb" "github.com/mitchellh/copystructure" "github.com/hashicorp/consul/acl" @@ -872,9 +872,10 @@ func ensureServiceTxn(tx WriteTxn, idx uint64, node string, preserveIndexes bool return fmt.Errorf("failed updating gateway mapping: %s", err) } } - } - if err := upsertKindServiceName(tx, idx, svc.Kind, svc.CompoundServiceName()); err != nil { - return fmt.Errorf("failed to persist service name: %v", err) + // Only upsert KindServiceName if service is local + if err := upsertKindServiceName(tx, idx, svc.Kind, svc.CompoundServiceName()); err != nil { + return fmt.Errorf("failed to persist service name: %v", err) + } } // Update upstream/downstream mappings if it's a connect service @@ -896,7 +897,8 @@ func ensureServiceTxn(tx WriteTxn, idx uint64, node string, preserveIndexes bool } sn := structs.ServiceName{Name: service, EnterpriseMeta: svc.EnterpriseMeta} - vip, err := assignServiceVirtualIP(tx, sn) + psn := structs.PeeredServiceName{Peer: svc.PeerName, ServiceName: sn} + vip, err := assignServiceVirtualIP(tx, psn) if err != nil { return fmt.Errorf("failed updating virtual IP: %s", err) } @@ -976,9 +978,8 @@ func ensureServiceTxn(tx WriteTxn, idx uint64, node string, preserveIndexes bool // assignServiceVirtualIP assigns a virtual IP to the target service and updates // the global virtual IP counter if necessary. -func assignServiceVirtualIP(tx WriteTxn, sn structs.ServiceName) (string, error) { - // TODO(peering): support VIPs - serviceVIP, err := tx.First(tableServiceVirtualIPs, indexID, sn) +func assignServiceVirtualIP(tx WriteTxn, psn structs.PeeredServiceName) (string, error) { + serviceVIP, err := tx.First(tableServiceVirtualIPs, indexID, psn) if err != nil { return "", fmt.Errorf("failed service virtual IP lookup: %s", err) } @@ -1049,7 +1050,7 @@ func assignServiceVirtualIP(tx WriteTxn, sn structs.ServiceName) (string, error) } assignedVIP := ServiceVirtualIP{ - Service: sn, + Service: psn, IP: newEntry.IP, } if err := tx.Insert(tableServiceVirtualIPs, assignedVIP); err != nil { @@ -1877,10 +1878,6 @@ func (s *Store) deleteServiceTxn(tx WriteTxn, idx uint64, nodeName, serviceID st return nil } - // TODO: accept a non-pointer value for EnterpriseMeta - if entMeta == nil { - entMeta = structs.DefaultEnterpriseMetaInDefaultPartition() - } // Delete any checks associated with the service. This will invalidate // sessions as necessary. nsq := NodeServiceQuery{ @@ -1965,7 +1962,8 @@ func (s *Store) deleteServiceTxn(tx WriteTxn, idx uint64, nodeName, serviceID st return fmt.Errorf("failed to clean up gateway-service associations for %q: %v", name.String(), err) } } - if err := freeServiceVirtualIP(tx, svc.ServiceName, nil, entMeta); err != nil { + psn := structs.PeeredServiceName{Peer: svc.PeerName, ServiceName: name} + if err := freeServiceVirtualIP(tx, psn, nil); err != nil { return fmt.Errorf("failed to clean up virtual IP for %q: %v", name.String(), err) } if err := cleanupKindServiceName(tx, idx, svc.CompoundServiceName(), svc.ServiceKind); err != nil { @@ -1981,7 +1979,11 @@ func (s *Store) deleteServiceTxn(tx WriteTxn, idx uint64, nodeName, serviceID st // freeServiceVirtualIP is used to free a virtual IP for a service after the last instance // is removed. -func freeServiceVirtualIP(tx WriteTxn, svc string, excludeGateway *structs.ServiceName, entMeta *acl.EnterpriseMeta) error { +func freeServiceVirtualIP( + tx WriteTxn, + psn structs.PeeredServiceName, + excludeGateway *structs.ServiceName, +) error { supported, err := virtualIPsSupported(tx, nil) if err != nil { return err @@ -1991,15 +1993,14 @@ func freeServiceVirtualIP(tx WriteTxn, svc string, excludeGateway *structs.Servi } // Don't deregister the virtual IP if at least one terminating gateway still references this service. - sn := structs.NewServiceName(svc, entMeta) termGatewaySupported, err := terminatingGatewayVirtualIPsSupported(tx, nil) if err != nil { return err } if termGatewaySupported { - svcGateways, err := tx.Get(tableGatewayServices, indexService, sn) + svcGateways, err := tx.Get(tableGatewayServices, indexService, psn.ServiceName) if err != nil { - return fmt.Errorf("failed gateway lookup for %q: %s", sn.Name, err) + return fmt.Errorf("failed gateway lookup for %q: %s", psn.ServiceName.Name, err) } for service := svcGateways.Next(); service != nil; service = svcGateways.Next() { @@ -2012,7 +2013,7 @@ func freeServiceVirtualIP(tx WriteTxn, svc string, excludeGateway *structs.Servi } } - serviceVIP, err := tx.First(tableServiceVirtualIPs, indexID, sn) + serviceVIP, err := tx.First(tableServiceVirtualIPs, indexID, psn) if err != nil { return fmt.Errorf("failed service virtual IP lookup: %s", err) } @@ -2879,11 +2880,11 @@ func (s *Store) GatewayServices(ws memdb.WatchSet, gateway string, entMeta *acl. return lib.MaxUint64(maxIdx, idx), results, nil } -func (s *Store) VirtualIPForService(sn structs.ServiceName) (string, error) { +func (s *Store) VirtualIPForService(psn structs.PeeredServiceName) (string, error) { tx := s.db.Txn(false) defer tx.Abort() - vip, err := tx.First(tableServiceVirtualIPs, indexID, sn) + vip, err := tx.First(tableServiceVirtualIPs, indexID, psn) if err != nil { return "", fmt.Errorf("failed service virtual IP lookup: %s", err) } @@ -3336,7 +3337,9 @@ func getTermGatewayVirtualIPs(tx WriteTxn, services []structs.LinkedService, ent addrs := make(map[string]structs.ServiceAddress, len(services)) for _, s := range services { sn := structs.ServiceName{Name: s.Name, EnterpriseMeta: *entMeta} - vip, err := assignServiceVirtualIP(tx, sn) + // Terminating Gateways cannot route to services in peered clusters + psn := structs.PeeredServiceName{ServiceName: sn, Peer: structs.DefaultPeerKeyword} + vip, err := assignServiceVirtualIP(tx, psn) if err != nil { return nil, err } @@ -3413,7 +3416,8 @@ func updateTerminatingGatewayVirtualIPs(tx WriteTxn, idx uint64, conf *structs.T return err } if len(nodes) == 0 { - if err := freeServiceVirtualIP(tx, sn.Name, &gatewayName, &sn.EnterpriseMeta); err != nil { + psn := structs.PeeredServiceName{Peer: structs.DefaultPeerKeyword, ServiceName: sn} + if err := freeServiceVirtualIP(tx, psn, &gatewayName); err != nil { return err } } diff --git a/agent/consul/state/catalog_oss.go b/agent/consul/state/catalog_oss.go index 706a26532..ee3a9e487 100644 --- a/agent/consul/state/catalog_oss.go +++ b/agent/consul/state/catalog_oss.go @@ -299,3 +299,15 @@ func updateKindServiceNamesIndex(tx WriteTxn, idx uint64, kind structs.ServiceKi } return nil } + +func indexFromPeeredServiceName(psn structs.PeeredServiceName) ([]byte, error) { + peer := structs.LocalPeerKeyword + if psn.Peer != "" { + peer = psn.Peer + } + + var b indexBuilder + b.String(strings.ToLower(peer)) + b.String(strings.ToLower(psn.ServiceName.Name)) + return b.Bytes(), nil +} diff --git a/agent/consul/state/catalog_oss_test.go b/agent/consul/state/catalog_oss_test.go index 7ed7429fc..36d15b954 100644 --- a/agent/consul/state/catalog_oss_test.go +++ b/agent/consul/state/catalog_oss_test.go @@ -669,8 +669,19 @@ func testIndexerTableServices() map[string]indexerTestCase { func testIndexerTableServiceVirtualIPs() map[string]indexerTestCase { obj := ServiceVirtualIP{ - Service: structs.ServiceName{ - Name: "foo", + Service: structs.PeeredServiceName{ + ServiceName: structs.ServiceName{ + Name: "foo", + }, + }, + IP: net.ParseIP("127.0.0.1"), + } + peeredObj := ServiceVirtualIP{ + Service: structs.PeeredServiceName{ + ServiceName: structs.ServiceName{ + Name: "foo", + }, + Peer: "Billing", }, IP: net.ParseIP("127.0.0.1"), } @@ -678,14 +689,33 @@ func testIndexerTableServiceVirtualIPs() map[string]indexerTestCase { return map[string]indexerTestCase{ indexID: { read: indexValue{ - source: structs.ServiceName{ - Name: "foo", + source: structs.PeeredServiceName{ + ServiceName: structs.ServiceName{ + Name: "foo", + }, }, - expected: []byte("foo\x00"), + expected: []byte("internal\x00foo\x00"), }, write: indexValue{ source: obj, - expected: []byte("foo\x00"), + expected: []byte("internal\x00foo\x00"), + }, + extra: []indexerTestCase{ + { + read: indexValue{ + source: structs.PeeredServiceName{ + ServiceName: structs.ServiceName{ + Name: "foo", + }, + Peer: "Billing", + }, + expected: []byte("billing\x00foo\x00"), + }, + write: indexValue{ + source: peeredObj, + expected: []byte("billing\x00foo\x00"), + }, + }, }, }, } diff --git a/agent/consul/state/catalog_schema.go b/agent/consul/state/catalog_schema.go index 552d9c0bd..d77487c4a 100644 --- a/agent/consul/state/catalog_schema.go +++ b/agent/consul/state/catalog_schema.go @@ -605,7 +605,7 @@ func (q NodeCheckQuery) PartitionOrDefault() string { // ServiceVirtualIP is used to store a virtual IP associated with a service. // It is also used to store assigned virtual IPs when a snapshot is created. type ServiceVirtualIP struct { - Service structs.ServiceName + Service structs.PeeredServiceName IP net.IP } @@ -631,14 +631,22 @@ func serviceVirtualIPTableSchema() *memdb.TableSchema { Name: indexID, AllowMissing: false, Unique: true, - Indexer: &ServiceNameIndex{ - Field: "Service", + Indexer: indexerSingle[structs.PeeredServiceName, ServiceVirtualIP]{ + readIndex: indexFromPeeredServiceName, + writeIndex: indexFromServiceVirtualIP, }, }, }, } } +func indexFromServiceVirtualIP(vip ServiceVirtualIP) ([]byte, error) { + if vip.Service.ServiceName.Name == "" { + return nil, errMissingValueForIndex + } + return indexFromPeeredServiceName(vip.Service) +} + func freeVirtualIPTableSchema() *memdb.TableSchema { return &memdb.TableSchema{ Name: tableFreeVirtualIPs, diff --git a/agent/consul/state/catalog_test.go b/agent/consul/state/catalog_test.go index fd88ac2e1..d2a970b07 100644 --- a/agent/consul/state/catalog_test.go +++ b/agent/consul/state/catalog_test.go @@ -11,7 +11,7 @@ import ( "time" "github.com/hashicorp/go-memdb" - uuid "github.com/hashicorp/go-uuid" + "github.com/hashicorp/go-uuid" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -1799,7 +1799,7 @@ func TestStateStore_EnsureService_VirtualIPAssign(t *testing.T) { require.NoError(t, s.EnsureService(10, "node1", ns1)) // Make sure there's a virtual IP for the foo service. - vip, err := s.VirtualIPForService(structs.ServiceName{Name: "foo"}) + vip, err := s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "foo"}}) require.NoError(t, err) assert.Equal(t, "240.0.0.1", vip) @@ -1830,7 +1830,7 @@ func TestStateStore_EnsureService_VirtualIPAssign(t *testing.T) { require.NoError(t, s.EnsureService(11, "node1", ns2)) // Make sure the virtual IP has been incremented for the redis service. - vip, err = s.VirtualIPForService(structs.ServiceName{Name: "redis"}) + vip, err = s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "redis"}}) require.NoError(t, err) assert.Equal(t, "240.0.0.2", vip) @@ -1846,7 +1846,7 @@ func TestStateStore_EnsureService_VirtualIPAssign(t *testing.T) { // Delete the first service and make sure it no longer has a virtual IP assigned. require.NoError(t, s.DeleteService(12, "node1", "foo", entMeta, "")) - vip, err = s.VirtualIPForService(structs.ServiceName{Name: "connect-proxy"}) + vip, err = s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "connect-proxy"}}) require.NoError(t, err) assert.Equal(t, "", vip) @@ -1867,7 +1867,7 @@ func TestStateStore_EnsureService_VirtualIPAssign(t *testing.T) { require.NoError(t, s.EnsureService(13, "node1", ns3)) // Make sure the virtual IP is unchanged for the redis service. - vip, err = s.VirtualIPForService(structs.ServiceName{Name: "redis"}) + vip, err = s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "redis"}}) require.NoError(t, err) assert.Equal(t, "240.0.0.2", vip) @@ -1895,7 +1895,7 @@ func TestStateStore_EnsureService_VirtualIPAssign(t *testing.T) { require.NoError(t, s.EnsureService(14, "node1", ns4)) // Make sure the virtual IP has allocated from the previously freed service. - vip, err = s.VirtualIPForService(structs.ServiceName{Name: "web"}) + vip, err = s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "web"}}) require.NoError(t, err) assert.Equal(t, "240.0.0.1", vip) @@ -1905,6 +1905,41 @@ func TestStateStore_EnsureService_VirtualIPAssign(t *testing.T) { taggedAddress = out.Services["web-proxy"].TaggedAddresses[structs.TaggedAddressVirtualIP] assert.Equal(t, vip, taggedAddress.Address) assert.Equal(t, ns4.Port, taggedAddress.Port) + + // Register a node1 in another peer (technically this node would be imported + // and stored through the peering stream handlers). + testRegisterNodeOpts(t, s, 15, "node1", func(node *structs.Node) error { + node.PeerName = "billing" + return nil + }) + // Register an identical service but imported from a peer + ns5 := &structs.NodeService{ + Kind: structs.ServiceKindConnectProxy, + ID: "web-proxy", + Service: "web-proxy", + Address: "4.4.4.4", + Port: 4444, + Weights: &structs.Weights{ + Passing: 1, + Warning: 1, + }, + Proxy: structs.ConnectProxyConfig{DestinationServiceName: "web"}, + EnterpriseMeta: *entMeta, + PeerName: "billing", + } + require.NoError(t, s.EnsureService(15, "node1", ns5)) + + // Make sure the virtual IP is different from the identically named local service. + vip, err = s.VirtualIPForService(structs.PeeredServiceName{Peer: "billing", ServiceName: structs.ServiceName{Name: "web"}}) + require.NoError(t, err) + assert.Equal(t, "240.0.0.3", vip) + + // Retrieve and verify + _, out, err = s.NodeServices(nil, "node1", nil, "billing") + require.NoError(t, err) + taggedAddress = out.Services["web-proxy"].TaggedAddresses[structs.TaggedAddressVirtualIP] + assert.Equal(t, vip, taggedAddress.Address) + assert.Equal(t, ns5.Port, taggedAddress.Port) } func TestStateStore_EnsureService_ReassignFreedVIPs(t *testing.T) { @@ -1931,7 +1966,7 @@ func TestStateStore_EnsureService_ReassignFreedVIPs(t *testing.T) { require.NoError(t, s.EnsureService(10, "node1", ns1)) // Make sure there's a virtual IP for the foo service. - vip, err := s.VirtualIPForService(structs.ServiceName{Name: "foo"}) + vip, err := s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "foo"}}) require.NoError(t, err) assert.Equal(t, "240.0.0.1", vip) @@ -1961,7 +1996,7 @@ func TestStateStore_EnsureService_ReassignFreedVIPs(t *testing.T) { require.NoError(t, s.EnsureService(11, "node1", ns2)) // Make sure the virtual IP has been incremented for the redis service. - vip, err = s.VirtualIPForService(structs.ServiceName{Name: "redis"}) + vip, err = s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "redis"}}) require.NoError(t, err) assert.Equal(t, "240.0.0.2", vip) @@ -1976,7 +2011,7 @@ func TestStateStore_EnsureService_ReassignFreedVIPs(t *testing.T) { // Delete the last service and make sure it no longer has a virtual IP assigned. require.NoError(t, s.DeleteService(12, "node1", "redis", entMeta, "")) - vip, err = s.VirtualIPForService(structs.ServiceName{Name: "redis"}) + vip, err = s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "redis"}}) require.NoError(t, err) assert.Equal(t, "", vip) @@ -1996,7 +2031,7 @@ func TestStateStore_EnsureService_ReassignFreedVIPs(t *testing.T) { } require.NoError(t, s.EnsureService(13, "node1", ns3)) - vip, err = s.VirtualIPForService(structs.ServiceName{Name: "backend"}) + vip, err = s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "backend"}}) require.NoError(t, err) assert.Equal(t, "240.0.0.2", vip) @@ -2026,7 +2061,7 @@ func TestStateStore_EnsureService_ReassignFreedVIPs(t *testing.T) { require.NoError(t, s.EnsureService(14, "node1", ns4)) // Make sure the virtual IP has been incremented for the frontend service. - vip, err = s.VirtualIPForService(structs.ServiceName{Name: "frontend"}) + vip, err = s.VirtualIPForService(structs.PeeredServiceName{ServiceName: structs.ServiceName{Name: "frontend"}}) require.NoError(t, err) assert.Equal(t, "240.0.0.3", vip) From 187c72ead6cfa7eb59bd4d2a1035fd6cf5654c8d Mon Sep 17 00:00:00 2001 From: Evan Culver Date: Fri, 24 Jun 2022 13:45:30 -0700 Subject: [PATCH 145/149] Remove trigger-oss-merge job (#13600) --- .circleci/config.yml | 25 ------------------------- 1 file changed, 25 deletions(-) diff --git a/.circleci/config.yml b/.circleci/config.yml index c22d04fb8..df9a60bde 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -930,21 +930,6 @@ jobs: path: *TEST_RESULTS_DIR - run: *notify-slack-failure - trigger-oss-merge: - docker: - - image: docker.mirror.hashicorp.services/alpine:3.12 - steps: - - run: apk add --no-cache --no-progress curl jq - - run: - name: trigger oss merge - command: | - curl -s -X POST \ - --header "Circle-Token: ${CIRCLECI_API_TOKEN}" \ - --header "Content-Type: application/json" \ - -d '{"build_parameters": {"CIRCLE_JOB": "oss-merge"}}' \ - "https://circleci.com/api/v1.1/project/github/hashicorp/consul-enterprise/tree/${CIRCLE_BRANCH}" | jq -r '.build_url' - - run: *notify-slack-failure - # Run load tests against a commit load-test: docker: @@ -1180,16 +1165,6 @@ workflows: requires: - ember-build-ent - noop - workflow-automation: - unless: << pipeline.parameters.trigger-load-test >> - jobs: - - trigger-oss-merge: - context: team-consul - filters: - branches: - only: - - main - - /release\/\d+\.\d+\.x$/ load-test: when: << pipeline.parameters.trigger-load-test >> From 5538ba212fe9c4e55a913e5955f4730f6d94df10 Mon Sep 17 00:00:00 2001 From: Michele Degges Date: Fri, 24 Jun 2022 13:45:57 -0700 Subject: [PATCH 146/149] [CI-only] Dev tag update for main (#13541) --- .github/workflows/build.yml | 12 ++++++++++-- version/version.go | 2 +- 2 files changed, 11 insertions(+), 3 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index c4c38efc3..3a2ec423b 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -232,6 +232,14 @@ jobs: steps: - uses: actions/checkout@v2 + + # Strip everything but MAJOR.MINOR from the version string and add a `-dev` suffix + # This naming convention will be used ONLY for per-commit dev images + - name: Set docker dev tag + run: | + version="${{ env.version }}" + echo "dev_tag=${version%.*}-dev" >> $GITHUB_ENV + - name: Docker Build (Action) uses: hashicorp/actions-docker-build@v1 with: @@ -242,8 +250,8 @@ jobs: docker.io/hashicorp/${{env.repo}}:${{env.version}} public.ecr.aws/hashicorp/${{env.repo}}:${{env.version}} dev_tags: | - docker.io/hashicorppreview/${{ env.repo }}:${{ env.version }} - docker.io/hashicorppreview/${{ env.repo }}:${{ env.version }}-${{ github.sha }} + docker.io/hashicorppreview/${{ env.repo }}:${{ env.dev_tag }} + docker.io/hashicorppreview/${{ env.repo }}:${{ env.dev_tag }}-${{ github.sha }} smoke_test: .github/scripts/verify_docker.sh v${{ env.version }} build-docker-redhat: diff --git a/version/version.go b/version/version.go index 8930b4822..edcdca85b 100644 --- a/version/version.go +++ b/version/version.go @@ -10,7 +10,7 @@ var ( // compiler. GitCommit string - // The main version number that is being run at the moment. + // The next version number that will be released. This will be updated after every release. // // Version must conform to the format expected by github.com/hashicorp/go-version // for tests to work. From 4333312be956328651a7b76b3964092c239c535a Mon Sep 17 00:00:00 2001 From: alex <8968914+acpana@users.noreply.github.com> Date: Fri, 24 Jun 2022 15:17:35 -0700 Subject: [PATCH 147/149] peering, internal: support UIServices, UINodes, UINodeInfo (#13577) --- agent/consul/internal_endpoint.go | 106 ++++++- agent/consul/internal_endpoint_test.go | 407 +++++++++++++++++-------- agent/structs/structs.go | 8 +- agent/ui_endpoint.go | 79 +++-- agent/ui_endpoint_test.go | 146 +++++++-- 5 files changed, 566 insertions(+), 180 deletions(-) diff --git a/agent/consul/internal_endpoint.go b/agent/consul/internal_endpoint.go index f1fda470f..5ed07524e 100644 --- a/agent/consul/internal_endpoint.go +++ b/agent/consul/internal_endpoint.go @@ -69,18 +69,60 @@ func (m *Internal) NodeDump(args *structs.DCSpecificRequest, &args.QueryOptions, &reply.QueryMeta, func(ws memdb.WatchSet, state *state.Store) error { - index, dump, err := state.NodeDump(ws, &args.EnterpriseMeta, args.PeerName) - if err != nil { - return err + // we don't support calling this endpoint for a specific peer + if args.PeerName != "" { + return fmt.Errorf("this endpoint does not support specifying a peer: %q", args.PeerName) } - reply.Index, reply.Dump = index, dump + + // this maxIndex will be the max of the NodeDump calls and the PeeringList call + var maxIndex uint64 + // Get data for local nodes + index, dump, err := state.NodeDump(ws, &args.EnterpriseMeta, structs.DefaultPeerKeyword) + if err != nil { + return fmt.Errorf("could not get a node dump for local nodes: %w", err) + } + + if index > maxIndex { + maxIndex = index + } + reply.Dump = dump + + // get a list of all peerings + index, listedPeerings, err := state.PeeringList(ws, args.EnterpriseMeta) + if err != nil { + return fmt.Errorf("could not list peers for node dump %w", err) + } + + if index > maxIndex { + maxIndex = index + } + + // get node dumps for all peerings + for _, p := range listedPeerings { + index, importedDump, err := state.NodeDump(ws, &args.EnterpriseMeta, p.Name) + if err != nil { + return fmt.Errorf("could not get a node dump for peer %q: %w", p.Name, err) + } + reply.ImportedDump = append(reply.ImportedDump, importedDump...) + + if index > maxIndex { + maxIndex = index + } + } + reply.Index = maxIndex raw, err := filter.Execute(reply.Dump) if err != nil { - return err + return fmt.Errorf("could not filter local node dump: %w", err) } reply.Dump = raw.(structs.NodeDump) + importedRaw, err := filter.Execute(reply.ImportedDump) + if err != nil { + return fmt.Errorf("could not filter peer node dump: %w", err) + } + reply.ImportedDump = importedRaw.(structs.NodeDump) + // Note: we filter the results with ACLs *after* applying the user-supplied // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include // results that would be filtered out even if the user did have permission. @@ -111,13 +153,47 @@ func (m *Internal) ServiceDump(args *structs.ServiceDumpRequest, reply *structs. &args.QueryOptions, &reply.QueryMeta, func(ws memdb.WatchSet, state *state.Store) error { - // Get, store, and filter nodes - maxIdx, nodes, err := state.ServiceDump(ws, args.ServiceKind, args.UseServiceKind, &args.EnterpriseMeta, args.PeerName) + // we don't support calling this endpoint for a specific peer + if args.PeerName != "" { + return fmt.Errorf("this endpoint does not support specifying a peer: %q", args.PeerName) + } + + // this maxIndex will be the max of the ServiceDump calls and the PeeringList call + var maxIndex uint64 + + // get a local dump for services + index, nodes, err := state.ServiceDump(ws, args.ServiceKind, args.UseServiceKind, &args.EnterpriseMeta, structs.DefaultPeerKeyword) if err != nil { - return err + return fmt.Errorf("could not get a service dump for local nodes: %w", err) + } + + if index > maxIndex { + maxIndex = index } reply.Nodes = nodes + // get a list of all peerings + index, listedPeerings, err := state.PeeringList(ws, args.EnterpriseMeta) + if err != nil { + return fmt.Errorf("could not list peers for service dump %w", err) + } + + if index > maxIndex { + maxIndex = index + } + + for _, p := range listedPeerings { + index, importedNodes, err := state.ServiceDump(ws, args.ServiceKind, args.UseServiceKind, &args.EnterpriseMeta, p.Name) + if err != nil { + return fmt.Errorf("could not get a service dump for peer %q: %w", p.Name, err) + } + + if index > maxIndex { + maxIndex = index + } + reply.ImportedNodes = append(reply.ImportedNodes, importedNodes...) + } + // Get, store, and filter gateway services idx, gatewayServices, err := state.DumpGatewayServices(ws) if err != nil { @@ -125,17 +201,23 @@ func (m *Internal) ServiceDump(args *structs.ServiceDumpRequest, reply *structs. } reply.Gateways = gatewayServices - if idx > maxIdx { - maxIdx = idx + if idx > maxIndex { + maxIndex = idx } - reply.Index = maxIdx + reply.Index = maxIndex raw, err := filter.Execute(reply.Nodes) if err != nil { - return err + return fmt.Errorf("could not filter local service dump: %w", err) } reply.Nodes = raw.(structs.CheckServiceNodes) + importedRaw, err := filter.Execute(reply.ImportedNodes) + if err != nil { + return fmt.Errorf("could not filter peer service dump: %w", err) + } + reply.ImportedNodes = importedRaw.(structs.CheckServiceNodes) + // Note: we filter the results with ACLs *after* applying the user-supplied // bexpr filter, to ensure QueryMeta.ResultsFilteredByACLs does not include // results that would be filtered out even if the user did have permission. diff --git a/agent/consul/internal_endpoint_test.go b/agent/consul/internal_endpoint_test.go index 3737f3a08..d24f08d1b 100644 --- a/agent/consul/internal_endpoint_test.go +++ b/agent/consul/internal_endpoint_test.go @@ -8,6 +8,7 @@ import ( "testing" "time" + "github.com/hashicorp/consul-net-rpc/net/rpc" "github.com/stretchr/testify/assert" "github.com/stretchr/testify/require" @@ -17,6 +18,7 @@ import ( "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" "github.com/hashicorp/consul/lib/stringslice" + "github.com/hashicorp/consul/proto/pbpeering" "github.com/hashicorp/consul/sdk/testutil" "github.com/hashicorp/consul/sdk/testutil/retry" "github.com/hashicorp/consul/testrpc" @@ -29,56 +31,79 @@ func TestInternal_NodeInfo(t *testing.T) { } t.Parallel() - dir1, s1 := testServer(t) - defer os.RemoveAll(dir1) - defer s1.Shutdown() + _, s1 := testServer(t) codec := rpcClient(t, s1) - defer codec.Close() testrpc.WaitForLeader(t, s1.RPC, "dc1") - arg := structs.RegisterRequest{ - Datacenter: "dc1", - Node: "foo", - Address: "127.0.0.1", - Service: &structs.NodeService{ - ID: "db", - Service: "db", - Tags: []string{"primary"}, + args := []*structs.RegisterRequest{ + { + Datacenter: "dc1", + Node: "foo", + Address: "127.0.0.1", + Service: &structs.NodeService{ + ID: "db", + Service: "db", + Tags: []string{"primary"}, + }, + Check: &structs.HealthCheck{ + Name: "db connect", + Status: api.HealthPassing, + ServiceID: "db", + }, }, - Check: &structs.HealthCheck{ - Name: "db connect", - Status: api.HealthPassing, - ServiceID: "db", + { + Datacenter: "dc1", + Node: "foo", + Address: "127.0.0.3", + PeerName: "peer1", }, } - var out struct{} - if err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", &arg, &out); err != nil { - t.Fatalf("err: %v", err) + + for _, reg := range args { + err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", reg, nil) + require.NoError(t, err) } - var out2 structs.IndexedNodeDump - req := structs.NodeSpecificRequest{ - Datacenter: "dc1", - Node: "foo", - } - if err := msgpackrpc.CallWithCodec(codec, "Internal.NodeInfo", &req, &out2); err != nil { - t.Fatalf("err: %v", err) - } + t.Run("get local node", func(t *testing.T) { + var out structs.IndexedNodeDump + req := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: "foo", + } + if err := msgpackrpc.CallWithCodec(codec, "Internal.NodeInfo", &req, &out); err != nil { + t.Fatalf("err: %v", err) + } - nodes := out2.Dump - if len(nodes) != 1 { - t.Fatalf("Bad: %v", nodes) - } - if nodes[0].Node != "foo" { - t.Fatalf("Bad: %v", nodes[0]) - } - if !stringslice.Contains(nodes[0].Services[0].Tags, "primary") { - t.Fatalf("Bad: %v", nodes[0]) - } - if nodes[0].Checks[0].Status != api.HealthPassing { - t.Fatalf("Bad: %v", nodes[0]) - } + nodes := out.Dump + if len(nodes) != 1 { + t.Fatalf("Bad: %v", nodes) + } + if nodes[0].Node != "foo" { + t.Fatalf("Bad: %v", nodes[0]) + } + if !stringslice.Contains(nodes[0].Services[0].Tags, "primary") { + t.Fatalf("Bad: %v", nodes[0]) + } + if nodes[0].Checks[0].Status != api.HealthPassing { + t.Fatalf("Bad: %v", nodes[0]) + } + }) + + t.Run("get peered node", func(t *testing.T) { + var out structs.IndexedNodeDump + req := structs.NodeSpecificRequest{ + Datacenter: "dc1", + Node: "foo", + PeerName: "peer1", + } + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Internal.NodeInfo", &req, &out)) + + nodes := out.Dump + require.Equal(t, 1, len(nodes)) + require.Equal(t, "foo", nodes[0].Node) + require.Equal(t, "peer1", nodes[0].PeerName) + }) } func TestInternal_NodeDump(t *testing.T) { @@ -87,53 +112,61 @@ func TestInternal_NodeDump(t *testing.T) { } t.Parallel() - dir1, s1 := testServer(t) - defer os.RemoveAll(dir1) - defer s1.Shutdown() + _, s1 := testServer(t) codec := rpcClient(t, s1) - defer codec.Close() testrpc.WaitForLeader(t, s1.RPC, "dc1") - arg := structs.RegisterRequest{ - Datacenter: "dc1", - Node: "foo", - Address: "127.0.0.1", - Service: &structs.NodeService{ - ID: "db", - Service: "db", - Tags: []string{"primary"}, + args := []*structs.RegisterRequest{ + { + Datacenter: "dc1", + Node: "foo", + Address: "127.0.0.1", + Service: &structs.NodeService{ + ID: "db", + Service: "db", + Tags: []string{"primary"}, + }, + Check: &structs.HealthCheck{ + Name: "db connect", + Status: api.HealthPassing, + ServiceID: "db", + }, }, - Check: &structs.HealthCheck{ - Name: "db connect", - Status: api.HealthPassing, - ServiceID: "db", + { + Datacenter: "dc1", + Node: "bar", + Address: "127.0.0.2", + Service: &structs.NodeService{ + ID: "db", + Service: "db", + Tags: []string{"replica"}, + }, + Check: &structs.HealthCheck{ + Name: "db connect", + Status: api.HealthWarning, + ServiceID: "db", + }, + }, + { + Datacenter: "dc1", + Node: "foo-peer", + Address: "127.0.0.3", + PeerName: "peer1", }, - } - var out struct{} - if err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", &arg, &out); err != nil { - t.Fatalf("err: %v", err) } - arg = structs.RegisterRequest{ - Datacenter: "dc1", - Node: "bar", - Address: "127.0.0.2", - Service: &structs.NodeService{ - ID: "db", - Service: "db", - Tags: []string{"replica"}, - }, - Check: &structs.HealthCheck{ - Name: "db connect", - Status: api.HealthWarning, - ServiceID: "db", - }, - } - if err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", &arg, &out); err != nil { - t.Fatalf("err: %v", err) + for _, reg := range args { + err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", reg, nil) + require.NoError(t, err) } + err := s1.fsm.State().PeeringWrite(1, &pbpeering.Peering{ + ID: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + Name: "peer1", + }) + require.NoError(t, err) + var out2 structs.IndexedNodeDump req := structs.DCSpecificRequest{ Datacenter: "dc1", @@ -175,6 +208,10 @@ func TestInternal_NodeDump(t *testing.T) { if !foundFoo || !foundBar { t.Fatalf("missing foo or bar") } + + require.Len(t, out2.ImportedDump, 1) + require.Equal(t, "peer1", out2.ImportedDump[0].PeerName) + require.Equal(t, "foo-peer", out2.ImportedDump[0].Node) } func TestInternal_NodeDump_Filter(t *testing.T) { @@ -183,60 +220,107 @@ func TestInternal_NodeDump_Filter(t *testing.T) { } t.Parallel() - dir1, s1 := testServer(t) - defer os.RemoveAll(dir1) - defer s1.Shutdown() + _, s1 := testServer(t) codec := rpcClient(t, s1) - defer codec.Close() testrpc.WaitForLeader(t, s1.RPC, "dc1") - arg := structs.RegisterRequest{ - Datacenter: "dc1", - Node: "foo", - Address: "127.0.0.1", - Service: &structs.NodeService{ - ID: "db", - Service: "db", - Tags: []string{"primary"}, + args := []*structs.RegisterRequest{ + { + Datacenter: "dc1", + Node: "foo", + Address: "127.0.0.1", + Service: &structs.NodeService{ + ID: "db", + Service: "db", + Tags: []string{"primary"}, + }, + Check: &structs.HealthCheck{ + Name: "db connect", + Status: api.HealthPassing, + ServiceID: "db", + }, }, - Check: &structs.HealthCheck{ - Name: "db connect", - Status: api.HealthPassing, - ServiceID: "db", + { + Datacenter: "dc1", + Node: "bar", + Address: "127.0.0.2", + Service: &structs.NodeService{ + ID: "db", + Service: "db", + Tags: []string{"replica"}, + }, + Check: &structs.HealthCheck{ + Name: "db connect", + Status: api.HealthWarning, + ServiceID: "db", + }, }, - } - var out struct{} - require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.Register", &arg, &out)) - - arg = structs.RegisterRequest{ - Datacenter: "dc1", - Node: "bar", - Address: "127.0.0.2", - Service: &structs.NodeService{ - ID: "db", - Service: "db", - Tags: []string{"replica"}, - }, - Check: &structs.HealthCheck{ - Name: "db connect", - Status: api.HealthWarning, - ServiceID: "db", + { + Datacenter: "dc1", + Node: "foo-peer", + Address: "127.0.0.3", + PeerName: "peer1", }, } - require.NoError(t, msgpackrpc.CallWithCodec(codec, "Catalog.Register", &arg, &out)) - - var out2 structs.IndexedNodeDump - req := structs.DCSpecificRequest{ - Datacenter: "dc1", - QueryOptions: structs.QueryOptions{Filter: "primary in Services.Tags"}, + for _, reg := range args { + err := msgpackrpc.CallWithCodec(codec, "Catalog.Register", reg, nil) + require.NoError(t, err) } - require.NoError(t, msgpackrpc.CallWithCodec(codec, "Internal.NodeDump", &req, &out2)) - nodes := out2.Dump - require.Len(t, nodes, 1) - require.Equal(t, "foo", nodes[0].Node) + err := s1.fsm.State().PeeringWrite(1, &pbpeering.Peering{ + ID: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + Name: "peer1", + }) + require.NoError(t, err) + + t.Run("filter on the local node", func(t *testing.T) { + var out2 structs.IndexedNodeDump + req := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{Filter: "primary in Services.Tags"}, + } + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Internal.NodeDump", &req, &out2)) + + nodes := out2.Dump + require.Len(t, nodes, 1) + require.Equal(t, "foo", nodes[0].Node) + }) + + t.Run("filter on imported dump", func(t *testing.T) { + var out3 structs.IndexedNodeDump + req2 := structs.DCSpecificRequest{ + Datacenter: "dc1", + QueryOptions: structs.QueryOptions{Filter: "friend in PeerName"}, + } + + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Internal.NodeDump", &req2, &out3)) + require.Len(t, out3.Dump, 0) + require.Len(t, out3.ImportedDump, 0) + }) + + t.Run("filter look for peer nodes (non local nodes)", func(t *testing.T) { + var out3 structs.IndexedNodeDump + req2 := structs.DCSpecificRequest{ + QueryOptions: structs.QueryOptions{Filter: "PeerName != \"\""}, + } + + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Internal.NodeDump", &req2, &out3)) + require.Len(t, out3.Dump, 0) + require.Len(t, out3.ImportedDump, 1) + }) + + t.Run("filter look for a specific peer", func(t *testing.T) { + var out3 structs.IndexedNodeDump + req2 := structs.DCSpecificRequest{ + QueryOptions: structs.QueryOptions{Filter: "PeerName == peer1"}, + } + + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Internal.NodeDump", &req2, &out3)) + require.Len(t, out3.Dump, 0) + require.Len(t, out3.ImportedDump, 1) + }) } func TestInternal_KeyringOperation(t *testing.T) { @@ -1665,6 +1749,89 @@ func TestInternal_GatewayServiceDump_Ingress_ACL(t *testing.T) { require.Equal(t, nodes[0].Checks[0].Status, api.HealthWarning) } +func TestInternal_ServiceDump_Peering(t *testing.T) { + if testing.Short() { + t.Skip("too slow for testing.Short") + } + + t.Parallel() + _, s1 := testServer(t) + codec := rpcClient(t, s1) + + testrpc.WaitForLeader(t, s1.RPC, "dc1") + + // prep the cluster with some data we can use in our filters + registerTestCatalogEntries(t, codec) + + doRequest := func(t *testing.T, filter string) structs.IndexedNodesWithGateways { + t.Helper() + args := structs.DCSpecificRequest{ + QueryOptions: structs.QueryOptions{Filter: filter}, + } + + var out structs.IndexedNodesWithGateways + require.NoError(t, msgpackrpc.CallWithCodec(codec, "Internal.ServiceDump", &args, &out)) + + return out + } + + t.Run("No peerings", func(t *testing.T) { + nodes := doRequest(t, "") + // redis (3), web (3), critical (1), warning (1) and consul (1) + require.Len(t, nodes.Nodes, 9) + require.Len(t, nodes.ImportedNodes, 0) + }) + + addPeerService(t, codec) + + err := s1.fsm.State().PeeringWrite(1, &pbpeering.Peering{ + ID: "9e650110-ac74-4c5a-a6a8-9348b2bed4e9", + Name: "peer1", + }) + require.NoError(t, err) + + t.Run("peerings", func(t *testing.T) { + nodes := doRequest(t, "") + // redis (3), web (3), critical (1), warning (1) and consul (1) + require.Len(t, nodes.Nodes, 9) + // service (1) + require.Len(t, nodes.ImportedNodes, 1) + }) + + t.Run("peerings w filter", func(t *testing.T) { + nodes := doRequest(t, "Node.PeerName == foo") + require.Len(t, nodes.Nodes, 0) + require.Len(t, nodes.ImportedNodes, 0) + + nodes2 := doRequest(t, "Node.PeerName == peer1") + require.Len(t, nodes2.Nodes, 0) + require.Len(t, nodes2.ImportedNodes, 1) + }) +} + +func addPeerService(t *testing.T, codec rpc.ClientCodec) { + // prep the cluster with some data we can use in our filters + registrations := map[string]*structs.RegisterRequest{ + "Peer node foo with peer service": { + Datacenter: "dc1", + Node: "foo", + ID: types.NodeID("e0155642-135d-4739-9853-a1ee6c9f945b"), + Address: "127.0.0.2", + PeerName: "peer1", + Service: &structs.NodeService{ + Kind: structs.ServiceKindTypical, + ID: "serviceID", + Service: "service", + Port: 1235, + Address: "198.18.1.2", + PeerName: "peer1", + }, + }, + } + + registerTestCatalogEntriesMap(t, codec, registrations) +} + func TestInternal_GatewayIntentions(t *testing.T) { if testing.Short() { t.Skip("too slow for testing.Short") diff --git a/agent/structs/structs.go b/agent/structs/structs.go index 918597b8d..4039b547b 100644 --- a/agent/structs/structs.go +++ b/agent/structs/structs.go @@ -2239,8 +2239,9 @@ type IndexedCheckServiceNodes struct { } type IndexedNodesWithGateways struct { - Nodes CheckServiceNodes - Gateways GatewayServices + ImportedNodes CheckServiceNodes + Nodes CheckServiceNodes + Gateways GatewayServices QueryMeta } @@ -2250,7 +2251,8 @@ type DatacenterIndexedCheckServiceNodes struct { } type IndexedNodeDump struct { - Dump NodeDump + ImportedDump NodeDump + Dump NodeDump QueryMeta } diff --git a/agent/ui_endpoint.go b/agent/ui_endpoint.go index 70d5d9def..2f74d8e59 100644 --- a/agent/ui_endpoint.go +++ b/agent/ui_endpoint.go @@ -37,6 +37,8 @@ type ServiceSummary struct { transparentProxySet bool ConnectNative bool + PeerName string `json:",omitempty"` + acl.EnterpriseMeta } @@ -117,7 +119,18 @@ RPC: if out.Dump == nil { out.Dump = make(structs.NodeDump, 0) } - return out.Dump, nil + + // Use empty list instead of nil + for _, info := range out.ImportedDump { + if info.Services == nil { + info.Services = make([]*structs.NodeService, 0) + } + if info.Checks == nil { + info.Checks = make([]*structs.HealthCheck, 0) + } + } + + return append(out.Dump, out.ImportedDump...), nil } // UINodeInfo is used to get info on a single node in a given datacenter. We return a @@ -139,6 +152,10 @@ func (s *HTTPHandlers) UINodeInfo(resp http.ResponseWriter, req *http.Request) ( return nil, HTTPError{StatusCode: http.StatusBadRequest, Reason: "Missing node name"} } + if peer := req.URL.Query().Get("peer"); peer != "" { + args.PeerName = peer + } + // Make the RPC request var out structs.IndexedNodeDump defer setMeta(resp, &out.QueryMeta) @@ -216,15 +233,17 @@ RPC: // Store the names of the gateways associated with each service var ( - serviceGateways = make(map[structs.ServiceName][]structs.ServiceName) - numLinkedServices = make(map[structs.ServiceName]int) + serviceGateways = make(map[structs.PeeredServiceName][]structs.PeeredServiceName) + numLinkedServices = make(map[structs.PeeredServiceName]int) ) for _, gs := range out.Gateways { - serviceGateways[gs.Service] = append(serviceGateways[gs.Service], gs.Gateway) - numLinkedServices[gs.Gateway] += 1 + psn := structs.PeeredServiceName{Peer: structs.DefaultPeerKeyword, ServiceName: gs.Service} + gpsn := structs.PeeredServiceName{Peer: structs.DefaultPeerKeyword, ServiceName: gs.Gateway} + serviceGateways[psn] = append(serviceGateways[psn], gpsn) + numLinkedServices[gpsn] += 1 } - summaries, hasProxy := summarizeServices(out.Nodes.ToServiceDump(), nil, "") + summaries, hasProxy := summarizeServices(append(out.Nodes, out.ImportedNodes...).ToServiceDump(), nil, "") sorted := prepSummaryOutput(summaries, false) // Ensure at least a zero length slice @@ -233,17 +252,18 @@ RPC: sum := ServiceListingSummary{ServiceSummary: *svc} sn := structs.NewServiceName(svc.Name, &svc.EnterpriseMeta) - if hasProxy[sn] { + psn := structs.PeeredServiceName{Peer: svc.PeerName, ServiceName: sn} + if hasProxy[psn] { sum.ConnectedWithProxy = true } // Verify that at least one of the gateways linked by config entry has an instance registered in the catalog - for _, gw := range serviceGateways[sn] { + for _, gw := range serviceGateways[psn] { if s := summaries[gw]; s != nil && sum.InstanceCount > 0 { sum.ConnectedWithGateway = true } } - sum.GatewayConfig.AssociatedServiceCount = numLinkedServices[sn] + sum.GatewayConfig.AssociatedServiceCount = numLinkedServices[psn] result = append(result, &sum) } @@ -389,31 +409,43 @@ RPC: return topo, nil } -func summarizeServices(dump structs.ServiceDump, cfg *config.RuntimeConfig, dc string) (map[structs.ServiceName]*ServiceSummary, map[structs.ServiceName]bool) { +func summarizeServices(dump structs.ServiceDump, cfg *config.RuntimeConfig, dc string) (map[structs.PeeredServiceName]*ServiceSummary, map[structs.PeeredServiceName]bool) { var ( - summary = make(map[structs.ServiceName]*ServiceSummary) - hasProxy = make(map[structs.ServiceName]bool) + summary = make(map[structs.PeeredServiceName]*ServiceSummary) + hasProxy = make(map[structs.PeeredServiceName]bool) ) - getService := func(service structs.ServiceName) *ServiceSummary { - serv, ok := summary[service] + getService := func(psn structs.PeeredServiceName) *ServiceSummary { + serv, ok := summary[psn] if !ok { serv = &ServiceSummary{ - Name: service.Name, - EnterpriseMeta: service.EnterpriseMeta, + Name: psn.ServiceName.Name, + EnterpriseMeta: psn.ServiceName.EnterpriseMeta, // the other code will increment this unconditionally so we // shouldn't initialize it to 1 InstanceCount: 0, + PeerName: psn.Peer, } - summary[service] = serv + summary[psn] = serv } return serv } for _, csn := range dump { + var peerName string + // all entities will have the same peer name so it is safe to use the node's peer name + if csn.Node == nil { + // this can happen for gateway dumps that call this summarize func + peerName = structs.DefaultPeerKeyword + } else { + peerName = csn.Node.PeerName + } + if cfg != nil && csn.GatewayService != nil { gwsvc := csn.GatewayService - sum := getService(gwsvc.Service) + + psn := structs.PeeredServiceName{Peer: peerName, ServiceName: gwsvc.Service} + sum := getService(psn) modifySummaryForGatewayService(cfg, dc, sum, gwsvc) } @@ -421,8 +453,10 @@ func summarizeServices(dump structs.ServiceDump, cfg *config.RuntimeConfig, dc s if csn.Service == nil { continue } + sn := structs.NewServiceName(csn.Service.Service, &csn.Service.EnterpriseMeta) - sum := getService(sn) + psn := structs.PeeredServiceName{Peer: peerName, ServiceName: sn} + sum := getService(psn) svc := csn.Service sum.Nodes = append(sum.Nodes, csn.Node.Node) @@ -432,9 +466,10 @@ func summarizeServices(dump structs.ServiceDump, cfg *config.RuntimeConfig, dc s sum.ConnectNative = svc.Connect.Native if svc.Kind == structs.ServiceKindConnectProxy { sn := structs.NewServiceName(svc.Proxy.DestinationServiceName, &svc.EnterpriseMeta) - hasProxy[sn] = true + psn := structs.PeeredServiceName{Peer: peerName, ServiceName: sn} + hasProxy[psn] = true - destination := getService(sn) + destination := getService(psn) for _, check := range csn.Checks { cid := structs.NewCheckID(check.CheckID, &check.EnterpriseMeta) uid := structs.UniqueID(csn.Node.Node, cid.String()) @@ -496,7 +531,7 @@ func summarizeServices(dump structs.ServiceDump, cfg *config.RuntimeConfig, dc s return summary, hasProxy } -func prepSummaryOutput(summaries map[structs.ServiceName]*ServiceSummary, excludeSidecars bool) []*ServiceSummary { +func prepSummaryOutput(summaries map[structs.PeeredServiceName]*ServiceSummary, excludeSidecars bool) []*ServiceSummary { var resp []*ServiceSummary // Ensure at least a zero length slice resp = make([]*ServiceSummary, 0) diff --git a/agent/ui_endpoint_test.go b/agent/ui_endpoint_test.go index f2a17f73e..0ddb180ac 100644 --- a/agent/ui_endpoint_test.go +++ b/agent/ui_endpoint_test.go @@ -2,6 +2,7 @@ package agent import ( "bytes" + "context" "fmt" "io" "io/ioutil" @@ -11,6 +12,7 @@ import ( "path/filepath" "sync/atomic" "testing" + "time" cleanhttp "github.com/hashicorp/go-cleanhttp" "github.com/stretchr/testify/assert" @@ -19,12 +21,14 @@ import ( "github.com/hashicorp/consul/agent/config" "github.com/hashicorp/consul/agent/structs" "github.com/hashicorp/consul/api" + "github.com/hashicorp/consul/proto/pbpeering" "github.com/hashicorp/consul/sdk/testutil" "github.com/hashicorp/consul/sdk/testutil/retry" "github.com/hashicorp/consul/testrpc" + "github.com/hashicorp/consul/types" ) -func TestUiIndex(t *testing.T) { +func TestUIIndex(t *testing.T) { if testing.Short() { t.Skip("too slow for testing.Short") } @@ -74,7 +78,7 @@ func TestUiIndex(t *testing.T) { } } -func TestUiNodes(t *testing.T) { +func TestUINodes(t *testing.T) { if testing.Short() { t.Skip("too slow for testing.Short") } @@ -84,15 +88,42 @@ func TestUiNodes(t *testing.T) { defer a.Shutdown() testrpc.WaitForTestAgent(t, a.RPC, "dc1") - args := &structs.RegisterRequest{ - Datacenter: "dc1", - Node: "test", - Address: "127.0.0.1", + args := []*structs.RegisterRequest{ + { + Datacenter: "dc1", + Node: "test", + Address: "127.0.0.1", + }, + { + Datacenter: "dc1", + Node: "foo-peer", + Address: "127.0.0.3", + PeerName: "peer1", + }, } - var out struct{} - if err := a.RPC("Catalog.Register", args, &out); err != nil { - t.Fatalf("err: %v", err) + for _, reg := range args { + var out struct{} + err := a.RPC("Catalog.Register", reg, &out) + require.NoError(t, err) + } + + // establish "peer1" + { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + + peerOne := &pbpeering.PeeringWriteRequest{ + Peering: &pbpeering.Peering{ + Name: "peer1", + State: pbpeering.PeeringState_INITIAL, + PeerCAPems: nil, + PeerServerName: "fooservername", + PeerServerAddresses: []string{"addr1"}, + }, + } + _, err := a.rpcClientPeering.PeeringWrite(ctx, peerOne) + require.NoError(t, err) } req, _ := http.NewRequest("GET", "/v1/internal/ui/nodes/dc1", nil) @@ -103,20 +134,32 @@ func TestUiNodes(t *testing.T) { } assertIndex(t, resp) - // Should be 2 nodes, and all the empty lists should be non-nil + // Should be 3 nodes, and all the empty lists should be non-nil nodes := obj.(structs.NodeDump) - if len(nodes) != 2 || - nodes[0].Node != a.Config.NodeName || - nodes[0].Services == nil || len(nodes[0].Services) != 1 || - nodes[0].Checks == nil || len(nodes[0].Checks) != 1 || - nodes[1].Node != "test" || - nodes[1].Services == nil || len(nodes[1].Services) != 0 || - nodes[1].Checks == nil || len(nodes[1].Checks) != 0 { - t.Fatalf("bad: %v", obj) - } + require.Len(t, nodes, 3) + + // check local nodes, services and checks + require.Equal(t, a.Config.NodeName, nodes[0].Node) + require.NotNil(t, nodes[0].Services) + require.Len(t, nodes[0].Services, 1) + require.NotNil(t, nodes[0].Checks) + require.Len(t, nodes[0].Checks, 1) + require.Equal(t, "test", nodes[1].Node) + require.NotNil(t, nodes[1].Services) + require.Len(t, nodes[1].Services, 0) + require.NotNil(t, nodes[1].Checks) + require.Len(t, nodes[1].Checks, 0) + + // peered node + require.Equal(t, "foo-peer", nodes[2].Node) + require.Equal(t, "peer1", nodes[2].PeerName) + require.NotNil(t, nodes[2].Services) + require.Len(t, nodes[2].Services, 0) + require.NotNil(t, nodes[1].Checks) + require.Len(t, nodes[2].Services, 0) } -func TestUiNodes_Filter(t *testing.T) { +func TestUINodes_Filter(t *testing.T) { if testing.Short() { t.Skip("too slow for testing.Short") } @@ -162,7 +205,7 @@ func TestUiNodes_Filter(t *testing.T) { require.Empty(t, nodes[0].Checks) } -func TestUiNodeInfo(t *testing.T) { +func TestUINodeInfo(t *testing.T) { if testing.Short() { t.Skip("too slow for testing.Short") } @@ -214,7 +257,7 @@ func TestUiNodeInfo(t *testing.T) { } } -func TestUiServices(t *testing.T) { +func TestUIServices(t *testing.T) { if testing.Short() { t.Skip("too slow for testing.Short") } @@ -318,6 +361,30 @@ func TestUiServices(t *testing.T) { Tags: []string{}, }, }, + // register peer node foo with peer service + { + Datacenter: "dc1", + Node: "foo", + ID: types.NodeID("e0155642-135d-4739-9853-a1ee6c9f945b"), + Address: "127.0.0.2", + TaggedAddresses: map[string]string{ + "lan": "127.0.0.2", + "wan": "198.18.0.2", + }, + NodeMeta: map[string]string{ + "env": "production", + "os": "linux", + }, + PeerName: "peer1", + Service: &structs.NodeService{ + Kind: structs.ServiceKindTypical, + ID: "serviceID", + Service: "service", + Port: 1235, + Address: "198.18.1.2", + PeerName: "peer1", + }, + }, } for _, args := range requests { @@ -325,6 +392,24 @@ func TestUiServices(t *testing.T) { require.NoError(t, a.RPC("Catalog.Register", args, &out)) } + // establish "peer1" + { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + + peerOne := &pbpeering.PeeringWriteRequest{ + Peering: &pbpeering.Peering{ + Name: "peer1", + State: pbpeering.PeeringState_INITIAL, + PeerCAPems: nil, + PeerServerName: "fooservername", + PeerServerAddresses: []string{"addr1"}, + }, + } + _, err := a.rpcClientPeering.PeeringWrite(ctx, peerOne) + require.NoError(t, err) + } + // Register a terminating gateway associated with api and cache { arg := structs.RegisterRequest{ @@ -393,7 +478,7 @@ func TestUiServices(t *testing.T) { // Should be 2 nodes, and all the empty lists should be non-nil summary := obj.([]*ServiceListingSummary) - require.Len(t, summary, 6) + require.Len(t, summary, 7) // internal accounting that users don't see can be blown away for _, sum := range summary { @@ -493,6 +578,21 @@ func TestUiServices(t *testing.T) { EnterpriseMeta: *structs.DefaultEnterpriseMetaInDefaultPartition(), }, }, + { + ServiceSummary: ServiceSummary{ + Kind: structs.ServiceKindTypical, + Name: "service", + Datacenter: "dc1", + Tags: nil, + Nodes: []string{"foo"}, + InstanceCount: 1, + ChecksPassing: 0, + ChecksWarning: 0, + ChecksCritical: 0, + EnterpriseMeta: *structs.DefaultEnterpriseMetaInDefaultPartition(), + PeerName: "peer1", + }, + }, } require.ElementsMatch(t, expected, summary) }) From 2c89485870d0f00dff299b0786926059908eabba Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Lo=C3=AFc=20Saint-Roch?= Date: Sun, 26 Jun 2022 15:50:25 +0200 Subject: [PATCH 148/149] Add HashiBox to community tools --- website/content/docs/download-tools.mdx | 1 + 1 file changed, 1 insertion(+) diff --git a/website/content/docs/download-tools.mdx b/website/content/docs/download-tools.mdx index 16fca097f..51da01366 100644 --- a/website/content/docs/download-tools.mdx +++ b/website/content/docs/download-tools.mdx @@ -55,6 +55,7 @@ These Consul tools are created and managed by the amazing members of the Consul - [Gonsul](https://github.com/miniclip/gonsul) - A Git to Consul standalone tool made in Go. Updates Consul KV from a repo with multiple strategies. - [gradle-consul-plugin](https://github.com/amirkibbar/red-apple) - A Consul Gradle plugin - [hashi-ui](https://github.com/jippi/hashi-ui) - A modern user interface for the Consul and Nomad +- [HashiBox](https://github.com/nunchistudio/hashibox) - Vagrant environment to simulate highly-available cloud with Consul, Nomad, Vault, and optional support for Waypoint. OSS & Enterprise supported. - [helios-consul](https://github.com/SVT/helios-consul) - Service registrar plugin for Helios - [Jenkins Consul Plugin](https://plugins.jenkins.io/consul) - Jenkins plugin for service discovery and K/V store - [marathon-consul](https://github.com/allegro/marathon-consul) - Service registry bridge for Marathon From cc2bcb2b9fd6d9be48a6e7dcad38aadcf89d3be0 Mon Sep 17 00:00:00 2001 From: Kyle Havlovitz Date: Mon, 27 Jun 2022 09:33:27 -0700 Subject: [PATCH 149/149] Update docs for prometheus TLS options --- website/content/commands/connect/envoy.mdx | 16 ++++++++++++++++ .../content/docs/k8s/annotations-and-labels.mdx | 12 ++++++++++++ 2 files changed, 28 insertions(+) diff --git a/website/content/commands/connect/envoy.mdx b/website/content/commands/connect/envoy.mdx index d453a7cbf..88f7fccb7 100644 --- a/website/content/commands/connect/envoy.mdx +++ b/website/content/commands/connect/envoy.mdx @@ -96,6 +96,22 @@ proxy configuration needed. be scrapable at `0.0.0.0:20200/scrape-metrics`. Only applicable when `envoy_prometheus_bind_addr` is set in proxy config. +- `-prometheus-ca-file` - Path to a CA file for Envoy to use when serving TLS on + the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr` + is set in proxy config. + +- `-prometheus-ca-path` - Path to a directory of CA certificates for Envoy to use when + serving the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr` + is set in proxy config. + +- `-prometheus-cert-file` - Path to a certificate file for Envoy to use when serving + TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr` + is set in proxy config. + +- `-prometheus-key-file` - Path to a private key file for Envoy to use when serving + TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr` + is set in proxy config. + - `-- [pass-through options]` - Any options given after a double dash are passed directly through to the `envoy` invocation. See [Envoy's documentation](https://www.envoyproxy.io/docs) for more details. The command diff --git a/website/content/docs/k8s/annotations-and-labels.mdx b/website/content/docs/k8s/annotations-and-labels.mdx index d08b82ee6..2eba477f0 100644 --- a/website/content/docs/k8s/annotations-and-labels.mdx +++ b/website/content/docs/k8s/annotations-and-labels.mdx @@ -218,6 +218,18 @@ The following Kubernetes resource annotations could be used on a pod to control - `consul.hashicorp.com/merged-metrics-port` - Override the default Helm value [`connectInject.metrics.defaultMergedMetricsPort`](/docs/k8s/helm#v-connectinject-metrics-defaultmergedmetricsport). - `consul.hashicorp.com/prometheus-scrape-port` - Override the default Helm value [`connectInject.metrics.defaultPrometheusScrapePort`](/docs/k8s/helm#v-connectinject-metrics-defaultprometheusscrapeport). - `consul.hashicorp.com/prometheus-scrape-path` - Override the default Helm value [`connectInject.metrics.defaultPrometheusScrapePath`](/docs/k8s/helm#v-connectinject-metrics-defaultprometheusscrapepath). +- `consul.hashicorp.com/prometheus-ca-file` - Local filesystem path to a CA file for Envoy to use + when serving TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr` + is set in proxy config. +- `consul.hashicorp.com/prometheus-ca-path` - Local filesystem path to a directory of CA certificates + for Envoy to use when serving TLS on the Prometheus metrics endpoint. Only applicable when + `envoy_prometheus_bind_addr` is set in proxy config. +- `consul.hashicorp.com/prometheus-cert-file` - Local filesystem path to a certificate file for Envoy to use + when serving TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr` + is set in proxy config. +- `consul.hashicorp.com/prometheus-key-file` - Local filesystem path to a private key file for Envoy to use + when serving TLS on the Prometheus metrics endpoint. Only applicable when `envoy_prometheus_bind_addr` + is set in proxy config. - `consul.hashicorp.com/service-metrics-port` - Set the port where the Connect service exposes metrics. - `consul.hashicorp.com/service-metrics-path` - Set the path where the Connect service exposes metrics. - `consul.hashicorp.com/connect-inject-mount-volume` - Comma separated list of container names to mount the connect-inject volume into. The volume will be mounted at `/consul/connect-inject`. The connect-inject volume contains Consul internals data needed by the other sidecar containers, for example the `consul` binary, and the Pod's Consul ACL token. This data can be valuable for advanced use-cases, such as making requests to the Consul API from within application containers.