Merge pull request #14011 from hashicorp/docs/cluster-peering-beta

docs: Cluster Peering beta
This commit is contained in:
Jeff Boruszak 2022-08-09 13:24:49 -05:00 committed by GitHub
commit e1287a41b0
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 286 additions and 70 deletions

View File

@ -7,24 +7,35 @@ description: >-
# Create and Manage Peering Connections
~> **Cluster peering is currently in technical preview:** Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support.
~> **Cluster peering is currently in beta:** Functionality associated with cluster peering is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may have performance issues, scaling issues, and limited support.
A peering token enables cluster peering between different datacenters. Once you generate a peering token, you can use it to establish a connection between clusters. Then you can export services and authorize other clusters to call those services.
A peering token enables cluster peering between different datacenters. Once you generate a peering token, you can use it to establish a connection between clusters. Then you can export services and create intentions so that peered clusters can call those services.
To peer clusters, you must complete the following steps in order:
## Create a peering connection
Cluster peering is not enabled by default on Consul servers. To peer clusters, you must first configure all Consul servers so that `peering` is `enabled`. For additional information, refer to [Configuration Files](/docs/agent/config/config-files).
After enabling peering for all Consul servers, complete the following steps in order:
1. Create a peering token
1. Establish a connection between clusters
1. Export service endpoints
1. Authorize connections between peers
1. Export services between clusters
1. Authorize services for peers
## Create a peering token
You can generate peering tokens and initiate connections on any available agent using either the API or the Consul UI. If you use the API, we recommend performing these operations through a client agent in the partition you want to connect.
You can generate peering tokens and initiate connections using the Consul API on any available agent. However, we recommend performing these operations through a client agent in the partition you want to connect.
The UI does not currently support exporting services between clusters or authorizing services for peers.
### Create a peering token
To begin the cluster peering process, generate a peering token in one of your clusters. The other cluster uses this token to establish the peering connection.
In `cluster-01`, issue a request for a peering token using the [HTTP API](/api-docs/peering).
Every time you generate a peering token, a single-use establishment secret is embedded in the token. Because regenerating a peering token invalidates the previously generated secret, you must use the most recently created token to establish peering connections.
<Tabs>
<Tab heading="Consul API">
In `cluster-01`, use the [`/peering/token` endpoint](/api-docs/peering#generate-a-peering-token) to issue a request for a peering token.
```shell-session
$ curl --request POST --data '{"PeerName":"cluster-02"}' --url http://localhost:8500/v1/peering/token
@ -44,10 +55,27 @@ Create a JSON file that contains the first cluster's name and the peering token.
```
</CodeBlockConfig>
</Tab>
## Establish a connection between clusters
<Tab heading="Consul UI">
Next, use `peering_token.json` to establish a secure connection between the clusters. In the client agents of "cluster-02," establish the peering connection using the HTTP API. This endpoint does not generate an output unless there is an error.
1. In the Consul UI for the datacenter associated with `cluster-01`, click **Peers**.
1. Click **Add peer connection**.
1. In the **Generate token** tab, enter `cluster-02` in the **Name of peer** field.
1. Click the **Generate token** button.
1. Copy the token before you proceed. Be careful not to lose the token, as you cannot view the token again after leaving this screen. If you lose your token, you must generate a new one.
</Tab>
</Tabs>
### Establish a connection between clusters
Next, use the peering token to establish a secure connection between the clusters.
<Tabs>
<Tab heading="Consul API">
In one of the client agents in "cluster-02," use `peering_token.json` and the [`/peering/establish` endpoint](/api-docs/peering#establish-a-peering-connection) to establish the peering connection. This endpoint does not generate an output unless there is an error.
```shell-session
$ curl --request POST --data @peering_token.json http://127.0.0.1:8500/v1/peering/establish
@ -55,7 +83,22 @@ $ curl --request POST --data @peering_token.json http://127.0.0.1:8500/v1/peerin
When you connect server agents through cluster peering, they peer their default partitions. To establish peering connections for other partitions through server agents, you must add the `Partition` field to `peering_token.json` and specify the partitions you want to peer. For additional configuration information, refer to [Cluster Peering - HTTP API](/api-docs/peering).
## Export service endpoints
You can dial the `peering/establish` endpoint once per peering token. Peering tokens cannot be reused after being used to establish a connection. If you need to re-establish a connection, you must generate a new peering token.
</Tab>
<Tab heading="Consul UI">
1. In the Consul UI for the datacenter associated with `cluster 02`, click **Peers** and then **Add peer connection**.
1. Click **Establish peering**.
1. In the **Name of peer** field, enter `cluster-01`. Then paste the peering token in the **Token** field.
1. Click **Add peer**.
</Tab>
</Tabs>
### Export services between clusters
After you establish a connection between the clusters, you need to create a configuration entry that defines the services that are available for other clusters. Consul uses this configuration entry to advertise service information and support service mesh connections across clusters.
@ -79,6 +122,7 @@ Services = [
## during the peering process.
Peer = "cluster-02"
}
}
]
```
@ -92,7 +136,7 @@ $ consul config write peering-config.hcl
Before you proceed, wait for the clusters to sync and make services available to their peers. You can issue an endpoint query to [check the peered cluster status](#check-peered-cluster-status).
## Authorize connections from peers
### Authorize services for peers
Before you can call services from peered clusters, you must set service intentions that authorize those clusters to use specific services. Consul prevents services from being exported to unauthorized clusters.
@ -123,23 +167,124 @@ Then, add the configuration entry to your cluster.
$ consul config write peering-intentions.hcl
```
## Check peered cluster status
## Manage peering connections
To confirm that you peered your clusters, you can [query the `/health/service` endpoint](/api-docs/health) of one cluster from the other cluster. For example, in "cluster-02," query the endpoint and add the `peer=cluster-01` query parameter to the end of the URL.
After you establish a peering connection, you can get a list of all active peering connections, read a specific peering connection's information, check peering connection health, and delete peering connections.
### List all peering connections
You can list all active peering connections in a cluster.
<Tabs>
<Tab heading="Consul API">
After you establish a peering connection, [query the `/peerings/` endpoint](/api-docs/peering#list-all-peerings) to get a list of all peering connections. For example, the following command requests a list of all peering connections on `localhost` and returns the information as a series of JSON objects:
```shell-session
$ curl http://127.0.0.1:8500/v1/peerings
[
{
"ID": "462c45e8-018e-f19d-85eb-1fc1bcc2ef12",
"Name": "cluster-02",
"State": "ACTIVE",
"Partition": "default",
"PeerID": "e83a315c-027e-bcb1-7c0c-a46650904a05",
"PeerServerName": "server.dc1.consul",
"PeerServerAddresses": [
"10.0.0.1:8300"
],
"CreateIndex": 89,
"ModifyIndex": 89
},
{
"ID": "1460ada9-26d2-f30d-3359-2968aa7dc47d",
"Name": "cluster-03",
"State": "INITIAL",
"Partition": "default",
"Meta": {
"env": "production"
},
"CreateIndex": 109,
"ModifyIndex": 119
},
]
```
</Tab>
<Tab heading="Consul UI">
In the Consul UI, click **Peers**. The UI lists peering connections you created for clusters in a datacenter.
The name that appears in the list is the name of the cluster in a different datacenter with an established peering connection.
</Tab>
</Tabs>
### Read a peering connection
You can get information about individual peering connections between clusters.
<Tabs>
<Tab heading="Consul API">
After you establish a peering connection, [query the `/peering/` endpoint](/api-docs/peering#read-a-peering-connection) to get peering information about for a specific cluster. For example, the following command requests peering connection information for "cluster-02" and returns the info as a JSON object:
```shell-session
$ curl http://127.0.0.1:8500/v1/peering/cluster-02
{
"ID": "462c45e8-018e-f19d-85eb-1fc1bcc2ef12",
"Name": "cluster-02",
"State": "INITIAL",
"PeerID": "e83a315c-027e-bcb1-7c0c-a46650904a05",
"PeerServerName": "server.dc1.consul",
"PeerServerAddresses": [
"10.0.0.1:8300"
],
"CreateIndex": 89,
"ModifyIndex": 89
}
```
</Tab>
<Tab heading="Consul UI">
In the Consul UI, click **Peers**. The UI lists peering connections you created for clusters in that datacenter. Click the name of a peered cluster to view additional details about the peering connection.
</Tab>
</Tabs>
### Check peering connection health
You can check the status of your peering connection to perform health checks.
To confirm that the peering connection between your clusters remains healthy, query the [`health/service` endpoint](/api-docs/health) of one cluster from the other cluster. For example, in "cluster-02," query the endpoint and add the `peer=cluster-01` query parameter to the end of the URL.
```shell-session
$ curl \
"http://127.0.0.1:8500/v1/health/service/<service-name>?peer=cluster-01"
```
A successful query will include service information in the output.
A successful query includes service information in the output.
## Remove peering connections
### Delete peering connections
After you create a peering connection between clusters in different datacenters, you can disconnect the peered clusters. Deleting a peering connection stops data replication to the peer and deletes imported data, including services and CA certificates.
You can disconnect the peered clusters by deleting their connection. Deleting a peering connection stops data replication to the peer and deletes imported data, including services and CA certificates.
In "cluster-01," request the deletion via the HTTP API.
<Tabs>
<Tab heading="Consul API">
In "cluster-01," request the deletion through the [`/peering/ endpoint`](/api-docs/peering#delete-a-peering-connection).
```shell-session
$ curl --request DELETE http://127.0.0.1:8500/v1/peering/cluster-02
```
</Tab>
<Tab heading="Consul UI">
In the Consul UI, click **Peers**. The UI lists peering connections you created for clusters in that datacenter.
Next to the name of the peer, click **More** (three horizontal dots) and then **Delete**. Click **Delete** to confirm and remove the peering connection.
</Tab>
</Tabs>

View File

@ -7,18 +7,19 @@ description: >-
# What is Cluster Peering?
~> **Cluster peering is currently in technical preview**: Functionality associated with cluster peering is subject to change. You should never use the technical preview release in secure environments or production scenarios. Features in technical preview may have performance issues, scaling issues, and limited support.
~> **Cluster peering is currently in beta**: Functionality associated with cluster peering is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may have performance issues, scaling issues, and limited support.
You can create peering connections between two or more independent clusters so that services deployed to different partitions or datacenters can communicate.
## Overview
Cluster peering allows Consul clusters in different datacenters to communicate with each other. The cluster peering process consists of the following steps:
1. Create a peering token to share with other clusters
1. Establish a connection between clusters
1. Make services available to other clusters
1. Create a peering token in one cluster.
1. Use the peering token to establish peering with a second cluster.
1. Export services between clusters.
1. Create intentions to authorize services for peers.
For detailed instructions on setting up cluster peering with the Consul CLI, refer to [Create and Manage Peering Connections](/docs/connect/cluster-peering/create-manage-peering).
For detailed instructions on setting up cluster peering, refer to [Create and Manage Peering Connections](/docs/connect/cluster-peering/create-manage-peering).
### Differences between WAN federation and cluster peering
@ -26,25 +27,33 @@ WAN federation and cluster peering are different ways to connect clusters. The m
Regardless of whether you connect your clusters through WAN federation or cluster peering, human and machine users can use either method to discover services in other clusters or dial them through the service mesh.
| | WAN Federation | Cluster Peering |
| :----------------------------------------------- | :------------: | :-------------: |
| Connects clusters across datacenters | &#9989; | &#9989; |
| Shares support queries and service endpoints | &#9989; | &#9989; |
| Connects clusters owned by different operators | &#10060; | &#9989; |
| Functions without declaring primary datacenter | &#10060; | &#9989; |
| Shares key/value stores | &#9989; | &#10060; |
| Uses gossip protocol | &#9989; | &#10060; |
| | WAN Federation | Cluster Peering |
| :------------------------------------------------- | :------------: | :-------------: |
| Connects clusters across datacenters | &#9989; | &#9989; |
| Shares support queries and service endpoints | &#9989; | &#9989; |
| Connects clusters owned by different operators | &#10060; | &#9989; |
| Functions without declaring primary datacenter | &#10060; | &#9989; |
| Replicates exported services for service discovery | &#10060; | &#9989; |
| Forwards service requests for service discovery | &#9989; | &#10060; |
| Shares key/value stores | &#9989; | &#10060; |
| Uses gossip protocol | &#9989; | &#10060; |
## Technical preview constraints
Not all features and functionality are available in the technical preview release. In particular, consider the following technical constraints:
## Beta release features and constraints
- Consul ACLs must be disabled or the ACL `default_policy` must be set to `allow`.
- Mesh gateways for _server to server traffic_ are not available. However, mesh gateways for _service to service traffic_ between clusters are available.
- Services exported to peered clusters must be configured to use the TCP protcol (not HTTP, HTTP 2 and gRPC).
- Support for dynamic routing such as splits, custom routes, or redirects is not available.
- The `consul intention CLI` command is not supported. To manage intentions that specify services in peered clusters, use [configuration entries](/docs/connect/config-entries/service-intentions).
- [L7 permissions](/docs/connect/l7-traffic) are not supported.
- Configuring service failover across peers is not supported.
The cluster peering beta includes the following features and functionality:
- Mesh gateways for _service to service traffic_ between clusters are available. For more information on configuring mesh gateways across peers, refer to [Service-to-service Traffic Across Peered Clusters](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-peers).
- You can generate peering tokens, establish, list, read, and delete peerings, and manage intentions for peering connections with both the API and the UI.
- You can configure [transparent proxies](/docs/connect/transparent-proxy) for peered services.
- You can use the [`peering` rule for ACL enforcement](/docs/security/acl/acl-rules#peering) of peering APIs.
Not all features and functionality are available in the beta release. In particular, consider the following technical constraints:
- Mesh gateways for _server to server traffic_ are not available.
- Services with node, instance, and check definitions totaling more than 4MB cannot be exported to a peer.
- Dynamic routing features such as splits, custom routes, and redirects cannot target services in a peered cluster.
- Configuring service failover across peers is not supported for service mesh.
- Consul datacenters that are already federated stay federated. You do not need to migrate WAN federated clusters to cluster peering.
- The `consul intention` CLI command is not supported. To manage intentions that specify services in peered clusters, use [configuration entries](/docs/connect/config-entries/service-intentions).
- Accessing key/value stores across peers is not supported.
- Consul datacenters that are already federated stay federated.
- Non-enterprise Consul instances cannot sync services with namespaces outside of the `default` namespace.
- Because non-Enterprise Consul instances are restricted to the `default` namespace, Consul Enterprise instances cannot export services from outside of the `default` namespace to non-Enterprise peers.

View File

@ -7,14 +7,13 @@ description: >-
# Cluster Peering on Kubernetes
~> **Cluster peering is currently in technical preview:** Functionality associated
with cluster peering is subject to change. You should never use the technical
preview release in secure environments or production scenarios. Features in
technical preview may have performance issues, scaling issues, and limited support.
~> **Cluster peering is currently in beta:** Functionality associated
with cluster peering is subject to change. You should never use the beta release in secure environments or production scenarios. Features in
beta may have performance issues, scaling issues, and limited support.
To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart and create custom resource definitions for each side of the peering.
To establish a cluster peering connection on Kubernetes, you need to enable the feature in the Helm chart and create custom resource definitions (CRDs) for each side of the peering.
The following Custom Resource Definitions (CRDs) are used to create and manage a peering connection:
The following CRDs are used to create and manage a peering connection:
- `PeeringAcceptor`: Generates a peering token and accepts an incoming peering connection.
- `PeeringDialer`: Uses a peering token to make an outbound peering connection with the cluster that generated the token.
@ -22,10 +21,10 @@ The following Custom Resource Definitions (CRDs) are used to create and manage a
## Prerequisites
You must implement the following requirements to create and use cluster peering connections with Kubernetes:
- Consul 1.13 Alpha 2 or later
- Consul version 1.13.0 or later
- At least two Kubernetes clusters
- The Kubernetes clusters must be running in a flat network
- The network must be running on Consul on Kubernetes v.0.45 or later
- The network must be running on Consul on Kubernetes version 0.45 or later
### Helm chart configuration
@ -35,7 +34,7 @@ To establish cluster peering through Kubernetes, deploy clusters with the follow
```yaml
global:
image: "hashicorp/consul:1.13.0-alpha2"
image: "hashicorp/consul:1.13.0"
peering:
enabled: true
connectInject:
@ -59,7 +58,7 @@ $ export HELM_RELEASE_NAME=cluster-name
$ helm install ${HELM_RELEASE_NAME} hashicorp/consul --version "0.45.0" --values values.yaml
```
## Create a peering connection
## Create a peering token
To peer Kubernetes clusters running Consul, you need to create a peering token and share it with the other cluster.
@ -94,6 +93,8 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a
$ kubectl get secret peering-token --output yaml > peering-token.yml
```
## Establish a peering connection between clusters
1. Apply the peering token to the second cluster.
```shell-session
@ -125,9 +126,9 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a
$ kubectl apply --filename dialer.yml
```
## Deploy and export cluster services
## Export services between clusters
1. For the service in "cluster-02" that you want to export, add the following [annotations](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams) to your service's pods. This service is referred to as "backend-service" in the following steps.
1. For the service in "cluster-02" that you want to export, add the following [annotations](/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service-upstreams) to your service's pods.
<CodeBlockConfig filename="backend-service.yml">
@ -159,6 +160,14 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a
</CodeBlockConfig>
1. Apply the service file and the `ExportedServices` resource for the second cluster.
```shell-session
$ kubectl apply --filename backend-service.yml --filename exportedsvc.yml
```
## Authorize services for peers
1. Create service intentions for the second cluster.
<CodeBlockConfig filename="intention.yml">
@ -180,16 +189,10 @@ To peer Kubernetes clusters running Consul, you need to create a peering token a
</CodeBlockConfig>
1. Apply the service file, the `ExportedServices` resource, and the intentions to the second cluster.
1. Apply the intentions to the second cluster.
```shell-session
$ kubectl apply --filename backend-service.yml --filename exportedsvc.yml --filename intention.yml
```
1. To confirm that you peered your clusters, in `cluster-01`, query the `/health` HTTP endpoint.
```shell-session
$ curl "localhost:8500/v1/health/connect/backend?peer=cluster-02"
$ kubectl apply --filename intention.yml
```
1. For the services in `cluster-01` that you want to access the "backend-service," add the following annotations to the service file.

View File

@ -0,0 +1,55 @@
---
layout: docs
page_title: Service-to-service Traffic Across Peered Clusters
description: >-
This topic describes how to configure mesh gateways to route a service's data to upstreams
in clusters that have a peering connection.
---
# Service-to-service Traffic Across Peered Clusters
~> **Cluster peering is currently in beta**: Functionality associated with cluster peering is subject to change. You should never use the beta release in secure environments or production scenarios. Features in beta may have performance issues, scaling issues, and limited support.
Mesh gateways are required for you to route service mesh traffic between different Consul clusters. Clusters can reside in different clouds or runtime environments where general interconnectivity between all services in all clusters is not feasible.
Unlike mesh gateways for datacenters and partitions, mesh gateways for cluster peering decrypt data to HTTP services within the mTLS session. Data must be decrypted in order to evaluate and apply dynamic routing rules at the destination cluster, which reduces coupling between peers.
## Prerequisites
To configure mesh gateways for cluster peering, make sure your Consul environment meets the following requirements:
- Consul version 1.13.0 or newer.
- A local Consul agent is required to manage mesh gateway configuration.
- [Enable Consul service mesh](/docs/agent/config/config-files#connect-parameters) in all clusters.
- [Enable `peering`](/docs/agent/config/config-files) on all Consul servers.
- Use [Envoy proxies](/docs/connect/proxies/envoy). Envoy is the only proxy with mesh gateway capabilities in Consul.
## Configuration
Configure the following settings to register and use the mesh gateway as a service in Consul.
### Gateway registration
- Specify `mesh-gateway` in the `kind` field to register the gateway with Consul.
- Define the `Proxy.Config` settings using opaque parameters compatible with your proxy. For Envoy, refer to the [Gateway Options](/docs/connect/proxies/envoy#gateway-options) and [Escape-hatch Overrides](/docs/connect/proxies/envoy#escape-hatch-overrides) documentation for additional configuration information.
Alternatively, you can also use the CLI to spin up and register a gateway in Consul. For additional information, refer to the [`consul connect envoy` command](/commands/connect/envoy#mesh-gateways).
### Sidecar registration
- Configure the `proxy.upstreams` parameters to route traffic to the correct service, namespace, and peer. Refer to the [`upstreams` documentation](/docs/connect/registration/service-registration#upstream-configuration-reference) for details.
- The service `proxy.upstreams.destination_name` is always required.
- The `proxy.upstreams.destination_peer` must be configured to enable cross-cluster traffic.
- The `proxy.upstream/destination_namespace` configuration is only necessary if the destination service is in a non-default namespace.
### Service exports
- Include the `exported-services` configuration entry to enable Consul to export services contained in a cluster to one or more additional clusters. For additional information, refer to the [Exported Services documentation](/docs/connect/config-entries/exported-services).
### ACL configuration
- If ACLs are enabled, you must add a token granting `service:write` for the gateway's service name and `service:read` for all services in the Enterprise admin partition or OSS datacenter to the gateway's service definition. These permissions authorize the token to route communications for other Consul service mesh services.
### Modes
Modes are not configurable for mesh gateways that connect peered clusters. By default, all proxies connecting to peered clusters use mesh gateways in [remote mode](/docs/connect/gateways/mesh-gateway/service-to-service-traffic-datacenters#remote).

View File

@ -267,6 +267,10 @@
{
"title": "Enabling Service-to-service Traffic Across Admin Partitions",
"path": "connect/gateways/mesh-gateway/service-to-service-traffic-partitions"
},
{
"title": "Enabling Service-to-service Traffic Across Peered Clusters",
"path": "connect/gateways/mesh-gateway/service-to-service-traffic-peers"
}
]
},
@ -284,7 +288,7 @@
"title": "Cluster Peering",
"routes": [
{
"title": "What is Cluster Peering",
"title": "What is Cluster Peering?",
"path": "connect/cluster-peering"
},
{