From 1758a6dc54887b6d9c3d378e3d90554e7256fd65 Mon Sep 17 00:00:00 2001 From: Iryna Shustava Date: Fri, 16 Apr 2021 12:49:02 -0700 Subject: [PATCH] docs: update helm ref docs and connect docs (#10032) All k8s connect-related docs now need to mention that we require a Kubernetes service for all Connect services --- website/content/docs/k8s/connect/health.mdx | 3 + website/content/docs/k8s/connect/index.mdx | 193 ++++++++++-------- .../docs/k8s/connect/ingress-gateways.mdx | 72 ++++--- .../docs/k8s/connect/terminating-gateways.mdx | 18 +- website/content/docs/k8s/helm.mdx | 76 ++++--- 5 files changed, 208 insertions(+), 154 deletions(-) diff --git a/website/content/docs/k8s/connect/health.mdx b/website/content/docs/k8s/connect/health.mdx index 1b94f3bd8..52d2a4244 100644 --- a/website/content/docs/k8s/connect/health.mdx +++ b/website/content/docs/k8s/connect/health.mdx @@ -9,6 +9,9 @@ description: Configuring Kubernetes Health Checks -> 0.26+: This feature is available in consul-helm versions 0.26 and higher and is defaulted on. To disable it, set `connectInject.healthChecks.enabled: false`. +-> **Note**: As of consul-k8s `v0.26.0-beta1` and Consul Helm `v0.32.0-beta1`, syncing of the Kubernetes readiness status +is always on, and turning it off is no longer configurable. + ~> This topic requires familiarity with [Kubernetes Health Checks](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/). This page describes how to enable Consul on Kubernetes to sync the Kubernetes readiness status to Consul for service mesh uses cases. diff --git a/website/content/docs/k8s/connect/index.mdx b/website/content/docs/k8s/connect/index.mdx index 565987001..f804aa6d1 100644 --- a/website/content/docs/k8s/connect/index.mdx +++ b/website/content/docs/k8s/connect/index.mdx @@ -31,57 +31,87 @@ accept and establish connections using Connect, enabling the pod to communicate to clients and dependencies exclusively over authorized and encrypted connections. --> **Note:** The pod specifications in this section are valid and use +-> **Note:** The examples in this section are valid and use publicly available images. If you've installed the Connect injector, feel free -to run the pod specifications in this section to try Connect with Kubernetes. +to run the examples in this section to try Connect with Kubernetes. Please note the documentation below this section on how to properly install and configure the Connect injector. ### Accepting Inbound Connections -An example pod is shown below with Connect enabled to accept inbound -connections. Notice that the pod would still be fully functional without -Connect. Minimal to zero modifications are required to pod specifications to -enable Connect in Kubernetes. +An example Deployment is shown below with Connect enabled to accept inbound +connections. Notice that the Deployment would still be fully functional without +Connect. Minimal to zero modifications are required to enable Connect in Kubernetes. +Notice also that even though we're using a Deployment here, the same configuration +would work on a Pod, a StatefulSet, or a DaemonSet. -This pod specification starts a server that responds to any +This Deployment specification starts a server that responds to any HTTP request with the static text "hello world". +-> **Note:** As of consul-k8s `v0.26.0-beta1` and Consul Helm `v0.32.0-beta1`, having a Kubernetes +service is **required** to run services on the Consul Service Mesh. + ```yaml apiVersion: v1 +kind: Service +metadata: + name: static-server +spec: + selector: + app: static-server + ports: + - protocol: TCP + port: 80 + targetPort: 8080 +--- +apiVersion: v1 kind: ServiceAccount metadata: name: static-server --- -apiVersion: v1 -kind: Pod +apiVersion: apps/v1 +kind: Deployment metadata: name: static-server - annotations: - 'consul.hashicorp.com/connect-inject': 'true' spec: - containers: - # This name will be the service name in Consul. - - name: static-server - image: hashicorp/http-echo:latest - args: - - -text="hello world" - - -listen=:8080 - ports: - - containerPort: 8080 - name: http - # If ACLs are enabled, the serviceAccountName must match the Consul service name. - serviceAccountName: static-server + replicas: 1 + selector: + matchLabels: + app: static-server + template: + metadata: + name: static-server + labels: + app: static-server + annotations: + 'consul.hashicorp.com/connect-inject': 'true' + spec: + containers: + # This name will be the service name in Consul. + - name: static-server + image: hashicorp/http-echo:latest + args: + - -text="hello world" + - -listen=:8080 + ports: + - containerPort: 8080 + name: http + # If ACLs are enabled, the serviceAccountName must match the Consul service name. + serviceAccountName: static-server ``` The only change for Connect is the addition of the `consul.hashicorp.com/connect-inject` annotation. This enables injection -for this pod. The injector can also be +for the Pod in this Deployment. The injector can also be [configured](/docs/k8s/connect#installation-and-configuration) to automatically inject unless explicitly disabled, but the default installation requires opt-in using the annotation shown above. -This will start a Connect sidecar that listens on a random port registered +~> **A common mistake** is to set the annotation on the Deployment or +other resource. Ensure that the injector annotations are specified on +the _pod specification template_ as shown above. + +This will start a sidecar proxy that listens on port `20000` registered with Consul and proxies valid inbound connections to port 8080 in the pod. To establish a connection to the pod using Connect, a client must use another Connect proxy. The client Connect proxy will use Consul service discovery to find @@ -93,40 +123,67 @@ This is useful to transition to Connect by allowing both Connect and non-Connect connections. To restrict access to only Connect-authorized clients, any listeners should bind to localhost only (such as `127.0.0.1`). -The service name registered in Consul will be set to the name of the first -container in the Pod. This can be customized with the `consul.hashicorp.com/connect-service` +-> **Note:** As of consul `v1.10.0-beta1`, consul-k8s `v0.26.0-beta1` and Consul Helm `v0.32.0-beta1`, +all Consul Service Mesh services will run with transparent proxy enabled by default. Running with transparent +proxy will enforce all inbound and outbound traffic to go through the Envoy proxy. + +The service name registered in Consul will be set to the name of the Kubernetes service +associated with the Pod. This can be customized with the `consul.hashicorp.com/connect-service` annotation. If using ACLs, this name must be the same as the Pod's `ServiceAccount` name. ### Connecting to Connect-Enabled Services -The example pod specification below configures a pod that is capable +The example Deployment specification below configures a Deployment that is capable of establishing connections to our previous example "static-server" service. The connection to this static text service happens over an authorized and encrypted connection via Connect. +-> **Note:** As of consul-k8s `v0.26.0-beta1` and Consul Helm `v0.32.0-beta1`, having a Kubernetes +Service is **required** to run services on the Consul Service Mesh. + ```yaml apiVersion: v1 +kind: Service +metadata: + name: static-client +spec: + selector: + app: static-client + ports: + - port: 80 +--- +apiVersion: v1 kind: ServiceAccount metadata: name: static-client --- -apiVersion: v1 -kind: Pod +apiVersion: apps/v1 +kind: Deployment metadata: name: static-client - annotations: - 'consul.hashicorp.com/connect-inject': 'true' - 'consul.hashicorp.com/connect-service-upstreams': 'static-server:1234' spec: - containers: - # This name will be the service name in Consul. - - name: static-client - image: tutum/curl:latest - # Just spin & wait forever, we'll use `kubectl exec` to demo - command: ['/bin/sh', '-c', '--'] - args: ['while true; do sleep 30; done;'] - # If ACLs are enabled, the serviceAccountName must match the Consul service name. - serviceAccountName: static-client + replicas: 1 + selector: + matchLabels: + app: static-client + template: + metadata: + name: static-client + labels: + app: static-client + annotations: + 'consul.hashicorp.com/connect-inject': 'true' + 'consul.hashicorp.com/connect-service-upstreams': 'static-server:1234' + spec: + containers: + # This name will be the service name in Consul. + - name: static-client + image: tutum/curl:latest + # Just spin & wait forever, we'll use `kubectl exec` to demo + command: ['/bin/sh', '-c', '--'] + args: ['while true; do sleep 30; done;'] + # If ACLs are enabled, the serviceAccountName must match the Consul service name. + serviceAccountName: static-client ``` Pods must specify upstream dependencies with the @@ -139,12 +196,12 @@ mutual TLS and identifying as the source service (`static-client` in this example). The injector will also set environment variables `_CONNECT_SERVICE_HOST` -and `_CONNECT_SERVICE_PORT` in every container in the pod for every defined +and `_CONNECT_SERVICE_PORT` in every container in the Pod for every defined upstream. This is analogous to the standard Kubernetes service environment variables, but point instead to the correct local proxy port to establish connections via Connect. -Any containers running in the pod that need to establish connections +Any containers running in the Pod that need to establish connections to dependencies must be reconfigured to use the local upstream address either directly or using the environment variables set by the injector (defined above). This means pods should not use Kubernetes service DNS or environment @@ -173,7 +230,7 @@ command terminated with exit code 52 ### Available Annotations -Annotations can be used to configure the injection behavior. +Pod annotations can be used to configure the injection behavior. - `consul.hashicorp.com/connect-inject` - If this is "true" then injection is enabled. If this is "false" then injection is explicitly disabled. @@ -181,6 +238,12 @@ Annotations can be used to configure the injection behavior. specifying this value as "true". This default can be changed in the injector's configuration if desired. +- `consul.hashicorp.com/transparent-proxy` Beta - If this is "true", this Pod + will run with transparent proxy enabled. This means you can use Kubernetes + DNS to access upstream services and all inbound and outbound traffic within + the pod is redirected to go through the proxy. + Using this annotation requires consul-k8s `v0.26.0-beta1` or higher. + - `consul.hashicorp.com/connect-service` - For pods that accept inbound connections, this specifies the name of the service that is being served. This defaults to the name of the first container in the pod. @@ -291,46 +354,6 @@ Annotations can be used to configure the injection behavior. - `consul.hashicorp.com/service-metrics-port` - Set the port where the Connect service exposes metrics. - `consul.hashicorp.com/service-metrics-path` - Set the path where the Connect service exposes metrics. -### Deployments, StatefulSets, etc. - -The annotations for configuring Connect must be on the pod specification. -Since higher level resources such as Deployments wrap pod specification -templates, Connect can be used with all of these higher level constructs, too. - -An example `Deployment` below shows how to enable Connect injection: - -```yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - name: consul-example-deployment -spec: - replicas: 1 - selector: - matchLabels: - app: consul-example - template: - metadata: - labels: - app: consul-example - annotations: - 'consul.hashicorp.com/connect-inject': 'true' - spec: - containers: - - name: consul-example - image: 'nginx' - serviceAccountName: consul-example ---- -apiVersion: v1 -kind: ServiceAccount -metadata: - name: consul-example -``` - -~> **A common mistake** is to set the annotation on the Deployment or -other resource. Ensure that the injector annotations are specified on -the _pod specification template_ as shown above. - ## Installation and Configuration The Connect sidecar proxy is injected via a diff --git a/website/content/docs/k8s/connect/ingress-gateways.mdx b/website/content/docs/k8s/connect/ingress-gateways.mdx index 99fc1ce46..4c1c87969 100644 --- a/website/content/docs/k8s/connect/ingress-gateways.mdx +++ b/website/content/docs/k8s/connect/ingress-gateways.mdx @@ -10,13 +10,13 @@ description: Configuring Ingress Gateways on Kubernetes ~> This topic requires familiarity with [Ingress Gateways](/docs/connect/ingress-gateway). -This page describes how to enable external access to Connect service mesh services running inside Kubernetes using Consul ingress gateways. +This page describes how to enable external access to Connect Service Mesh services running inside Kubernetes using Consul ingress gateways. See [Ingress Gateways](/docs/connect/ingress-gateway) for more information on use-cases and how it works. Adding an ingress gateway is a multi-step process that consists of the following steps: -- Setting the helm chart configuration -- Deploying the helm chart +- Setting the Helm chart configuration +- Deploying the Helm chart - Configuring the gateway - Defining an Intention (if ACLs are enabled) - Deploying your application to Kubernetes @@ -24,7 +24,7 @@ Adding an ingress gateway is a multi-step process that consists of the following ## Setting the helm chart configuration -When deploying the helm chart you must provide helm with a custom yaml file that contains your environment configuration. +When deploying the Helm chart you must provide Helm with a custom YAML file that contains your environment configuration. ```yaml global: @@ -43,8 +43,9 @@ ingressGateways: ~> **Note:** this will create a public unauthenticated LoadBalancer in your cluster, please take appropriate security considerations. -The yaml snippet is the launching point for a valid configuration that must be supplied when installing using the [official consul-helm chart](https://hub.helm.sh/charts/hashicorp/consul). -Information on additional options can be found in the [Helm reference](/docs/k8s/helm). Configuration options for ingress gateways reside under the [ingressGateways](/docs/k8s/helm#v-ingressgateways) entry. +The YAML snippet is the launching point for a valid configuration that must be supplied when installing using the [official consul-helm chart](https://hub.helm.sh/charts/hashicorp/consul). +Information on additional options can be found in the [Helm reference](/docs/k8s/helm). +Configuration options for ingress gateways reside under the [ingressGateways](/docs/k8s/helm#v-ingressgateways) entry. The gateways stanza is where you will define and configure the set of ingress gateways you want deployed to your environment. The only required field for each entry is `name`, though entries may contain any of the fields found in the `defaults` stanza. @@ -52,7 +53,7 @@ Values in this section override the values from the defaults stanza for the give the annotations from the defaults stanza will be _appended_ to any user-defined annotations defined in the gateways stanza rather than being overridden. Please refer to the ingress gateway configuration [documentation](/docs/k8s/helm#v-ingressgateways-defaults) for a detailed explanation of each option. -## Deploying the helm chart +## Deploying the Helm chart Ensure you have the latest consul-helm chart and install Consul via helm using the following [guide](/docs/k8s/installation/install#installing-consul) while being sure to provide the yaml configuration @@ -60,7 +61,8 @@ as previously discussed. ## Configuring the gateway -Now that Consul has been installed with ingress gateways enabled, you must configure the gateways via the [`IngressGateway`](/docs/connect/config-entries/ingress-gateway) custom resource. +Now that Consul has been installed with ingress gateways enabled, +you can configure the gateways via the [`IngressGateway`](/docs/connect/config-entries/ingress-gateway) custom resource. Here is an example `IngressGateway` resource: @@ -163,29 +165,51 @@ Now you will deploy a sample application which echoes “hello world” ```yaml apiVersion: v1 +kind: Service +metadata: + name: static-server +spec: + selector: + app: static-server + ports: + - protocol: TCP + port: 80 + targetPort: 8080 +--- +apiVersion: v1 kind: ServiceAccount metadata: name: static-server --- -apiVersion: v1 -kind: Pod +apiVersion: apps/v1 +kind: Deployment metadata: name: static-server - annotations: - 'consul.hashicorp.com/connect-inject': 'true' spec: - containers: - # This name will be the service name in Consul. - - name: static-server - image: hashicorp/http-echo:latest - args: - - -text="hello world" - - -listen=:8080 - ports: - - containerPort: 8080 - name: http - # If ACLs are enabled, the serviceAccountName must match the Consul service name. - serviceAccountName: static-server + replicas: 1 + selector: + matchLabels: + app: static-server + template: + metadata: + name: static-server + labels: + app: static-server + annotations: + 'consul.hashicorp.com/connect-inject': 'true' + spec: + containers: + # This name will be the service name in Consul. + - name: static-server + image: hashicorp/http-echo:latest + args: + - -text="hello world" + - -listen=:8080 + ports: + - containerPort: 8080 + name: http + # If ACLs are enabled, the serviceAccountName must match the Consul service name. + serviceAccountName: static-server ``` ```shell-session diff --git a/website/content/docs/k8s/connect/terminating-gateways.mdx b/website/content/docs/k8s/connect/terminating-gateways.mdx index d96d6ba3b..2f065e39a 100644 --- a/website/content/docs/k8s/connect/terminating-gateways.mdx +++ b/website/content/docs/k8s/connect/terminating-gateways.mdx @@ -12,8 +12,8 @@ description: Configuring Terminating Gateways on Kubernetes Adding a terminating gateway is a multi-step process: -- Update the helm chart with terminating gateway config options -- Deploy the helm chart +- Update the Helm chart with terminating gateway config options +- Deploy the Helm chart - Access the Consul agent - Register external services with Consul @@ -32,7 +32,7 @@ terminatingGateways: enabled: true ``` -## Deploying the helm chart +## Deploying the Helm chart Ensure you have the latest consul-helm chart and install Consul via helm using the following [guide](/docs/k8s/installation/install#installing-consul) while being sure to provide the yaml configuration @@ -145,7 +145,7 @@ service "example-https" { } ``` -Now fetch the id of the terminating gateway token +Now fetch the ID of the terminating gateway token ```shell-session consul acl token list | grep -B 6 -- "- terminating-gateway-terminating-gateway-token" | grep AccessorID @@ -220,6 +220,16 @@ An example deployment is provided which will serve as a static client for the te ```yaml apiVersion: v1 +kind: Service +metadata: + name: static-client +spec: + selector: + app: static-client + ports: + - port: 80 +--- +apiVersion: v1 kind: ServiceAccount metadata: name: static-client diff --git a/website/content/docs/k8s/helm.mdx b/website/content/docs/k8s/helm.mdx index 0b12a4ad0..07b16fc3b 100644 --- a/website/content/docs/k8s/helm.mdx +++ b/website/content/docs/k8s/helm.mdx @@ -434,6 +434,26 @@ and consider if they're appropriate for your deployment. should be a multi-line string matching the Tolerations (https://kubernetes.io/docs/concepts/configuration/taint-and-toleration/) array in a Pod spec. + - `topologySpreadConstraints` ((#v-server-topologyspreadconstraints)) (`string: ""`) - Pod topology spread constraints for server pods. + This should be a multi-line YAML string matching the `topologySpreadConstraints` array + (https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/) in a Pod Spec. + + This requires K8S >= 1.18 (beta) or 1.19 (stable). + + Example: + + ```yaml + topologySpreadConstraints: | + - maxSkew: 1 + topologyKey: topology.kubernetes.io/zone + whenUnsatisfiable: DoNotSchedule + labelSelector: + matchLabels: + app: {{ template "consul.name" . }} + release: "{{ .Release.Name }}" + component: server + ``` + - `nodeSelector` ((#v-server-nodeselector)) (`string: null`) - This value defines `nodeSelector` (https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector) labels for server pod assignment, formatted as a multi-line string. @@ -987,6 +1007,16 @@ and consider if they're appropriate for your deployment. - `consulWriteInterval` ((#v-synccatalog-consulwriteinterval)) (`string: null`) - Override the default interval to perform syncing operations creating Consul services. + - `extraLabels` ((#v-synccatalog-extralabels)) (`map`) - Extra labels to attach to the sync catalog pods. This should be a YAML map. + + Example: + + ```yaml + extraLabels: + labelKey: label-value + anotherLabelKey: another-label-value + ``` + - `connectInject` ((#v-connectinject)) - Configures the automatic Connect sidecar injector. - `enabled` ((#v-connectinject-enabled)) (`boolean: false`) - True if you want to enable connect injection. Set to "-" to inherit from @@ -1000,16 +1030,12 @@ and consider if they're appropriate for your deployment. to opt-in to Connect injection. If this is true, pods can use the same annotation to explicitly opt-out of injection. - - `healthChecks` ((#v-connectinject-healthchecks)) - Enables synchronization of Kubernetes health probe status with Consul. - NOTE: It is highly recommended to enable TLS with this feature because it requires - making calls to Consul clients across the cluster. Without TLS enabled, these calls - could leak ACL tokens should the cluster network become compromised. + - `transparentProxy` ((#v-connectinject-transparentproxy)) - Configures Transparent Proxy for Consul Service mesh services. + Using this feature requires Consul 1.10.0-beta1+ and consul-k8s 0.26.0-beta1+. - - `enabled` ((#v-connectinject-healthchecks-enabled)) (`boolean: true`) - Enables the Consul Health Check controller which syncs the readiness status of - connect-injected pods with Consul. - - - `reconcilePeriod` ((#v-connectinject-healthchecks-reconcileperiod)) (`string: 1m`) - If `healthChecks.enabled` is set to `true`, `reconcilePeriod` defines how often a full state - reconcile is done after the initial reconcile at startup is completed. + - `defaultEnabled` ((#v-connectinject-transparentproxy-defaultenabled)) (`boolean: true`) - If true, then all Consul Service mesh will run with transparent proxy enabled by default, + i.e. we enforce that all traffic within the pod will go through the proxy. + This value is overridable via the "consul.hashicorp.com/transparent-proxy" pod annotation. - `metrics` ((#v-connectinject-metrics)) - Configures metrics for Consul Connect services. All values are overridable via annotations on a per-pod basis. @@ -1048,16 +1074,6 @@ and consider if they're appropriate for your deployment. That can be configured with the `consul.hashicorp.com/service-metrics-path` annotation. - - `cleanupController` ((#v-connectinject-cleanupcontroller)) - Cleanup controller cleans up Consul service instances that remain registered - despite their pods no longer running. This could happen if the pod's `preStop` - hook failed to execute for some reason. - - - `reconcilePeriod` ((#v-connectinject-cleanupcontroller-reconcileperiod)) (`string: 5m`) - How often to do a full reconcile where the controller looks at all pods - and service instances and ensure the state is correct. - The controller reacts to each delete event immediately but if it misses - an event due to being down or a network issue, the reconcile loop will - handle cleaning up any missed deleted pods. - - `envoyExtraArgs` ((#v-connectinject-envoyextraargs)) (`string: null`) - Used to pass arguments to the injected envoy sidecar. Valid arguments to pass to envoy can be found here: https://www.envoyproxy.io/docs/envoy/latest/operations/cli e.g "--log-level debug --disable-hot-restart" @@ -1131,28 +1147,6 @@ and consider if they're appropriate for your deployment. pod in the k8s `staging` namespace will be registered into the `k8s-staging` Consul namespace. - - `certs` ((#v-connectinject-certs)) - The certs section configures how the webhook TLS certs are configured. - These are the TLS certs for the Kube apiserver communicating to the - webhook. By default, the injector will generate and manage its own certs, - but this requires the ability for the injector to update its own - MutatingWebhookConfiguration. In a production environment, custom certs - should probably be used. Configure the values below to enable this. - - - `secretName` ((#v-connectinject-certs-secretname)) (`string: null`) - Name of the secret that has the TLS certificate and - private key to serve the injector webhook. If this is null, then the - injector will default to its automatic management mode that will assign - a service account to the injector to generate its own certificates. - - - `caBundle` ((#v-connectinject-certs-cabundle)) (`string: ""`) - Base64-encoded PEM-encoded certificate bundle for the - CA that signed the TLS certificate that the webhook serves. This must - be set if secretName is non-null. - - - `certName` ((#v-connectinject-certs-certname)) (`string: tls.crt`) - Name of the file within the secret for - the TLS cert. - - - `keyName` ((#v-connectinject-certs-keyname)) (`string: tls.key`) - Name of the file within the secret for - the private TLS key. - - `nodeSelector` ((#v-connectinject-nodeselector)) (`string: null`) - Selector labels for connectInject pod assignment, formatted as a multi-line string. ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector