luxolus
/
open-consul
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity

Backport of docs - add jobs use case for service mesh k8s into release/1.16.x (#18043) 

* resolve conflicts
* fix backport
* address feedback

---------

Co-authored-by: David Yu <dyu@hashicorp.com>
This commit is contained in:
hc-github-team-consul-core 2023-07-07 12:30:11 -05:00 committed by GitHub
parent b8f2de3aa0
commit 97a57b476f
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 197 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

219
website/content/docs/k8s/connect/index.mdx
View File

@ -19,10 +19,33 @@ Consul service mesh is enabled by default when you install Consul on Kubernetes
If `connectInject.default` is set to `false` or you want to explicitly enable service mesh sidecar proxy injection for a specific deployment, add the `consul.hashicorp.com/connect-inject` annotation to the pod specification template and set it to `true` when connecting services to the mesh. If `connectInject.default` is set to `false` or you want to explicitly enable service mesh sidecar proxy injection for a specific deployment, add the `consul.hashicorp.com/connect-inject` annotation to the pod specification template and set it to `true` when connecting services to the mesh.
### Example ### Service names
When the service is onboarded, the name registered in Consul is set to the name of the Kubernetes Service associated with the Pod. You can use the [`consul.hashicorp.com/connect-service` annotation](/consul/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service) to specify a custom name for the service, but if ACLs are enabled then the name of the service registered in Consul must match the Pod's `ServiceAccount` name.
### Transparent proxy mode
By default, the Consul service mesh runs in transparent proxy mode. This mode forces inbound and outbound traffic through the sidecar proxy even though the service binds to all interfaces. Transparent proxy infers the location of upstream services using Consul service intentions, and also allows you to use Kubernetes DNS as you normally would for your workloads.
When transparent proxy mode is enabled, all service-to-service traffic is required to use mTLS. When onboarding new services to service mesh, your network may have mixed mTLS and non-mTLS traffic, which can result in broken service-to-service communication. You can temporarily enable permissive mTLS mode during the onboarding process so that existing mesh services can accept traffic from services that are not yet fully onboarded. Permissive mTLS enables sidecar proxies to access both mTLS and non-mTLS traffic. Refer to [Onboard mesh services in transparent proxy mode](/consul/docs/k8s/connect/onboarding-tproxy-mode) for additional information.
### Kubernetes service mesh workload scenarios
-> **Note:** A Kubernetes Service is required in order to register services on the Consul service mesh. Consul monitors the lifecyle of the Kubernetes Service and its service instances using the service object. In addition, the Kubernetes service is used to register and de-register the service from Consul's catalog.
The following configurations are examples for registering workloads on Kubernetes into Consul's service mesh in different scenarios. Each scenario provides an example Kubernetes manifest to demonstrate how to use Consul's service mesh with a specific Kubernetes workload type.
- [Kubernetes Pods running as a deployment](#kubernetes-pods-running-as-a-deployment)
- [Connecting to mesh-enabled Services](#connecting-to-mesh-enabled-services)
- [Kubernetes Jobs](#kubernetes-jobs)
- [Kubernetes Pods with multiple ports](#kubernetes-pods-with-multiple-ports)
#### Kubernetes Pods running as a deployment
The following example shows a Kubernetes configuration that specifically enables service mesh connections for the `static-server` service. Consul starts and registers a sidecar proxy that listens on port 20000 by default and proxies valid inbound connections to port 8080. The following example shows a Kubernetes configuration that specifically enables service mesh connections for the `static-server` service. Consul starts and registers a sidecar proxy that listens on port 20000 by default and proxies valid inbound connections to port 8080.
<CodeBlockConfig filename="static-server.yaml">
```yaml ```yaml
apiVersion: v1 apiVersion: v1
kind: Service kind: Service
@ -72,27 +95,19 @@ spec:
serviceAccountName: static-server serviceAccountName: static-server
``` ```
To establish a connection to the Pod using service mesh, a client must use another mesh proxy. The client mesh proxy will use Consul service discovery to find all available upstream proxies and their public ports. </CodeBlockConfig>
### Service names To establish a connection to the upstream Pod using service mesh, a client must dial the upstream workload using a mesh proxy. The client mesh proxy will use Consul service discovery to find all available upstream proxies and their public ports.
When the service is onboarded, the name registered in Consul is set to the name of the Kubernetes Service associated with the Pod. You can specify a custom name for the service in the [`consul.hashicorp.com/connect-service` annotation](/consul/docs/k8s/annotations-and-labels#consul-hashicorp-com-connect-service), but if ACLs are enabled, then the name of the service registered in Consul must match the Pod's `ServiceAccount` name. #### Connecting to mesh-enabled Services
### Transparent proxy mode
By default, the Consul service mesh runs in transparent proxy mode. This mode forces inbound and outbound traffic through the sidecar proxy even though the service binds to all interfaces. Transparent proxy infers the location of upstream services using Consul service intentions, and also allows you to use Kubernetes DNS as you normally would for your workloads.
When transparent proxy mode is enabled, all service-to-service traffic is required to use mTLS. While onboarding new services to service mesh, your network may have mixed mTLS and non-mTLS traffic, which can result in broken service-to-service communication. You can temporarily enable permissive mTLS mode during the onboarding process so that existing mesh services can accept traffic from services that are not yet fully onboarded. Permissive mTLS enables sidecar proxies to access both mTLS and non-mTLS traffic. Refer to [Onboard mesh services in transparent proxy mode](/consul/docs/k8s/connect/onboarding-tproxy-mode) for additional information.
### Connecting to Mesh-Enabled Services
The example Deployment specification below configures a Deployment that is capable The example Deployment specification below configures a Deployment that is capable
of establishing connections to our previous example "static-server" service. The of establishing connections to our previous example "static-server" service. The
connection to this static text service happens over an authorized and encrypted connection to this static text service happens over an authorized and encrypted
connection via service mesh. connection via service mesh.
-> **Note:** As of consul-k8s `v0.26.0` and Consul Helm `v0.32.0`, having a Kubernetes
Service is **required** to run services on the Consul Service Mesh. <CodeBlockConfig filename="static-client.yaml">
```yaml ```yaml
apiVersion: v1 apiVersion: v1
@ -138,6 +153,8 @@ spec:
serviceAccountName: static-client serviceAccountName: static-client
``` ```
</CodeBlockConfig>
By default when ACLs are enabled or when ACLs default policy is `allow`, By default when ACLs are enabled or when ACLs default policy is `allow`,
Consul will automatically configure proxies with all upstreams from the same datacenter. Consul will automatically configure proxies with all upstreams from the same datacenter.
When ACLs are enabled with default `deny` policy, When ACLs are enabled with default `deny` policy,
@ -172,7 +189,96 @@ $ kubectl exec deploy/static-client -- curl --silent http://static-server/
command terminated with exit code 52 command terminated with exit code 52
``` ```
### Kubernetes Pods with Multiple ports #### Kubernetes Jobs
Kubernetes Jobs run pods that only make outbound requests to services on the mesh and successfully terminate when they are complete. In order to register a Kubernetes Job with the mesh, you must provide an integer value for the `consul.hashicorp.com/sidecar-proxy-lifecycle-shutdown-grace-period-seconds` annotation. Then, issue a request to the `http://127.0.0.1:20600/graceful_shutdown` API endpoint so that Kubernetes gracefully shuts down the `consul-dataplane` sidecar after the job is complete.
Below is an example Kubernetes manifest that deploys a job correctly.
<CodeBlockConfig filename="test-job.yaml">
```yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-job
namespace: default
---
apiVersion: v1
kind: Service
metadata:
name: test-job
namespace: default
spec:
selector:
app: test-job
ports:
- port: 80
---
apiVersion: batch/v1
kind: Job
metadata:
name: test-job
namespace: default
labels:
app: test-job
spec:
template:
metadata:
annotations:
'consul.hashicorp.com/connect-inject': 'true'
'consul.hashicorp.com/sidecar-proxy-lifecycle-shutdown-grace-period-seconds': '5'
labels:
app: test-job
spec:
containers:
- name: test-job
image: alpine/curl:3.14
ports:
- containerPort: 80
command:
- /bin/sh
- -c
- |
echo "Started test job"
sleep 10
echo "Killing proxy"
curl --max-time 2 -s -f -XPOST http://127.0.0.1:20600/graceful_shutdown
sleep 10
echo "Ended test job"
serviceAccountName: test-job
restartPolicy: Never
```
</CodeBlockConfig>
Upon completing the job you should be able to verify that all containers are shut down within the pod.
```shell-session
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
test-job-49st7 0/2 Completed 0 3m55s
```
```shell-session
$ kubectl get job
NAME COMPLETIONS DURATION AGE
test-job 1/1 30s 4m31s
```
In addition, based on the logs emitted by the pod you can verify that the proxy was shut down before the Job completed.
```shell-session
$ kubectl logs test-job-49st7 -c test-job
Started test job
Killing proxy
Ended test job
```
#### Kubernetes Pods with multiple ports
To configure a pod with multiple ports to be a part of the service mesh and receive and send service mesh traffic, you To configure a pod with multiple ports to be a part of the service mesh and receive and send service mesh traffic, you
will need to add configuration so that a Consul service can be registered per port. This is because services in Consul will need to add configuration so that a Consul service can be registered per port. This is because services in Consul
currently support a single port per service instance. currently support a single port per service instance.
@ -184,6 +290,9 @@ First, decide on the names for the two Consul services that will correspond to t
chooses the names `web` for `8080` and `web-admin` for `9090`. chooses the names `web` for `8080` and `web-admin` for `9090`.
Create two service accounts for `web` and `web-admin`: Create two service accounts for `web` and `web-admin`:
<CodeBlockConfig filename="multiport-web-sa.yaml">
```yaml ```yaml
apiVersion: v1 apiVersion: v1
kind: ServiceAccount kind: ServiceAccount
@ -195,7 +304,14 @@ kind: ServiceAccount
metadata: metadata:
name: web-admin name: web-admin
``` ```
</CodeBlockConfig>
Create two Service objects for `web` and `web-admin`: Create two Service objects for `web` and `web-admin`:
<CodeBlockConfig filename="multiport-web-svc.yaml">
```yaml ```yaml
apiVersion: v1 apiVersion: v1
kind: Service kind: Service
@ -221,12 +337,17 @@ spec:
port: 80 port: 80
targetPort: 9090 targetPort: 9090
``` ```
</CodeBlockConfig>
`web` will target `containerPort` `8080` and select pods labeled `app: web`. `web-admin` will target `containerPort` `web` will target `containerPort` `8080` and select pods labeled `app: web`. `web-admin` will target `containerPort`
`9090` and will also select the same pods. `9090` and will also select the same pods.
~> Kubernetes 1.24+ only ~> Kubernetes 1.24+ only
In Kubernetes 1.24+ you need to [create a Kubernetes secret](https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets) for each multi-port service that references the ServiceAccount, and the Kubernetes secret must have the same name as the ServiceAccount: In Kubernetes 1.24+ you need to [create a Kubernetes secret](https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets) for each multi-port service that references the ServiceAccount, and the Kubernetes secret must have the same name as the ServiceAccount:
<CodeBlockConfig filename="multiport-web-secret.yaml">
```yaml ```yaml
apiVersion: v1 apiVersion: v1
kind: Secret kind: Secret
@ -245,12 +366,15 @@ metadata:
type: kubernetes.io/service-account-token type: kubernetes.io/service-account-token
``` ```
</CodeBlockConfig>
Create a Deployment with any chosen name, and use the following annotations: Create a Deployment with any chosen name, and use the following annotations:
```yaml ```yaml
consul.hashicorp.com/connect-inject: true annotations:
consul.hashicorp.com/transparent-proxy: false 'consul.hashicorp.com/connect-inject': 'true'
consul.hashicorp.com/connect-service: web,web-admin 'consul.hashicorp.com/transparent-proxy': 'false'
consul.hashicorp.com/connect-service-port: 8080,9090 'consul.hashicorp.com/connect-service': 'web,web-admin'
'consul.hashicorp.com/connect-service-port': '8080,9090'
``` ```
Note that the order the ports are listed in the same order as the service names, i.e. the first service name `web` Note that the order the ports are listed in the same order as the service names, i.e. the first service name `web`
corresponds to the first port, `8080`, and the second service name `web-admin` corresponds to the second port, `9090`. corresponds to the first port, `8080`, and the second service name `web-admin` corresponds to the second port, `9090`.
@ -260,7 +384,10 @@ The service account on the pod spec for the deployment should be set to the firs
serviceAccountName: web serviceAccountName: web
``` ```
For reference, the full deployment example could look something like the following: The following deployment example demonstrates the required annotations for the manifest. In addition, the previous YAML manifests can also be combined into a single manifest for easier deployment.
<CodeBlockConfig filename="multiport-web.yaml">
```yaml ```yaml
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
@ -302,13 +429,61 @@ spec:
serviceAccountName: web serviceAccountName: web
``` ```
</CodeBlockConfig>
After deploying the `web` application, you can test service mesh connections by deploying the `static-client` After deploying the `web` application, you can test service mesh connections by deploying the `static-client`
application with the configuration in the [previous section](#connecting-to-mesh-enabled-services) and add the application with the configuration in the [previous section](#connecting-to-mesh-enabled-services) and add the
following annotation to the pod template on `static-client`: `consul.hashicorp.com/connect-service-upstreams: 'web:1234,web-admin:2234'` annotation to the pod template on `static-client`:
<CodeBlockConfig filename="multiport-static-client.yaml" lineNumbers highlight="33">
```yaml ```yaml
consul.hashicorp.com/connect-service-upstreams: "web:1234,web-admin:2234" apiVersion: v1
kind: Service
metadata:
# This name will be the service name in Consul.
name: static-client
spec:
selector:
app: static-client
ports:
- port: 80
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: static-client
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: static-client
spec:
replicas: 1
selector:
matchLabels:
app: static-client
template:
metadata:
name: static-client
labels:
app: static-client
annotations:
'consul.hashicorp.com/connect-inject': 'true'
'consul.hashicorp.com/connect-service-upstreams': 'web:1234,web-admin:2234'
spec:
containers:
- name: static-client
image: curlimages/curl:latest
# Just spin & wait forever, we'll use `kubectl exec` to demo
command: ['/bin/sh', '-c', '--']
args: ['while true; do sleep 30; done;']
# If ACLs are enabled, the serviceAccountName must match the Consul service name.
serviceAccountName: static-client
``` ```
</CodeBlockConfig>
If you exec on to a static-client pod, using a command like: If you exec on to a static-client pod, using a command like:
```shell-session ```shell-session
$ kubectl exec -it static-client-5bd667fbd6-kk6xs -- /bin/sh $ kubectl exec -it static-client-5bd667fbd6-kk6xs -- /bin/sh