Merge pull request #4766 from hashicorp/f-connect-inject
website: add docs for the Connect injection
This commit is contained in:
Mitchell Hashimoto
GitHub
commit
1e81b23bbb
|
|
@ -0,0 +1,264 @@
|
|||
---
|
||||
layout: "docs"
|
||||
page_title: "Connect Sidecar - Kubernetes"
|
||||
sidebar_current: "docs-platform-k8s-connect"
|
||||
description: |-
|
||||
Connect is a feature built into to Consul that enables automatic service-to-service authorization and connection encryption across your Consul services. Connect can be used with Kubernetes to secure pod communication with other services.
|
||||
---
|
||||
|
||||
# Connect Sidecar on Kubernetes
|
||||
|
||||
[Connect](/docs/connect/index.html) is a feature built into to Consul that enables
|
||||
automatic service-to-service authorization and connection encryption across
|
||||
your Consul services. Connect can be used with Kubernetes to secure pod
|
||||
communication with other pods and external Kubernetes services.
|
||||
|
||||
The Connect sidecar running Envoy can be automatically injected into pods in
|
||||
your cluster, making configuration for Kubernetes automatic.
|
||||
This functionality is provided by the
|
||||
[consul-k8s project](https://github.com/hashicorp/consul-k8s) and can be
|
||||
automatically installed and configured using the
|
||||
[Consul Helm chart](/docs/platform/k8s/helm.html).
|
||||
|
||||
## Usage
|
||||
|
||||
When the
|
||||
[Connect injector is installed](/docs/platform/k8s/connect.html#installation-and-configuration),
|
||||
the Connect sidecar is automatically added to all pods. This sidecar can both
|
||||
accept and establish connections using Connect, enabling the pod to communicate
|
||||
to clients and dependencies exclusively over authorized and encrypted
|
||||
connections.
|
||||
|
||||
-> **Note:** The pod specifications in this section are valid and use
|
||||
publicly available images. If you've installed the Connect injector, feel free
|
||||
to run the pod specifications in this section to try Connect with Kubernetes.
|
||||
Please note the documentation below this sectionn on how to properly install
|
||||
and configure the Connect injector.
|
||||
|
||||
### Accepting Inbound Connections
|
||||
|
||||
An example pod is shown below with Connect enabled to accept inbound
|
||||
connections. Notice that the pod would still be fully functional without
|
||||
Connect. Minimal to zero modifications are required to pod specifications to
|
||||
enable Connect in Kubernetes.
|
||||
|
||||
This pod specification starts a server that responds to any
|
||||
HTTP request with the static text "hello world".
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: static-server
|
||||
annotations:
|
||||
"consul.hashicorp.com/connect-inject": "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: static-server
|
||||
image: hashicorp/http-echo:latest
|
||||
args:
|
||||
- -text="hello world"
|
||||
- -listen=:8080
|
||||
ports:
|
||||
- containerPort: 8080
|
||||
name: http
|
||||
```
|
||||
|
||||
The only change for Connect is the addition of the
|
||||
`consul.hashicorp.com/connect-inject` annotation. This enables injection
|
||||
for this pod. The injector can also be
|
||||
[configured](/docs/platform/k8s/connect.html#installation-and-configuration)
|
||||
to automatically inject unless explicitly disabled, but the default
|
||||
installation requires opt-in using the annotation shown above.
|
||||
|
||||
This will start a Connect sidecar that listens on a random port registered
|
||||
with Consul and proxies valid inbound connections to port 8080 in the pod.
|
||||
To establish a connection to the pod using Connect, a client must use another Connect
|
||||
proxy. The client Connect proxy will use Consul service discovery to find
|
||||
all available upstream proxies and their public ports.
|
||||
|
||||
In the example above, the server is listening on `:8080`. This means
|
||||
the server will still bind to the pod IP and allow external connections.
|
||||
This is useful to transition to Connect by allowing both Connect and
|
||||
non-Connect connections. To restrict access to only Connect-authorized clients,
|
||||
any listeners should bind to localhost only (such as `127.0.0.1`).
|
||||
|
||||
### Connecting to Connect-Enabled Services
|
||||
|
||||
The example pod specification below configures a pod that is capable
|
||||
of establishing connections to our previous example "static-server" service. The
|
||||
connection to this static text service happens over an authorized and encrypted
|
||||
connection via Connect.
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
name: static-client
|
||||
annotations:
|
||||
"consul.hashicorp.com/connect-inject": "true"
|
||||
"consul.hashicorp.com/connect-service-upstreams": "static-server:1234"
|
||||
spec:
|
||||
containers:
|
||||
- name: static-client
|
||||
image: tutum/curl:latest
|
||||
# Just spin & wait forever, we'll use `kubectl exec` to demo
|
||||
command: [ "/bin/sh", "-c", "--" ]
|
||||
args: [ "while true; do sleep 30; done;" ]
|
||||
```
|
||||
|
||||
Pods must specify upstream dependencies with the
|
||||
[`consul.hashicorp.com/connect-service-upstreams` annotation](/docs/platform/k8s/connect.html#consul-hashicorp-com-connect-service-upstreams).
|
||||
This annotation declares the names of any upstream dependencies and a
|
||||
local port for the proxy to listen on. When a connection is established to that local
|
||||
port, the proxy establishes a connection to the target service
|
||||
("static-server" in this example) using
|
||||
mutual TLS and identifying as the source service ("static-client" in this
|
||||
example).
|
||||
|
||||
The injector will also set environment variables `<NAME>_CONNECT_SERVICE_HOST`
|
||||
and `<NAME>_CONNECT_SERVICE_PORT` in every container in the pod for every defined
|
||||
upstream. This is analogous to the standard Kubernetes service environment variables, but
|
||||
point instead to the correct local proxy port to establish connections via
|
||||
Connect.
|
||||
|
||||
Any containers running in the pod that need to establish connections
|
||||
to dependencies must be reconfigured to use the local upstream address either
|
||||
directly or using the environment variables set by the injector (defined above).
|
||||
This means pods should not use Kubernetes service DNS or environment
|
||||
variables for these connections.
|
||||
|
||||
|
||||
We can verify access to the static text server using `kubectl exec`. Notice
|
||||
that we use the local address and port from the upstream annotation (1234)
|
||||
for this verification.
|
||||
|
||||
```sh
|
||||
$ kubectl exec static-client -- curl -s http://127.0.0.1:1234/
|
||||
"hello world"
|
||||
```
|
||||
|
||||
We can control access to the server using [intentions](/docs/connect/intentions.html).
|
||||
If you use the Consul UI or [CLI](/docs/commands/intention/create.html) to
|
||||
create a deny [intention](/docs/connect/intentions.html) between
|
||||
"static-client" and "static-server", connections are immediately rejected
|
||||
without updating either of the running pods. You can then remove this
|
||||
intention to allow connections again.
|
||||
|
||||
```sh
|
||||
$ kubectl exec static-client -- curl -s http://127.0.0.1:1234/
|
||||
command terminated with exit code 52
|
||||
```
|
||||
|
||||
### Available Annotations
|
||||
|
||||
Annotations can be used to configure the injection behavior.
|
||||
|
||||
* `consul.hashicorp.com/connect-inject` - If this is "true" then injection
|
||||
is enabled. If this is "false" then injection is explicitly disabled.
|
||||
The default injector behavior requires pods to opt-in to injection by
|
||||
specifying this value as "true". This default can be changed in the
|
||||
injector's configuration if desired.
|
||||
|
||||
* `consul.hashicorp.com/connect-service` - For pods that accept inbound
|
||||
connections, this specifies the name of the service that is being
|
||||
served. This defaults to the name of the first container in the pod.
|
||||
|
||||
* `consul.hashicorp.com/connect-service-port` - For pods that accept inbound
|
||||
connections, this specifies the port to route inbound connections to. This
|
||||
is the port that the service is listening on. The service port defaults to
|
||||
the first exposed port on any container in the pod. If specified, the value
|
||||
can be the _name_ of a configured port, such as "http" or it can be a direct
|
||||
port value such as "8080". This is the port of the _service_, the proxy
|
||||
public listener will listen on a dynamic port.
|
||||
|
||||
* `consul.hashicorp.com/connect-service-upstreams` - The list of upstream
|
||||
services that this pod needs to connect to via Connect along with a static
|
||||
local port to listen for those connections. Example: `db:1234,auth:6789`
|
||||
will start two local listeners for `db` on port 1234 and `auth` on port
|
||||
6789, respectively. The name of the service is the name of the service
|
||||
registered with Consul. This value defaults to no upstreams.
|
||||
|
||||
### Deployments, StatefulSets, etc.
|
||||
|
||||
The annotations for configuring Connect must be on the pod specification.
|
||||
Since higher level resources such as Deployments wrap pod specification
|
||||
templates, Connect can be used with all of these higher level constructs, too.
|
||||
|
||||
An example `Deployment` below shows how to enable Connect injection:
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: consul-example-deployment
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: consul-example
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: consul-example
|
||||
annotations:
|
||||
"consul.hashicorp.com/connect-inject": "true"
|
||||
spec:
|
||||
containers:
|
||||
- name: example
|
||||
image: "nginx"
|
||||
```
|
||||
|
||||
~> **A common mistake** is to set the annotation on the Deployment or
|
||||
other resource. Ensure that the injector annotations are specified on
|
||||
the _pod specification template_ as shown above.
|
||||
|
||||
## Installation and Configuration
|
||||
|
||||
The Connect sidecar proxy is injected via a
|
||||
[mutating admission webhook](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#admission-webhooks)
|
||||
provided by the
|
||||
[consul-k8s project](https://github.com/hashicorp/consul-k8s).
|
||||
This enables the automatic pod mutation shown in the usage section above.
|
||||
Installation of the mutating admission webhook is automated using the
|
||||
[Helm chart](/docs/platform/k8s/helm.html).
|
||||
|
||||
To install the Connect injector, enable the Connect injection feature using
|
||||
[Helm values](/docs/platform/k8s/helm.html#configuration-values-) and
|
||||
upgrade the installation using `helm upgrade` for existing installs or
|
||||
`helm install` for a fresh install. The Connect injector **also requires**
|
||||
[client agents](/docs/platform/k8s/helm.html#v-client) are enabled on
|
||||
the node with pods that are using Connect and that
|
||||
[gRPC is enabled](/docs/platform/k8s/helm.html#v-client-grpc).
|
||||
|
||||
```yaml
|
||||
connectInject:
|
||||
enabled: true
|
||||
|
||||
client:
|
||||
enabled: true
|
||||
grpc: true
|
||||
```
|
||||
|
||||
This will configure the injector to inject when the
|
||||
[injection annotation](#)
|
||||
is present. Other values in the Helm chart can be used to limit the namespaces
|
||||
the injector runs in, enable injection by default, and more.
|
||||
|
||||
As noted above, the Connect auto-injection requires that local client agents
|
||||
are configured. These client agents must be successfully joined to a Consul
|
||||
cluster.
|
||||
The Consul server cluster can run either in or out of a Kubernetes cluster.
|
||||
|
||||
### Verifying the Installation
|
||||
|
||||
To verify the installation, run the
|
||||
["Accepting Inbound Connections"](/docs/platform/k8s/connect.html#accepting-inbound-connections)
|
||||
example from the "Usage" section above. After running this example, run
|
||||
`kubectl get pod static-server -o yaml`. In the raw YAML output, you should
|
||||
see injected Connect containers and an annotation
|
||||
`consul.hashicorp.com/connect-inject-status` set to `injected`. This
|
||||
confirms that injection is working properly.
|
||||
|
||||
If you do not see this, then use `kubectl logs` against the injector pod
|
||||
and note any errors.
|
||||
|
|
@ -198,6 +198,11 @@ and consider if they're appropriate for your deployment.
|
|||
The name of the Docker image (including any tag) for the containers running
|
||||
Consul client agents.
|
||||
|
||||
- <a name="v-client-grpc" href="#v-client-grpc">`grpc`</a> (`boolean: false`) -
|
||||
If true, agents will enable their GRPC listener on port 8502 and expose
|
||||
it to the host. This will use slightly more resources, but is required for
|
||||
[Connect](/docs/platform/k8s/connect.html).
|
||||
|
||||
- <a name="v-client-extraconfig" href="#v-client-extraconfig">`extraConfig`</a> (`string: "{}"`) -
|
||||
A raw string of extra JSON or HCL configuration for Consul clients. This
|
||||
will be saved as-is into a ConfigMap that is read by the Consul agents.
|
||||
|
|
@ -292,3 +297,60 @@ and consider if they're appropriate for your deployment.
|
|||
by Kubernetes. The available service types are documented on
|
||||
[the Kubernetes website](https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types).
|
||||
|
||||
* <a name="v-connectinject" href="#v-connectinject">`connectInject`</a> - Values that
|
||||
configure running the [Connect injector](/docs/platform/k8s/connect.html).
|
||||
|
||||
- <a name="v-connectinject-enabled" href="#v-connectinject-enabled">`enabled`</a> (`boolean: false`) -
|
||||
If true, the chart will install all the resources necessary for the
|
||||
Connect injector process to run. This will enable the injector but will
|
||||
require pods to opt-in with an annotation by default.
|
||||
|
||||
- <a name="v-connectinject-imageConsul" href="#v-connectinject-imageConsul">`imageConsul`</a> (`string: global.image`) -
|
||||
The name of the Docker image (including any tag) for Consul. This is used
|
||||
for proxy service registration, Envoy configuration, etc.
|
||||
|
||||
- <a name="v-connectinject-imageEnvoy" href="#v-connectinject-imageEnvoy">`imageEnvoy`</a> (`string: ""`) -
|
||||
The name of the Docker image (including any tag) for the Envoy sidecar.
|
||||
`envoy` must be on the executable path within this image. This Envoy
|
||||
version must be compatible with the Consul version used by the injector.
|
||||
This defaults to letting the injector choose the Envoy image, which is
|
||||
usually `envoy/envoy-alpine`.
|
||||
|
||||
- <a name="v-connectinject-default" href="#v-connectinject-default">`default`</a> (`boolean: false`) -
|
||||
If true, the injector will inject the Connect sidecar into all pods by
|
||||
default. Otherwise, pods must specify the
|
||||
[injection annotation](/docs/platform/k8s/connect.html#consul-hashicorp-com-connect-inject)
|
||||
to opt-in to Connect injection. If this is true, pods can use the same
|
||||
annotation to explicitly opt-out of injection.
|
||||
|
||||
- <a name="v-connectinject-namespaceselector" href="#v-connectinject-namespaceselector">`namespaceSelector`</a> (`string: ""`) -
|
||||
A [selector](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/)
|
||||
for restricting injection to only matching namespaces. By default
|
||||
all namespaces except the system namespace will have injection enabled.
|
||||
|
||||
- <a name="v-connectinject-certs" href="#v-connectinject-certs">`certs`</a> -
|
||||
The certs section configures how the webhook TLS certs are configured.
|
||||
These are the TLS certs for the Kube apiserver communicating to the
|
||||
webhook. By default, the injector will generate and manage its own certs,
|
||||
but this requires the ability for the injector to update its own
|
||||
`MutatingWebhookConfiguration`. In a production environment, custom certs
|
||||
should probably be used. Configure the values below to enable this.
|
||||
|
||||
* <a name="v-connectinject-certs-secretname" href="#v-connectinject-certs-secretname">`secretName`</a> (`string: null`) -
|
||||
secretName is the name of the Kubernetes secret that has the TLS certificate and
|
||||
private key to serve the injector webhook. If this is null, then the
|
||||
injector will default to its automatic management mode.
|
||||
|
||||
* <a name="v-connectinject-cabundle" href="#v-connectinject-cabundle">`caBundle`</a> (`string: ""`) -
|
||||
The PEM-encoded CA public certificate bundle for the TLS certificate served by the
|
||||
injector. This must be specified as a string and can't come from a
|
||||
secret because it must be statically configured on the Kubernetes
|
||||
`MutatingAdmissionWebhook` resource. This only needs to be specified
|
||||
if `secretName` is not null.
|
||||
|
||||
* <a name="v-connectinject-certs-certname" href="#v-connectinject-certs-certname">`certName`</a> (`string: "tls.crt"`) -
|
||||
The name of the certificate file within the `secretName` secret.
|
||||
|
||||
* <a name="v-connectinject-certs-keynamkeyname" href="#v-connectinject-certs-keyname">`keyName`</a> (`string: "tls.key"`) -
|
||||
The name of the private key for the certificate file within the
|
||||
`secretName` secret.
|
||||
|
|
|
|||
|
|
@ -302,6 +302,9 @@
|
|||
<li<%= sidebar_current("docs-connect-platform-nomad") %>>
|
||||
<a href="/docs/connect/platform/nomad.html">Nomad</a>
|
||||
</li>
|
||||
<li<%= sidebar_current("docs-connect-platform-k8s") %>>
|
||||
<a href="/docs/platform/k8s/connect.html">Kubernetes</a>
|
||||
</li>
|
||||
<li<%= sidebar_current("docs-connect-security") %>>
|
||||
<a href="/docs/connect/security.html">Security</a>
|
||||
</li>
|
||||
|
|
@ -326,6 +329,9 @@
|
|||
<li<%= sidebar_current("docs-platform-k8s-service-sync") %>>
|
||||
<a href="/docs/platform/k8s/service-sync.html">Service Sync</a>
|
||||
</li>
|
||||
<li<%= sidebar_current("docs-platform-k8s-connect") %>>
|
||||
<a href="/docs/platform/k8s/connect.html">Connect Sidecar</a>
|
||||
</li>
|
||||
</ul>
|
||||
</li>
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue