[docs] New K8s-Consul deployment guide (#5859)
* New K8s-Consul deployment guide * Update website/source/docs/guides/kubernetes-production-deploy.md * Update website/source/docs/guides/kubernetes-production-deploy.md Co-Authored-By: Rebecca Zanzig <rebecca@hashicorp.com> * Update website/source/docs/guides/kubernetes-production-deploy.md Co-Authored-By: Rebecca Zanzig <rebecca@hashicorp.com> * Update website/source/docs/guides/kubernetes-production-deploy.md Co-Authored-By: Rebecca Zanzig <rebecca@hashicorp.com> * Update website/source/docs/guides/kubernetes-production-deploy.md Co-Authored-By: Rebecca Zanzig <rebecca@hashicorp.com> * Update website/source/docs/guides/kubernetes-production-deploy.md Co-Authored-By: Rebecca Zanzig <rebecca@hashicorp.com> * updating based on comments * Update website/source/docs/guides/kubernetes-production-deploy.md Co-Authored-By: Rebecca Zanzig <rebecca@hashicorp.com> * Update website/source/docs/guides/kubernetes-production-deploy.md * Update website/source/docs/guides/kubernetes-production-deploy.md
This commit is contained in:
kaitlincarter-hc
GitHub
parent
60e9703588
commit
e89b6dd85d
|
|
@ -0,0 +1,286 @@
|
|||
---
|
||||
name: "Consul-Kubernetes Deployment Guide"
|
||||
content_length: 14
|
||||
id: kubernetes-production-deploy
|
||||
layout: content_layout
|
||||
products_used:
|
||||
- Consul
|
||||
description: This guide covers the necessary steps to install and configure a new Consul cluster on Kubernetes.
|
||||
level: Advanced
|
||||
___
|
||||
|
||||
|
||||
This guide covers the necessary steps to install and configure a new Consul
|
||||
cluster on Kubernetes, as defined in the [Consul Reference Architecture
|
||||
guide](/consul/day-1-operations/kubernetes-reference#consul-datacenter-deployed-in-kubernetes).
|
||||
By the end of this guide, you will be able to identify the installation
|
||||
prerequisites, customize the Helm chart to fit your environment requirements,
|
||||
and interact with your new Consul cluster.
|
||||
|
||||
~> You should have the following configured before starting this guide: Helm
|
||||
installed and configured locally, tiller running in the Kubernetes cluster, and
|
||||
the Kubernetes CLI configured.
|
||||
|
||||
## Configure Kubernetes Permissions to Deploy Consul
|
||||
|
||||
Before deploying Consul, you will need to create a new Kubernetes service
|
||||
account with the correct permissions and to authenticate it on the command
|
||||
line. You will need Kubernetes operators permissions to create and modify
|
||||
policies, deploy services, access the Kubernetes dashboard, create secrets, and
|
||||
create RBAC objects. You can find documentation for RBAC and service accounts
|
||||
for the following cloud providers.
|
||||
|
||||
- [AKS](https://docs.microsoft.com/en-us/azure/aks/kubernetes-service-principal)
|
||||
- [EKS](https://docs.aws.amazon.com/eks/latest/userguide/install-aws-iam-authenticator.html)
|
||||
- [GCP](https://console.cloud.google.com/iam-admin/serviceaccounts)
|
||||
|
||||
Note, Consul can be deployed on any properly configured Kubernetes cluster in
|
||||
the cloud or on premises.
|
||||
|
||||
Once you have a service account, you will also need to add a permission to
|
||||
deploy the helm chart. This is done with the `clusterrolebinding` method.
|
||||
|
||||
```sh
|
||||
$ kubectl create clusterrolebinding kubernetes-dashboard -n kube-system --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard
|
||||
```
|
||||
|
||||
Finally, you may need to create Kubernetes secrets to store Consul data. You
|
||||
can reference these secrets in the customized Helm chart values file.
|
||||
|
||||
- If you have purchased Enterprise Consul, the enterprise license file should be
|
||||
used with the official image, `hashicorp/consul-enterprise:1.5.0-ent`.
|
||||
|
||||
- Enable
|
||||
[encryption](https://www.consul.io/docs/agent/encryption.html#gossip-encryption) to secure gossip traffic within the Consul cluster.
|
||||
|
||||
|
||||
~> Note, depending on your environment, the previous secrets may not be
|
||||
necessary.
|
||||
|
||||
## Configure Helm Chart
|
||||
|
||||
Now that you have prepared your Kubernetes cluster, you can customize the Helm
|
||||
chart. First, you will need to download the latest official Helm chart.
|
||||
|
||||
```sh
|
||||
$ git clone https://github.com/hashicorp/consul-helm.git
|
||||
```
|
||||
|
||||
The `consul-helm` directory will contain a `values.yaml` file with example
|
||||
parameters. You can update this file to customize your Consul deployment. Below
|
||||
we detail some of the parameters you should customize and provide an example
|
||||
file, however you should consider your particular production needs when
|
||||
configuring your chart.
|
||||
|
||||
### Global Values
|
||||
|
||||
The global values will affect all the other parameters in the chart.
|
||||
|
||||
To enable all of the Consul components in the Helm chart, set `enabled` to
|
||||
`true`. This means servers, clients, Consul DNS, and the Consul UI will be
|
||||
installed with their defaults. You should also set the following global
|
||||
parameters based on your specific environment requirements.
|
||||
|
||||
- `image` is the name and tag of the Consul Docker image.
|
||||
- `imagek8s` is the name and tag of the Docker image for the consul-k8s binary.
|
||||
- `datacenter` the name of your Consul datacenter.
|
||||
- `domain` the domain Consul uses for DNS queries.
|
||||
|
||||
For security, set the `bootstrapACLs` parameter to true. This will enable
|
||||
Kubernetes to initially setup Consul's [ACL
|
||||
system](https://www.consul.io/docs/acl/acl-system.html).
|
||||
|
||||
Read the Consul Helm chart documentation to review all the [global
|
||||
parameters](https://www.consul.io/docs/platform/k8s/helm.html#v-global).
|
||||
|
||||
### Consul UI
|
||||
|
||||
To enable the Consul web UI update the `ui` section to your values file and set
|
||||
`enabled` to `true`.
|
||||
|
||||
Note, you can also set up a [loadbalancer
|
||||
resource](https://github.com/hashicorp/demo-consul-101/tree/master/k8s#implement-load-balancer)
|
||||
or other service type in Kubernetes to make it easier to access the UI.
|
||||
|
||||
### Consul Servers
|
||||
|
||||
For production deployments, you will need to deploy [3 or 5 Consul
|
||||
servers](https://www.consul.io/docs/internals/consensus.html#deployment-table)
|
||||
for quorum and failure tolerance. For most deployments, 3 servers are adequate.
|
||||
|
||||
In the server section set both `replicas` and `bootstrapExpect` to 3. This will
|
||||
deploy three servers and cause Consul to wait to perform leader election until
|
||||
all three are healthy. The `resources` will depend on your environment; in the
|
||||
example at the end of the guide, the resources are set for a large environment.
|
||||
|
||||
#### Affinity
|
||||
|
||||
To ensure the Consul servers are placed on different Kubernetes nodes, you will
|
||||
need to configure affinity. Otherwise, the failure of one Kubernetes node could
|
||||
cause the loss of multiple Consul servers, and result in quorum loss. By
|
||||
default, the example `values.yaml` has affinity configured correctly.
|
||||
|
||||
#### Enterprise License
|
||||
|
||||
If you have an [Enterprise
|
||||
license](https://www.hashicorp.com/products/consul/enterprise) you should
|
||||
reference the Kubernetes secret in the `enterpriseLicense` parameter.
|
||||
|
||||
Read the Consul Helm chart documentation to review all the [server
|
||||
parameters](https://www.consul.io/docs/platform/k8s/helm.html#v-server)
|
||||
|
||||
### Consul Clients
|
||||
|
||||
A Consul client is deployed on every Kubernetes node, so you do not need to
|
||||
specify the number of clients for your deployments. You will need to specify
|
||||
resources and enable gRPC. The resources in the example at the end of this guide
|
||||
should be
|
||||
sufficient for most production scenarios since Consul clients are designed for
|
||||
horizontal scalability. Enabling `grpc` enables the GRPC listener on port 8502
|
||||
and exposes it to the host. It is required to use Consul Connect.
|
||||
|
||||
Read the Consul Helm chart documentation to review all the [client
|
||||
parameters](https://www.consul.io/docs/platform/k8s/helm.html#v-client)
|
||||
|
||||
### Consul Connect Injection Security
|
||||
|
||||
Even though you enabled Consul server communication over Connect in the server section, you will also
|
||||
need to enable `connectInject` by setting `enabled` to `true`. In the
|
||||
`connectInject` section you will also configure security features. Enabling the
|
||||
`default` parameter will allow the injector to automatically inject the Connect
|
||||
sidecar into all pods. If you would prefer to manually annotate which pods to inject, you
|
||||
can set this to false. Setting the 'aclBindingRuleSelector` parameter to
|
||||
`serviceaccount.name!=default` ensures that new services do not all receive the
|
||||
same token if you are only using a default service account. This setting is
|
||||
only necessary if you have enabled ACLs in the global section.
|
||||
|
||||
Read more about the [Connect Inject
|
||||
parameters](https://www.consul.io/docs/platform/k8s/helm.html#v-connectinject).
|
||||
|
||||
## Complete Example
|
||||
|
||||
Your finished values file should resemble the following example. For more
|
||||
complete descriptions of all the available parameters see the `values.yaml`
|
||||
file provided with the Helm chart and the [reference
|
||||
documentation](https://www.consul.io/docs/platform/k8s/helm.html).
|
||||
|
||||
```yaml
|
||||
# Configure global settings in this section.
|
||||
global:
|
||||
# Enable all the components within this chart by default.
|
||||
enabled: true
|
||||
# Specify the Consul and consul-k8s images to use
|
||||
image: "consul:1.5.0"
|
||||
imagek8s: "hashicorp/consul-k8s:0.8.1"
|
||||
domain: consul
|
||||
datacenter: primarydc
|
||||
# Bootstrap ACLs within Consul. This is highly recommended.
|
||||
bootstrapACLs: true
|
||||
# Gossip encryption
|
||||
gossipEncryption: |
|
||||
secretName: "encrypt-key"
|
||||
secretKey: "key
|
||||
# Configure your Consul servers in this section.
|
||||
server:
|
||||
enabled: true
|
||||
connect: true
|
||||
# Specify three servers that wait till all are healthy to bootstrap the Consul cluster.
|
||||
replicas: 3
|
||||
bootstrapExpect: 3
|
||||
# Specify the resources that servers request for placement. These values will serve a large environment.
|
||||
resources: |
|
||||
requests:
|
||||
memory: "32Gi"
|
||||
cpu: "4"
|
||||
disk: "50Gi"
|
||||
limits:
|
||||
memory: "32Gi"
|
||||
cpu: "4"
|
||||
disk: "50Gi"
|
||||
# If using Enterprise, reference the Kubernetes secret that holds your license here
|
||||
enterpriseLicense:
|
||||
secretName: "consul-license"
|
||||
secretKey: "key"
|
||||
# Prevent Consul servers from co-location on Kubernetes nodes.
|
||||
affinity: |
|
||||
podAntiAffinity:
|
||||
requiredDuringSchedulingIgnoredDuringExecution:
|
||||
- labelSelector:
|
||||
matchLabels:
|
||||
app: {{ template "consul.name" . }}
|
||||
release: "{{ .Release.Name }}"
|
||||
component: server
|
||||
topologyKey: kubernetes.io/hostname
|
||||
# Configure Consul clients in this section
|
||||
client:
|
||||
enabled: true
|
||||
# Specify the resources that clients request for deployment.
|
||||
resources: |
|
||||
requests:
|
||||
memory: "8Gi"
|
||||
cpu: "2"
|
||||
disk: "15Gi"
|
||||
limits:
|
||||
memory: "8Gi"
|
||||
cpu: "2"
|
||||
disk: "15Gi"
|
||||
grpc: true
|
||||
# Enable and configure the Consul UI.
|
||||
ui:
|
||||
enabled: true
|
||||
# Configure security for Consul Connect pod injection
|
||||
connectInject:
|
||||
enabled: true
|
||||
default: true
|
||||
namespaceSelector: "my-namespace"
|
||||
aclBindingRuleSelector: “serviceaccount.name!=default”
|
||||
```
|
||||
## Deploy Consul
|
||||
|
||||
Now that you have customized the `values.yml` file, you can deploy Consul with
|
||||
Helm. This should only take a few minutes. The Consul pods should appear in the
|
||||
Kubernetes dashboard immediately and you can monitor the deployment process
|
||||
there.
|
||||
|
||||
```sh
|
||||
$ helm install ./consul-helm -f values.yaml
|
||||
```
|
||||
|
||||
To check the deployment process on the command line you can use `kubectl`.
|
||||
|
||||
```sh
|
||||
$ kubectl get pods
|
||||
```
|
||||
|
||||
## Summary
|
||||
|
||||
In this guide, you configured Consul, using the Helm chart, for a production
|
||||
environment. This involved ensuring that your cluster had a properly
|
||||
distributed server cluster, specifying enough resources for your agents,
|
||||
securing the cluster with ACLs and gossip encryption, and enabling other Consul
|
||||
functionality including Connect and the Consul UI.
|
||||
|
||||
Now you can interact with your Consul cluster through the UI or CLI.
|
||||
|
||||
If you exposed the UI using a load balancer it will be available at the
|
||||
`LoadBalancer Ingress` IP address and `Port` that is output from the following
|
||||
command. Note, you will need to replace _consul server_ with the server name
|
||||
from your cluster.
|
||||
|
||||
```sh
|
||||
$ kubectl describe services consul-server
|
||||
```
|
||||
|
||||
To access the Consul CLI, open a terminal session using the Kubernetes CLI.
|
||||
|
||||
```sh
|
||||
$ kubectl exec <pod name> -it /bin/ash
|
||||
```
|
||||
|
||||
To learn more about how to interact with your Consul cluster or use it for
|
||||
service discovery, configuration or segmentation, try one of Learn’s
|
||||
[Operations or Development tracks](/consul/#advanced). Follow the [Security and
|
||||
Networking track](/consul/?track=security-networking#security-networking) to
|
||||
learn more about securing your Consul cluster.
|
||||
|
||||
|
||||
Loading…
Reference in New Issue