luxolus
/
open-consul
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity

docs: consul-vault kubernetes integration docs (#11766) 

* Add documentation for the consul with vault integration that covers Server TLS, Connect CA and gossip encryption

Co-authored-by: David Yu <dyu@hashicorp.com>
Co-authored-by: Iryna Shustava <iryna@hashicorp.com>
This commit is contained in:
Kyle Schochenmaier 2021-12-15 13:59:36 -05:00 committed by GitHub
parent 8656709fc9
commit cef506ce6d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 458 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

51
website/content/docs/k8s/connect/connect-ca-provider.mdx
View File

@ -6,7 +6,7 @@ description: Configuring a Connect CA Provider
# Configuring a Connect CA Provider
~> NOTE: Instructions below should only be used for initially bootstrapping a cluster.
~> **NOTE:** The instructions below should only be used for initially bootstrapping a cluster with **Consul K8s 0.38.0+.**
To update the Connect CA provider on an existing cluster or to update any properties, such as tokens, of the CA provider,
please use the [Update CA Configuration Endpoint](/api/connect/ca#update-ca-configuration).
@ -14,17 +14,22 @@ Consul has support for different certificate authority (CA) providers to be used
Please see [Connect Certificate Management](/docs/connect/ca) for the information on the providers
we currently support.
Generally, to configure a provider via the Consul Helm chart, you need to follow three steps:
To configure a provider via the Consul Helm chart, you need to follow three steps:
1. Create a configuration file containing your provider information.
1. Create a Kubernetes secret containing the configuration file.
1. Reference the Kubernetes secret in the [`server.extraVolumes`](/docs/k8s/helm#v-server-extravolumes) value in the Helm chart.
Below we will go over this process for configuring Vault as the Connect CA.
To configure the Vault Connect Provider please see [Vault as the Service Mesh Certificate Provider on Kubernetes](/docs/k8s/installation/vault/connect-ca).
~> **NOTE:** The following instructions are only valid for Consul-k8s 0.37.0 and prior.
Below we will go over the process for configuring Vault as the Connect CA.
However, other providers can be configured similarly by providing the appropriate `ca_config`
and `ca_provider` values for the provider you're using.
## Configuring Vault as a Connect CA
## Configuring Vault as a Connect CA (Consul K8s 0.37.0 and earlier)
-> **NOTE:** If using Vault as your Connect CA, it's highly recommended to run a Consul version >= 1.8.5 that supports
token auto-renewal. With this feature, if the Vault token is [renewable](https://www.vaultproject.io/api-docs/auth/token#renewable)
@ -89,27 +94,23 @@ $ kubectl create secret generic vault-config --from-file=config=vault-config.jso
We will provide this secret and the Vault CA secret, to the Consul server via the
`server.extraVolumes` Helm value.
<CodeBlockConfig filename="config.yaml" highlight="4-13">
```yaml
global:
name: consul
server:
extraVolumes:
- type: secret
name: vault-config
load: true
items:
- key: config
path: vault-config.json
- type: secret
name: vault-ca
load: false
connectInject:
enabled: true
```
</CodeBlockConfig>
```yaml
global:
name: consul
server:
extraVolumes:
- type: secret
name: vault-config
load: true
items:
- key: config
path: vault-config.json
- type: secret
name: vault-ca
load: false
connectInject:
enabled: true
```
Finally, [install](/docs/k8s/installation/install#installing-consul) the Helm chart using the above config file:

72
website/content/docs/k8s/installation/vault/connect-ca.mdx Normal file
View File

@ -0,0 +1,72 @@
---
layout: docs
page_title: Vault as the Service Mesh Certificate Provider on Kubernetes
description: >-
Using Vault as the provider for the Service Mesh certificates on Kubernetes.
---
# Vault as the Service Mesh Certificate Provider on Kubernetes
-> **Note:** This feature requires Consul 1.11 or higher. As of v1.11,
Consul allows using Kubernetes auth methods to configure Connect CA.
This allows for automatic token rotation once the renewal is no longer possible.
To configure [Vault as the provider](/docs/connect/ca/vault) for the Consul service certificates,
you will first need to decide on the type of policy that is suitable for you.
To see the permissions that Consul would need in Vault, please see [Vault ACL policies](/docs/connect/ca/vault#vault-acl-policies)
documentation.
Once you have a policy, you will need to link that policy to the Consul server service account.
```shell-session
vault write auth/kubernetes/role/consul-server \
bound_service_account_names=<Consul server service account> \
bound_service_account_namespaces=<Consul installation namespace> \
policies=<Connect CA policy> \
ttl=1h
```
To find out the service account name of the Consul server,
you can run:
```shell-session
helm template --release-name <your release name> -s templates/server-serviceaccount.yaml hashicorp/consul
```
Now we can configure the Consul Helm chart to use Vault as the Connect CA provider:
```yaml
global:
secretsBackend:
vault:
enabled: true
consulServerRole: consul-server
consulClientRole: consul-client
consulCARole: consul-ca
connectCA:
address: <the address of the Vault server>
rootPKIPath: <the path to root PKI>
intermediatePKIPath: <the path to intermediate PKI>
ca:
secretName: <vaultCASecret>
```
The `address` you provide to the `connectCA` configuration can be a Kubernetes DNS
address if the Vault cluster is running the same Kubernetes cluster.
The `rootPKIPath` and `intermediatePKIPath` should be the same as the ones
defined in your Connect CA policy. Behind the scenes, Consul will authenticate to Vault using a Kubernetes
service account using the [Kubernetes auth method](https://www.vaultproject.io/docs/auth/kubernetes) and will use the Vault token for any API calls to Vault. If the Vault token can not be renewed, Consul will re-authenticate to
generate a new Vault token.
The `vaultCASecret` is the Kubernetes secret that stores the CA Certificate that is used for Vault communication. To provide a CA, you first need to create a Kubernetes secret containing the CA. For example, you may create a secret with the Vault CA like so:
```
kubectl create secret generic vault-ca --from-file vault.ca=/path/to/your/vault/ca
```
### Secondary Datacenters
To configure Vault as the Connect CA in secondary datacenters, you need to make sure that the Root CA path is the same,
but the intermediate is different for each datacenter. In the `connectCA` Helm configuration for a secondary datacenter,
you can specify a `intermediatePKIPath` that is, for example, prefixed with the datacenter
for which this configuration is intended (e.g. `dc2/connect-intermediate`).

92
website/content/docs/k8s/installation/vault/gossip.mdx Normal file
View File

@ -0,0 +1,92 @@
---
layout: docs
page_title: Storing Gossip Encryption Key in Vault
description: >-
Configuring the Consul Helm chart to use gossip encryption key stored in Vault.
---
# Storing Gossip Encryption Key in Vault
To use a gossip encryption key stored in Vault we need the following:
1. Generate and store an encryption key in Vault.
1. Create policies that will allow Consul client and server to access that key.
1. Create a Kubernetes auth roles that link policies from step 2 to Kubernetes service accounts of the Consul servers and clients.
## Configuring Vault
First, generate and store the gossip key in Vault:
```shell-session
vault kv put secret/consul/gossip key="$(consul keygen)"
```
Next, we will need to create a policy that allows read access to this secret:
```shell-session
# gossip-policy.hcl
path "secret/data/consul/gossip" {
capabilities = ["read"]
}
```
```shell-session
vault policy write gossip-policy gossip-policy.hcl
```
Prior to creating auth roles for the Consul server and client, ensure that the Vault Kubernetes auth method is enabled:
```shell-session
vault auth enable kubernetes
```
Next, we will create Kubernetes auth roles for the Consul server and client:
```shell-session
vault write auth/kubernetes/role/consul-server \
bound_service_account_names=<Consul server service account> \
bound_service_account_namespaces=<Consul installation namespace> \
policies=gossip-policy \
ttl=1h
```
```shell-session
vault write auth/kubernetes/role/consul-client \
bound_service_account_names=<Consul client service account> \
bound_service_account_namespaces=<Consul installation namespace> \
policies=gossip-policy \
ttl=1h
```
To find out the service account names of the Consul server and client,
you can run the following `helm template` commands with your Consul on Kubernetes values file:
```
# Generate Consul server service account name
helm template --release-name <your release name> -s templates/server-serviceaccount.yaml hashicorp/consul
# Generate Consul client service account name
helm template --release-name <your release name> -s templates/client-serviceaccount.yaml hashicorp/consul
```
## Deploying the Consul Helm chart
Now that we've configured Vault, you can configure the Consul Helm chart to
use the gossip key in Vault:
```yaml
global:
secretsBackend:
vault:
enabled: true
consulServerRole: consul-server
consulClientRole: consul-client
gossipEncryption:
secretName: secret/data/consul/gossip
secretKey: key
```
Note that `global.gossipEncryption.secretName` is the path of the secret in Vault.
This should be the same path as the one you'd include in your Vault policy.
`global.gossipEncryption.secretKey` is the key inside the secret data. This should be the same
as the key we passed when we created the gossip secret in Vault.

77
website/content/docs/k8s/installation/vault/index.mdx Normal file
View File

@ -0,0 +1,77 @@
---
layout: docs
page_title: Vault as Secrets Backend Overview
description: >-
Using Vault as secrets backend for Consul on Kubernetes.
---
# Vault as Secrets Backend Overview
By default, Consul Helm chart will expect that any credentials it needs are stored as Kubernetes secrets.
As of Consul 1.11 and Consul Helm chart v0.38.0, we integrate more natively with Vault making it easier
to use Consul Helm chart with Vault as the secrets storage backend.
At a high level, there are two points of integration with Vault:
- **Gossip encryption** - The encryption key for gossip communication is stored in Vault.
- **TLS certificates and keys**:
- **Consul Server TLS credentials** - TLS certificate and key for the Consul server is stored in Vault and issued from Vault.
- **Service Mesh and Consul client TLS credentials** - Consul uses Vault as the provider for mTLS certificates and keys for the service mesh services
and TLS certificates and keys for the Consul clients.
## Requirements
1. Vault 1.9+ and Vault-k8s 0.14+ is required.
1. Vault must be installed and accessible to the Consul on Kubernetes installation.
1. `global.tls.enableAutoencrypt=true` is required if TLS is enabled for the Consul installation when using the Vault secrets backend.
1. The Vault installation must have been initialized, unsealed and the KV2 and PKI secrets engines enabled and the Kubernetes Auth Method enabled.
A minimal valid installation of Vault must include the Agent Injector:
```yaml
injector:
enabled: "true"
```
## Known Limitations
- TLS
- `global.tls.serverAdditionalDNSSans` is not currently configurable and must be manually added to the server certificate in Vault.
- `global.tls.serverAdditionalIPSans` is not currently configurable and must be manually added to the server certificate in Vault.
- Mesh gateway is not currently supported.
- Multi-DC Federation is not currently supported.
- Certificate rotation is not currently supported, ensure the TTL for your certificates is sufficiently long. Should your certificates
expire it will be necessary to issue a `consul reload` on each server.
- CA rotation is not currently supported.
## Next Steps
To utilize Vault as a secrets backend with Consul it is necessary to add several configuration fields to the Vault installation
which bootstrap Vault Auth roles and Policies for Consul to use. For the supported Vault secrets please see the individual secret
guides and ensure to, when combining the secrets, append the Vault Policies to your Vault Kube Auth Roles via a comma separated value (i.e. `policies=gossip-policy,consul-ca,consul-server,custom-policy`).
Ex:
```shell-session
vault write auth/kubernetes/role/consul-server \
bound_service_account_names=<Consul server service account> \
bound_service_account_namespaces=<Consul installation namespace> \
policies=gossip-policy,consul-ca,consul-server \
ttl=1h
```
## Troubleshooting
The Vault integration with Consul on Kubernetes makes use of the Vault Agent Injectors. Kubernetes annotations are added to the
deployments of the Consul components which cause the Vault Agent Injector to be added as an init-container that will then attach
Vault secrets to Consul's pods at startup. Additionally the Vault Agent sidecar is added to the Consul component pods which
is responsible for synchronizing and reissuing secrets at runtime.
As a result of these additional sidecar containers the typical location for logging is expanded in the Consul components.
As a general rule the best way to troubleshoot startup issues for your Consul installation when using the Vault integration
is to establish if the `vault-agent-init` container has completed or not via `kubectl logs -f <your-consul-component> -c vault-agent-int`
and checking to see if the secrets have completed rendering.
* If the secrets are not properly rendered the underlying problem will be logged in `vault-agent-init` init-container
and generally is related to the Vault Kube Auth Role not having the correct policies for the specific secret
e.g. `global.secretsBackend.vault.consulServerRole` not having the correct policies for TLS.
* If the secrets are rendered and the `vault-agent-init` container has completed AND the Consul component has not become `Ready`,
this generally points to an issue with Consul being unable to utilize the Vault secret. This can occur if, for example, the Vault Role
created for the PKI engine does not have the correct `alt_names` or otherwise is not properly configured. The best logs for this
circumstance are the Consul container logs: `kubectl logs -f <your-consul-component> -c consul`.

170
website/content/docs/k8s/installation/vault/server-tls.mdx Normal file
View File

@ -0,0 +1,170 @@
---
layout: docs
page_title: Storing Server TLS certificates in Vault
description: >-
Configuring the Consul Helm chart to use TLS certificates issued by Vault for the Consul server.
---
# Storing Server TLS certificates in Vault
To use Vault to issue Server TLS certificates the following will be needed:
1. Bootstrap the Vault PKI engine and boostrap it with any configuration required for your infrastructure.
1. Create Vault Policies that will allow the Consul server to access the certificate issuing url.
1. Create Vault Policies that will allow the Consul components, e.g. ingress gateways, controller, to access the CA url.
1. Create Kubernetes auth roles that link these policies to the Kubernetes service accounts of the Consul components.
### Bootstrapping the PKI Engine
First, we need to bootstrap the Vault cluster by enabling and configuring the PKI Secrets Engine to be able to serve
TLS certificates to Consul. The process can be as simple as the following, or more complicated such as in this [example](https://learn.hashicorp.com/tutorials/consul/vault-pki-consul-secure-tls)
which also uses an intermediate signing authority.
* Enable the PKI Secrets Engine:
```shell-session
vault secrets enable pki
```
* Tune the engine to enable longer TTL:
```shell-session
vault secrets tune -max-lease-ttl=87600h pki
```
* Generate the root CA
```shell-session
vault write -field=certificate pki/root/generate/internal \
common_name="dc1.consul" \
ttl=87600h
```
-> **Note:** Where `common_name` is comprised of combining `global.datacenter` dot `global.domain`.
### Create Vault Policies for the Server TLS Certificates
Next we will create a policy that allows `["create", "update"]` access to the
[certificate issuing URL](https://www.vaultproject.io/api/secret/pki#generate-certificate) so the Consul servers can
fetch a new certificate/key pair.
```shell-session
# consul-server-policy.hcl
path "pki/issue/consul-server" {
capabilities = ["create", "update"]
}
```
```shell-session
vault policy write consul-server consul-server-policy.hcl
```
-> **Note:** The PKI secret path referenced by the above Policy will be your `server.serverCert.secretName` Helm value.
### Create Vault Policies for the CA URL
Next, we will create a policy that allows `["read"]` access to the [CA URL](https://www.vaultproject.io/api/secret/pki#read-certificate),
this is required for the Consul components to communicate with the Consul servers in order to fetch their auto-encryption certificates.
```shell-session
# ca-policy.hcl
path "pki/cert/ca" {
capabilities = ["read"]
}
```
```shell-session
vault policy write ca-policy ca-policy.hcl
```
-> **Note:** The PKI secret path referenced by the above Policy will be your `global.tls.caCert.secretName` Helm value.
### Create Vault Roles for the PKI engine, Consul servers and components
Next, a Vault role for the PKI engine will set the default certificate issuance parameters:
```shell-session
vault write pki/roles/consul-server \
allowed_domains="<Allowed-domains-string>" \
allow_subdomains=true \
allow_bare_domains=true \
allow_localhost=true \
generate_lease=true \
max_ttl="720h"
```
To generate the `<Allowed-domains-string>` use the following script as a template:
```shell-session
#!/bin/sh
# NAME is set to either the value from `global.name` from your Consul K8s value file, or your $HELM_RELEASE_NAME-consul
export NAME=consulk8s
# NAMESPACE is where the Consul on Kubernetes is installed
export NAMESPACE=consul
# DATACENTER is the value of `global.datacenter` from your Helm values config file
export DATACENTER=dc1
echo allowed_domains=\"$DATACENTER.consul, $NAME-server, $NAME-server.$NAMESPACE, $NAME-server.$NAMESPACE.svc\"
```
Prior to creating the Kubernetes auth roles required for Consul to securely access Vault, ensure that the Vault Kubernetes Auth method is enabled:
```shell-session
vault auth enable kubernetes
```
Finally, two Kubernetes auth roles need to be created, one for the Consul servers and one for the Consul components:
```shell-session
vault write auth/kubernetes/role/consul-server \
bound_service_account_names=<Consul server service account> \
bound_service_account_namespaces=<Consul installation namespace> \
policies=consul-server \
ttl=1h
```
To find out the service account name of the Consul server,
you can run:
```shell-session
helm template --release-name <your release name> -s templates/server-serviceaccount.yaml hashicorp/consul
```
-> **Note:** Should you enable other supported features such as gossip-encryption be sure to append additional policies to
the Kube auth role in a comma separated value e.g. `policies=consul-server,consul-gossip`
```shell-session
vault write auth/kubernetes/role/consul-ca \
bound_service_account_names="*" \
bound_service_account_namespaces=<Consul installation namespace> \
policies=consul-ca \
ttl=1h
```
The above Vault Roles will now be your Helm values for `global.secretsBackend.vault.consulServerRole` and
`global.secretsBAckend.vault.consulCARole` respectively.
## Deploying the Consul Helm chart
Now that we've configured Vault, you can configure the Consul Helm chart to
use the Server TLS certificates from Vault:
```yaml
global:
secretsBackend:
vault:
enabled: true
consulServerRole: consul-server
consulClientRole: consul-client
consulCARole: consul-ca
tls:
enabled: true
caCert:
secretName: "pki/cert/ca"
server:
serverCert:
secretName: "pki/issue/consul-server"
extraVolumes:
- type: "secret"
name: <vaultCASecret>
load: "false"
```
The `vaultCASecret` is the Kubernetes secret that stores the CA Certificate that is used for Vault communication. To provide a CA, you first need to create a Kubernetes secret containing the CA. For example, you may create a secret with the Vault CA like so:
```
kubectl create secret generic vault-ca --from-file vault.ca=/path/to/your/vault/
```

21
website/data/docs-nav-data.json
View File

@ -436,6 +436,27 @@
"path": "k8s/installation/multi-cluster/vms-and-kubernetes"
}
]
},
{
"title": "Vault as Secrets Backend",
"routes": [
{
"title": "Overview",
"path": "k8s/installation/vault"
},
{
"title": "Gossip Encryption Key",
"path": "k8s/installation/vault/gossip"
},
{
"title": "Server TLS",
"path": "k8s/installation/vault/server-tls"
},
{
"title": "Service Mesh Certificates",
"path": "k8s/installation/vault/connect-ca"
}
]
}
]
},