luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity

Switch to cfssl and get everything working

Browse source
This commit is contained in:
Michael Schurter 2017-07-27 14:47:43 -07:00
parent ac7009085b
commit a11244ac62
2 changed files with 115 additions and 116 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
demo/vagrant/Vagrantfile vendored
View file

@ -7,8 +7,7 @@ sudo apt-get update
sudo DEBIAN_FRONTEND=noninteractive apt-get install -y unzip curl vim \
apt-transport-https \
ca-certificates \
software-properties-common \
openssl
software-properties-common
# Download Nomad
NOMAD_VERSION=0.6.0
@ -68,6 +67,13 @@ EOF
sudo systemctl enable consul.service
sudo systemctl start consul
for bin in cfssl cfssl-certinfo cfssljson
do
echo "Installing $bin..."
curl -sSL https://pkg.cfssl.org/R1.2/${bin}_linux-amd64 > /tmp/${bin}
sudo install /tmp/${bin} /usr/local/bin/${bin}
done
SCRIPT
Vagrant.configure(2) do |config|

221
website/source/guides/tls.html.md
View file

@ -24,7 +24,7 @@ the expected region and configured for the expected role (e.g.
`client.us-west.nomad`).
Configuring TLS can be unfortunately complex process, but if you used the
[Getting Started guide's Vagrantfile][Vagrantfile] or have [OpenSSL][] and Nomad
[Getting Started guide's Vagrantfile][Vagrantfile] or have [cfssl][] and Nomad
installed this guide will provide you with a production ready TLS
configuration.
@ -38,72 +38,75 @@ XXX TODO XXX - serf encryption key
The first step to configuring TLS for Nomad is generating certificates. In
order to prevent unauthorized cluster access, Nomad requires all certificates
are signed by the sign Certificate Authority (CA). This should be a *private*
CA and not a public like [Let's Encrypt][letsencrypt] as any certificate signed
by this CA will be allowed to communicate with the cluster.
be signed by the same Certificate Authority (CA). This should be a *private* CA
and not a public one like [Let's Encrypt][letsencrypt] as any certificate
signed by this CA will be allowed to communicate with the cluster.
### Certificate Authority
You can generate a private CA certificate and key with OpenSSL:
There are a variety of tools for managing your own CA, [like the PKI secret
backend in Vault][vault-pki], but for the sake of simplicity in this guide
we'll use [cfssl][]. You can generate a private CA certificate and key with
[cfssl][]:
```shell
# Generate the CA's private key
# This file (nomad-ca.key) must be kept *secret*
openssl genrsa -out nomad-ca.key 4096
# Generate the CA's self-signed certicate
# This file (nomad-ca.crt) will be distributed to all nodes
openssl req -new -x509 -key nomad-ca.key -out nomad-ca.crt
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:.
State or Province Name (full name) [Some-State]:.
Locality Name (eg, city) []:.
Organization Name (eg, company) [Internet Widgits Pty Ltd]:.
Organizational Unit Name (eg, section) []:.
Common Name (e.g. server FQDN or YOUR name) []:Nomad CA
Email Address []:.
# Generate the CA's private key and certificate
cfssl print-defaults csr | cfssl gencert -initca - | cfssljson -bare nomad-ca
```
Your answers to OpenSSL's prompts are purely informational and not used by
Nomad.
The CA key (`nomad-ca.key`) will be used to sign certificates for Nomad nodes
and must be kept private. The CA certificate (`nomad-ca.crt`) contains the
public key necessary to validate Nomad certificates and therefore must be
The CA key (`nomad-ca-key.pem`) will be used to sign certificates for Nomad
nodes and must be kept private. The CA certificate (`nomad-ca.pem`) contains
the public key necessary to validate Nomad certificates and therefore must be
distributed to every node that requires access.
### Node Certificates
Once you have a CA certifacte and key you can generate and sign the
certificates Nomad will use directly. Traditionally TLS certificates use the
certificates Nomad will use directly. TLS certificates commonly use the
fully-qualified domain name of the system being identified as the certificate's
Common Name (CN). However, hosts (and therefore hostnames and IPs) are often
ephemeral in Nomad clusters. They come and go as clusters are scaled up and
down or outages occur. Not only would signing a new certificate per Nomad node
be difficult, but using a hostname provides no security or functional benefits
to Nomad. To fulfill the desired security properties (see above) Nomad
certificates are signed with their region and role such as:
to Nomad. To fulfill the desired security properties (above) Nomad certificates
are signed with their region and role such as:
* `client.global.nomad` for a client node in the `global` region
* `server.us-west.nomad` for a server node in the `us-west` region
To create certificates for the client and server in the cluster from the
[Getting Started guide][guide-cluster] with OpenSSL create the following
configuration file `nomad.conf`:
[Getting Started guide][guide-cluster] with [cfssl][] create ([or
download][cfssl.json]) the following configuration file as `cfssl.json` to
increase the default certificate expiration time:
```ini
basicConstraints = CA:FALSE
subjectAltName = @alt_names
```json
{
"signing": {
"default": {
"expiry": "87600h",
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
]
}
}
}
```
[alt_names]
DNS.1 = ${commonName}
DNS.2 = localhost
```shell
# Generate a certificate for the Nomad server
echo '{}' | cfssl gencert -ca=nomad-ca.pem -ca-key=nomad-ca-key.pem -config=cfssl.json \
-hostname="server.global.nomad,localhost" - | cfssljson -bare server
# Generate a certificate for the Nomad client
echo '{}' | cfssl gencert -ca=nomad-ca.pem -ca-key=nomad-ca-key.pem -config=cfssl.json \
-hostname="client.global.nomad,localhost" - | cfssljson -bare client
# Generate a certificate for the CLI
echo '{}' | cfssl gencert -ca nomad-ca.pem -ca-key nomad-ca-key.pem -profile=client \
- | cfssljson -bare cli
```
Using `localhost` as a subject alternate name (SAN) allows tools like `curl` to
@ -111,61 +114,32 @@ be able to communicate with Nomad's HTTP API when run on the same host. Other
SANs may be added including a DNS resolvable hostname to allow remote HTTP
requests from third party tools.
Then create client and server certificate and key pairs:
```shell
# Client key and certificate
openssl genrsa -out client.global.nomad.key 4096
openssl req -new -sha256 \
-out client.global.nomad.csr \
-key client.global.nomad.key \
-subj /CN=client.global.nomad/
openssl x509 -req \
-in client.global.nomad.csr \
-CA nomad-ca.crt \
-CAkey nomad-ca.key \
-days 3650 \
-set_serial $(hexdump -e '"0x%x%x%x%x"' -n 16 /dev/urandom) \
-extfile nomad.conf \
-out client.global.nomad.crt
# Server key and certificate
openssl genrsa -out server.global.nomad.key 4096
openssl req -new -sha256 \
-out server.global.nomad.csr \
-key server.global.nomad.key \
-subj /CN=server.global.nomad/
openssl x509 -req \
-in server.global.nomad.csr \
-CA nomad-ca.crt \
-CAkey nomad-ca.key \
-days 3650 \
-set_serial $(hexdump -e '"0x%x%x%x%x"' -n 16 /dev/urandom) \
-extfile nomad.conf \
-out server.global.nomad.crt
```
You should now have the following files:
* `nomad-ca.key` - CA private key. Keep safe!
* `nomad-ca.crt` - CA public certificate.
* `client.global.nomad.key` - Nomad client node private key for the `global` region.
* `client.global.nomad.csr` - Nomad client node certificate signing request for the `global` region.
* `client.global.nomad.crt` - Nomad client node public certificate for the `global` region.
* `server.global.nomad.key` - Nomad server node private key for the `global` region.
* `server.global.nomad.csr` - Nomad server node certificate signing request for the `global` region.
* `server.global.nomad.crt` - Nomad server node public certificate for the `global` region.
* `cfssl.json` - cfssl configuration.
* `nomad-ca.csr` - CA signing request.
* `nomad-ca-key.pem` - CA private key. Keep safe!
* `nomad-ca.pem` - CA public certificate.
* `cli.csr` - Nomad CLI certificate signing request.
* `cli.pem` - Nomad CLI certificate.
* `cli-key.pem` - Nomad CLI private key.
* `client.csr` - Nomad client node certificate signing request for the `global` region.
* `client-key.pem` - Nomad client node private key for the `global` region.
* `client.pem` - Nomad client node public certificate for the `global` region.
* `server.csr` - Nomad server node certificate signing request for the `global` region.
* `server-key.pem` - Nomad server node private key for the `global` region.
* `server.pem` - Nomad server node public certificate for the `global` region.
Each Nomad node should have the appropriate key (`.key`) and certificate
(`.crt`) file for its region and role. In addition each node needs the CA's
public certificate (`nomad-ca.crt`).
Each Nomad node should have the appropriate key (`-key.pem`) and certificate
(`.pem`) file for its region and role. In addition each node needs the CA's
public certificate (`nomad-ca.pem`).
## Configuring Nomad
Once you have the appropriate key and certificates installed you're ready to
configure Nomad to use them for mTLS. Starting with the [server configuration
from the Getting Started guide][guide-server] add the following TLS specific
configuration options:
from the Getting Started guide][guide-server] add the following TLS
CONFIGUration options:
```hcl
# Increase log verbosity
@ -187,9 +161,9 @@ tls {
http = true
rpc = true
ca_file = "nomad-ca.crt"
cert_file = "server.global.nomad.crt"
key_file = "server.global.nomad.key"
ca_file = "nomad-ca.pem"
cert_file = "server.pem"
key_file = "server-key.pem"
verify_server_hostname = true
verify_https_client = true
@ -208,13 +182,13 @@ doesn't use separate ports for TLS and non-TLS traffic: your cluster should
either use TLS or not.
```hcl
ca_file = "nomad-ca.crt"
cert_file = "server.global.nomad.crt"
key_file = "server.global.nomad.key"
ca_file = "nomad-ca.pem"
cert_file = "server.pem"
key_file = "server-key.pem"
```
The file lines should point to whereever you placed the certificate files on
the node. This guide assumes they're in Nomad's current directory.
the node. This guide assumes they are in Nomad's current directory.
```hcl
verify_server_hostname = true
@ -227,11 +201,12 @@ cerificate will be checked to ensure it is signed by the same CA, but its role
and region will not be verified. This means any service with a certificate from
the same CA as Nomad can act as a client or server of any region.
`verify_https_client` may be disabled to allow HTTP API clients (eg Nomad CLI, Consul, or
curl) to communicate with the HTTPS API without presenting a client-side
certificate. If `verify_https_client` is enabled ony HTTP API clients
presenting a certificate signed by the same CA as Nomad's certificate are
allowed to access Nomad.
`verify_https_client` requires HTTP API clients to present a certificate signed
by the same CA as Nomad's certificate. It may be disabled to allow HTTP API
clients (eg Nomad CLI, Consul, or curl) to communicate with the HTTPS API
without presenting a client-side certificate. If `verify_https_client` is
enabled ony HTTP API clients presenting a certificate signed by the same CA as
Nomad's certificate are allowed to access Nomad.
~> Enabling `verify_https_client` feature effectively protects Nomad from
unauthorized network access at the cost of breaking compatibility with Consul
@ -269,9 +244,9 @@ tls {
http = true
rpc = true
ca_file = "nomad-ca.crt"
cert_file = "client.global.nomad.crt"
key_file = "client.global.nomad.key"
ca_file = "nomad-ca.pem"
cert_file = "client.pem"
key_file = "client-key.pem"
verify_server_hostname = true
verify_https_client = true
@ -307,17 +282,33 @@ Don't worry, the Nomad CLI just defaults to `http://...` instead of
```shell
export NOMAD_ADDR=https://localhost:4646
export NOMAD_CACERT=nomad-ca.crt
export NOMAD_CLIENT_CERT=client.global.nomad.crt
export NOMAD_CACERT=nomad-ca.pem
export NOMAD_CLIENT_CERT=client.pem
export NOMAD_CLIENT_KEY=client-key.pem
```
The `NOMAD_CACERT` also needs to be set so the CLI can verify it's talking to
an actual Nomad node. Finally, the `NOMAD_CLIENT_CERT` needs to be set since we
enabled `verify_https_client` above which prevents any access lacking a client
certificate. Operators may wish to generate a certificate specifically for the
CLI as any certificate signed by Nomad's CA will work.
an actual Nomad node. Finally, `NOMAD_CLIENT_CERT` and `NOMAD_CLIENT_KEY` need
to be set since we enabled `verify_https_client` above which prevents any
access lacking a client certificate.
XXX TODO XXX - an example of everything working
Now the CLI works as expected:
```text
vagrant@nomad:~$ nomad node-status
ID DC Name Class Drain Status
237cd4c5 dc1 nomad <none> false ready
vagrant@nomad:~$ nomad init
Example job file written to example.nomad
vagrant@nomad:~$ nomad run example.nomad
==> Monitoring evaluation "e9970e1d"
Evaluation triggered by job "example"
Allocation "a1f6c3e7" created: node "237cd4c5", group "cache"
Evaluation within deployment: "080460ce"
Evaluation status changed: "pending" -> "complete"
==> Evaluation "e9970e1d" finished with status "complete"
```
## Switching an existing cluster to TLS
@ -326,6 +317,8 @@ XXX TODO XXX
[guide-server]: https://raw.githubusercontent.com/hashicorp/nomad/master/demo/vagrant/server.hcl
[guide-cluster]: https://www.nomadproject.io/intro/getting-started/cluster.html
[letsencrypt]: https://letsencrypt.org/
[OpenSSL]: https://www.openssl.org/
[cfssl]: https://cfssl.org/
[cfssl.json]: https://raw.githubusercontent.com/hashicorp/nomad/master/demo/vagrant/cfssl.json
[Vagrantfile]: https://raw.githubusercontent.com/hashicorp/nomad/master/demo/vagrant/Vagrantfile
[Vault]: https://www.vaultproject.io/docs/secrets/pki/index.html
[Vault]: https://www.vaultproject.io/
[vault-pki]: https://www.vaultproject.io/docs/secrets/pki/index.html