open-nomad/terraform/README.md

# Provision a Nomad cluster on AWS with Packer & Terraform

Use this to easily provision a Nomad sandbox environment on AWS with [Packer](https://packer.io) and [Terraform](https://terraform.io). [Consul](https://www.consul.io/intro/index.html) and [Vault](https://www.vaultproject.io/intro/index.html) are also installed (colocated for convenience). The intention is to allow easy exploration of Nomad and its integrations with the HashiCorp stack. This is not meant to be a production ready environment. A demonstration of [Nomad's Apache Spark integration](examples/spark/README.md) is included. 

## Setup

Clone this repo and (optionally) use [Vagrant](https://www.vagrantup.com/intro/index.html) to bootstrap a local staging environment:

```bash
$ git clone git@github.com:hashicorp/nomad.git
$ cd terraform/aws
$ vagrant up && vagrant ssh
```

The Vagrant staging environment pre-installs Packer, Terraform and Docker.

### Pre-requisites

You will need the following:

- AWS account
- [API access keys](http://aws.amazon.com/developers/access-keys/)
- [SSH key pair](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html)

Set environment variables for your AWS credentials:

```bash
$ export AWS_ACCESS_KEY_ID=[ACCESS_KEY_ID]
$ export AWS_SECRET_ACCESS_KEY=[SECRET_ACCESS_KEY]
```

## Provision a cluster

`cd` to an environment subdirectory:

```bash
$ cd env/us-east
```

Update terraform.tfvars with your SSH key name:

```bash
region                  = "us-east-1"
ami                     = "ami-68291d7e"
instance_type           = "t2.medium"
key_name                = "KEY_NAME"
server_count            = "3"
client_count            = "4"
```

Note that a pre-provisioned, publicly available AMI is used by default (for the `us-east-1` region). To provision your own customized AMI with [Packer](https://www.packer.io/intro/index.html), follow the instructions [here](aws/packer/README.md). You will need to replace the AMI ID in terraform.tfvars with your own. You can also modify the `region`, `instance_type`, `server_count` and `client_count`. At least one client and one server are required.

Provision the cluster:

```bash
terraform get
terraform plan
terraform apply
```

## Access the cluster

SSH to one of the servers using its public IP:

```bash
$ ssh -i /path/to/key ubuntu@PUBLIC_IP
```

Note that the AWS security group is configured by default to allow all traffic over port 22. This is not recommended for production deployments.

Run a few basic commands to verify that Consul and Nomad are up and running properly:

```bash
$ consul members
$ nomad server-members
$ nomad node-status
```

Optionally, initialize and unseal Vault:

```bash
$ vault init -key-shares=1 -key-threshold=1
$ vault unseal
$ export VAULT_TOKEN=[INITIAL_ROOT_TOKEN]
```

The `vault init` command above creates a single [Vault unseal key](https://www.vaultproject.io/docs/concepts/seal.html). For a production environment, it is recommended that you create at least five unseal key shares and securely distribute them to independent operators. The `vault init` command defaults to five key shares and a key threshold of three. If you provisioned more than one server, the others will become standby nodes (but should still be unsealed). You can query the active and standby nodes independently:

```bash
$ dig active.vault.service.consul
$ dig active.vault.service.consul SRV
$ dig standby.vault.service.consul
``` 

## Getting started with Nomad & the HashiCorp stack

See:

* [Getting Started with Nomad](https://www.nomadproject.io/intro/getting-started/jobs.html)
* [Consul integration](https://www.nomadproject.io/docs/service-discovery/index.html)
* [Vault integration](https://www.nomadproject.io/docs/vault-integration/index.html)
* [consul-template integration](https://www.nomadproject.io/docs/job-specification/template.html) 

## Apache Spark integration

Nomad is well-suited for analytical workloads, given its performance characteristics and first-class support for batch scheduling. Apache Spark is a popular data processing engine/framework that has been architected to use third-party schedulers. The Nomad ecosystem includes a fork that natively integrates Nomad with Spark. A detailed walkthrough of the integration is included [here](examples/spark/README.md).