luxolus/open-nomad
2
0
Fork 0
Code Issues 1 Pull requests Packages Releases Wiki Activity

Merge pull request #457 from hashicorp/docs-networking

Browse source 
Documentation for ports and other networking things
This commit is contained in:
Chris Bednarski 2015-11-18 17:49:42 -08:00
parent 07c8d4d1ab 51163eea36
commit 49446ba607
4 changed files with 261 additions and 86 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

216
website/source/docs/drivers/docker.html.md
View file

@ -16,28 +16,48 @@ and cleaning up after containers.
## Task Configuration
The `docker` driver supports the following configuration in the job
specification:
The `docker` driver is configured via a `config` block:
* `image` - The Docker image to run. The image may include a tag or
custom URL. By default it will be fetched from Docker Hub.
```
task "webservice" {
driver = "docker"
config = {
image = "redis"
labels = {
group = "webservice-cache"
}
}
}
```
The following options are available for use in the job specification.
* `image` - The Docker image to run. The image may include a tag or custom URL.
By default it will be fetched from Docker Hub.
* `command` - (Optional) The command to run when starting the container.
* `args` - (Optional) A list of arguments to the optional `command`. If no
`command` is present, `args` are ignored.
* `labels` - (Optional) A key/value map of labels to set to the containers on
start.
* `privileged` - (Optional) `true` or `false` (default). Privileged mode gives
the container access to devices on the host. Note that this also requires the
nomad agent and docker daemon to be configured to allow privileged
containers.
* `network_mode` - (Optional) The network mode to be used for the container. In
order to support userspace networking plugins in Docker 1.9 this accepts any
value. The default is `bridge`. Other networking modes may not work without
additional configuration on the host (which is outside the scope of Nomad).
Valid values pre-docker 1.9 are `default`, `bridge`, `host`, `none`, or
`container:name`.
`container:name`. See below for more details.
* `privileged` - (Optional) Privileged mode gives the container full access to
the host. Valid options are `"true"` and `"false"` (defaults to `"false"`).
Tasks with `privileged` set can only run on Nomad Agents with
`docker.privileged.enabled = "true"`.
* `hostname` - (Optional) The hostname to assign to the container. When
launching more than one of a task (using `count`) with this option set, every
container the task starts will have the same hostname.
* `dns_servers` - (Optional) A list of DNS servers for the container to use
(e.g. ["8.8.8.8", "8.8.4.4"]). *Docker API v1.10 and above only*
@ -45,87 +65,144 @@ specification:
* `search_domains` - (Optional) A list of DNS search domains for the container
to use.
* `hostname` - (Optional) The hostname to assign to the container. When
launching more than one of a task (using `count`) with this option set, every
container the task starts will have the same hostname.
* `port_map` - (Optional) A key/value map of port labels (see below).
* `labels` - (Optional) A key/value map of labels to set to the containers on
start.
* `auth` - (Optional) Provide authentication for a private registry (see below).
**Authentication** Registry authentication can be set per task with the
following authentication parameters. These options can provide access to
private repositories that utilize the docker remote api (e.g. dockerhub,
quay.io)
### Container Name
Nomad creates a container after pulling an image. Containers are named
`{taskName}-{allocId}`. This is necessary in order to place more than one
container from the same task on a host (e.g. with count > 1). This also means
that each container's name is unique across the cluster.
This is not configurable.
### Authentication
If you want to pull from a private repo (for example on dockerhub or quay.io),
you will need to specify credentials in your job via the `auth` option.
The `auth` object supports the following keys:
* `username` - (Optional) The account username.
* `password` - (Optional) The account password.
* `email` - (Optional) The account email.
* `server_address` - (Optional) The server domain/ip without the protocol.
Docker Hub is used by default.
### Port Mapping
Nomad uses port binding to expose services running in containers using the port
space on the host's interface. For example, Nomad host running on `1.2.3.4` may
allocate port `22333` to a task, so you would access that service via
`1.2.3.4:22333`.
Nomad provides automatic and manual mapping schemes for Docker. You can use
either or both schemes for a task. Nomad binds both tcp and udp protocols to
ports used for Docker containers. This is not configurable.
Note: You are not required to map any ports, for example if your task is
running a crawler or aggregator and does not provide a network service. Tasks
without a port mapping will still be able to make outbound network connections.
#### Automatic Port Mapping
Typically when you create a Docker container you configure the service to start
listening on a port (or ports) when you start the container. For example, redis
starts listening on `6379` when you `docker run redis`. Nomad can support this
by mapping a random port on the host machine to the port inside the container.
You need to tell Nomad which ports your container is using so Nomad can map
allocated ports for you. You do so by specifying a **numeric port value** for
the `dynamic_ports` option in your job specification.
Example:
```
dynamic_ports = ["6379"]
# or
dynamic_ports = [6379]
task "secretservice" {
driver = "docker"
config {
image = "secret/service"
auth {
username = "dockerhub_user"
username = "dockerhub_password"
}
}
}
```
This instructs Nomad to create a port mapping from the random port on the host
to the port inside the container. So in our example above, when you contact the
host on `1.2.3.4:22333` you will actually hit the service running inside the
container on port `6379`. You can see which port was actually bound by reading
the `NOMAD_PORT_6379` [environment variable](/docs/jobspec/environment.html).
**Please note that these credentials are stored in Nomad in plain text.**
Secrets management will be added in a later release.
In most cases, the automatic port mapping will be the easiest to use, but you
can also use manual port mapping (described below).
## Networking
#### Manual Port Mapping
Docker supports a variety of networking configurations, including using host
interfaces, SDNs, etc. Nomad uses `bridged` networking by default, like Docker.
The `dynamic_ports` option takes any alphanumeric string as a label, so you
could also specify a label for the port like `http` or `admin` to designate how
the port will be used.
You can specify other networking options, including custom networking plugins
in Docker 1.9. **You may need to perform additional configuration on the host
in order to make these work.** This additional configuration is outside the
scope of Nomad.
In this case, Nomad doesn't know which container port to map to, so it maps 1:1
with the host port. For example, `1.2.3.4:22333` will map to `22333` inside the
### Allocating Ports
You can allocate ports to your task using the port syntax described on the
[networking page](/docs/jobspec/networking.html). Here is a recap:
```
task "webservice" {
driver = "docker"
port "http" {}
port "https" {}
}
```
### Forwarding and Exposing Ports
A Docker container typically specifies which port a service will listen on by
specifying the `EXPOSE` directive in the `Dockerfile`.
Because dynamic ports will not match the ports exposed in your Dockerfile,
Nomad will automatically expose all of the ports it allocates to your
container.
These ports will be identified via environment variables. For example:
```
dynamic_ports = ["http"]
port "http" {}
```
Your process will need to read the `NOMAD_PORT_HTTP` environment variable to
determine which port to bind to.
If Nomad allocates port `23332` to your task for `http`, `23332` will be
automatically exposed and forwarded to your container, and the driver will set
an environment variable `NOMAD_PORT_http` with the value `23332` that you can
read inside your container.
## Client Requirements
This provides an easy way to use the `host` networking option for better
performance.
### Using the Port Map
If you prefer to use the traditional port-mapping method, you can specify the
`port_map` option in your job specification. It looks like this:
```
task "redis" {
driver = "docker"
port "redis" {}
config {
image = "redis"
port_map {
redis = "6379"
}
}
}
```
If Nomad allocates port `23332` to your task, the Docker driver will
automatically setup the port mapping from `23332` on the host to `6379` in your
container, so it will just work!
Note that by default this only works with `bridged` networking mode. It may
also work with custom networking plugins which implement the same API for
expose and port forwarding.
### Networking Protocols
The Docker driver configures ports on both the `tcp` and `udp` protocols.
This is not configurable.
### Other Networking Modes
Some networking modes like `container` or `none` will require coordination
outside of Nomad. First-class support for these options may be improved later
through Nomad plugins or dynamic job configuration.
## Host Requirements
Nomad requires Docker to be installed and running on the host alongside the
Nomad agent. Nomad was developed against Docker `1.8.2`.
Nomad agent. Nomad was developed against Docker `1.8.2` and `1.9`.
By default Nomad communicates with the Docker daemon using the daemon's unix
socket. Nomad will need to be able to read/write to this socket. If you do not
@ -140,9 +217,9 @@ user to the `docker` group so you can run Nomad without root:
For the best performance and security features you should use recent versions
of the Linux Kernel and Docker daemon.
## Client Configuration
## Agent Configuration
The `docker` driver has the following configuration options:
The `docker` driver has the following host-level configuration options:
* `docker.endpoint` - Defaults to `unix:///var/run/docker.sock`. You will need
to customize this if you use a non-standard socket (http or another
@ -155,14 +232,15 @@ The `docker` driver has the following configuration options:
prevent Nomad from removing images from stopped tasks.
* `docker.privileged.enabled` Defaults to `false`. Changing this to `true` will
allow containers to use "privileged" mode, which gives the containers full
access to the host.
allow containers to use `privileged` mode, which gives the containers full
access to the host's devices. Note that you must set a similar setting on the
Docker daemon for this to work.
Note: When testing or using the `-dev` flag you can use `DOCKER_HOST`,
`DOCKER_TLS_VERIFY`, and `DOCKER_CERT_PATH` to customize Nomad's behavior. In
production Nomad will always read `docker.endpoint`.
## Client Attributes
## Agent Attributes
The `docker` driver will set the following client attributes:

19
website/source/docs/jobspec/environment.html.md
View file

@ -36,24 +36,9 @@ job specification without needing to change your code. You can also schedule wor
that accept dynamic resource allocations so they can scale down/up as your
cluster gets more or less busy.
### IPs and Named Ports
### Networking
Each task will receive port allocations on a single IP address. The IP is made
available through `NOMAD_IP.`
Both dynamic and reserved ports are exposed through environment variables in the
following format, `NOMAD_PORT_{LABEL}={PORT}`. For example, a dynamic port
`port "HTTP" {}` becomes `NOMAD_PORT_HTTP=48907`.
Some drivers such as Docker and QEMU use port mapping. If a driver supports port
mapping and it has been set in the driver configuration, container ports and
their mapped host port are exposed as environment variables with the following
format, `NOMAD_PORT_{CONTAINER_PORT}={HOST_PORT}`. To give a concrete example,
imagine you had the following port configuration, `port "db" { static = 8181 }`
and you had the following port mapping, `port_map { db = 6379 }`. The following
environment variable would then be set, `NOMAD_PORT_6379=8181`.
Please see the relevant driver documentation for details.
Nomad assigns IPs and ports to your jobs and exposes them via environment variables. See the [Networking](/docs/jobspec/networking.html) page for more details.
### Task Directories <a id="task_dir"></a>

109
website/source/docs/jobspec/networking.html.md Normal file
View file

@ -0,0 +1,109 @@
---
layout: "docs"
page_title: "Nomad Networking"
sidebar_current: "docs-jobspec-networking"
description: |-
Learn how to configure networking and ports for Nomad tasks.
---
# Networking
When scheduling jobs in Nomad they are provisioned across your fleet of
machines along with other jobs and services. Because you don't know in advance
what host your job will be provisioned on, Nomad will provide your task with
network configuration when they start up.
Note that this document applies only applies to services that want to _listen_
on a port. Batch jobs or services that only make outbound connections do not
need to allocate ports, since they will use any available interface to make an
outbound connection.
## IP Address
Hosts in Nomad may have multiple network interfaces attached to them. This
allows you to have a higher density of services, or bind to interfaces on
different subnets (for example public vs. private subnets).
Each task will receive port allocations on a single interface. The IP is passed
to your job via the `NOMAD_IP` environment variable.
## Ports
In addition to allocating an interface, Nomad can allocate static or dynamic
ports to your task.
### Dynamic Ports
Dynamic ports are allocated in a range from `20000` to `60000`.
Most services run in your cluster should use dynamic ports. This means that the
port will be allocated dynamically by the scheduler, and your service will have
to read an environment variable (see below) to know which port to bind to at
startup.
```
task "webservice" {
port "http" {}
port "https" {}
}
```
### Static Ports
Static ports bind your job to a specific port on the host they're placed on.
Since multiple services cannot share a port, the port must be open in order to
place your task.
```
task "dnsservice" {
port "dns" {
static = "53"
}
}
```
We recommend _only_ using static ports for [system
jobs](/docs/jobspec/schedulers.html) or specialized jobs like load balancers.
### Labels and Environment Variables
The label assigned to the port is used to identify the port in service
discovery, and used for the name of the environment variable that indicates
which port your application should bind to. For example, we've labeled this
port `http`:
```
port "http" {}
```
When the task is started, it is passed an environment variable named
`NOMAD_PORT_http` which indicates the port.
```
NOMAD_PORT_http=53423 ./start-command
```
### Mapped Ports
Some drivers (such as Docker and QEMU) allow you to map ports. A mapped port
means that your application can listen on a fixed port (it does not need to
read the environment variable) and the dynamic port will be mapped to the port
in your container or VM.
```
driver = "docker"
port "http" {}
config {
port_map = {
http = 8080
}
}
```
The above example is for the Docker driver. The service is listening on port
`8080` inside the container. The driver will automatically map the dynamic port
to this service.
Please refer to the [Docker](/docs/drivers/docker.html) and [QEMU](/docs/drivers/qemu.html) drivers for additional information.

3
website/source/layouts/docs.erb
View file

@ -44,6 +44,9 @@
<li<%= sidebar_current("docs-jobspec-service-discovery") %>>
<a href="/docs/jobspec/servicediscovery.html">Service Discovery</a>
</li>
<li<%= sidebar_current("docs-jobspec-networking") %>>
<a href="/docs/jobspec/networking.html">Networking</a>
</li>
</ul>
</li>