508 lines
22 KiB
Plaintext
508 lines
22 KiB
Plaintext
---
|
|
layout: docs
|
|
page_title: Manual Installation - AWS ECS
|
|
description: >-
|
|
Manually Install Consul Service Mesh on AWS ECS (Elastic Container Service).
|
|
---
|
|
|
|
# Manual Installation
|
|
|
|
While the [Consul ECS Terraform module](/docs/ecs/install) is the easiest way to use Consul on ECS,
|
|
this page will describe how to directly create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs)
|
|
for use without Terraform.
|
|
|
|
## Pre-requisites
|
|
|
|
* This page assumes you are familiar with AWS ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details.
|
|
* This page does not show how to create all necessary AWS resources, such as a VPC or the ECS Cluster.
|
|
For complete runnable examples, see the links in the [Getting Started](/docs/ecs#getting-started) section.
|
|
|
|
## Task Definition
|
|
|
|
You must define a Task Definition which includes the following containers:
|
|
|
|
* Your application container
|
|
* An Envoy sidecar-proxy container
|
|
* A Consul client container
|
|
* The `consul-ecs-mesh-init` container for service mesh setup
|
|
* Optionally, a `consul-ecs-health-sync` container to sync ECS health checks into Consul
|
|
|
|
## Top-level fields
|
|
|
|
In your task definition, you'll need to define these important top-level fields:
|
|
|
|
```json
|
|
{
|
|
"family": "my-example-client-app",
|
|
"networkMode": "awsvpc",
|
|
"volumes": [
|
|
{
|
|
"name": "consul_data",
|
|
},
|
|
{
|
|
"name": "consul_binary",
|
|
}
|
|
],
|
|
"containerDefinitions": [...]
|
|
}
|
|
```
|
|
|
|
| Field name | Type | Description |
|
|
| ---------------------- | ------ | ------------------------------------------------------------------------------------------------------------------ |
|
|
| `family` | string | The task family name. This is used as the Consul service name, by default. |
|
|
| `networkMode` | string | Must be `awsvpc`, which is the only network mode supported by Consul on ECS. |
|
|
| `volumes` | list | Must be defined as shown above. Volumes ares used to share configuration between containers for intial task setup. |
|
|
| `containerDefinitions` | list | The list of containers to run in this task (see below). |
|
|
|
|
## Application container
|
|
|
|
First, include your application container in the `containerDefinitions` list
|
|
in the task definition. Ensure that the `dependsOn` field is set as shown below
|
|
so that your application container starts in the correct order (see [task
|
|
startup](/docs/ecs/architecture#task-startup) for more information):
|
|
|
|
```json
|
|
{
|
|
"containerDefinitions": [
|
|
{
|
|
"name": "example-client-app",
|
|
"image": "docker.io/org/my_task:v0.0.1",
|
|
"essential": true,
|
|
"dependsOn": [
|
|
{
|
|
"containerName": "consul-ecs-mesh-init",
|
|
"condition": "SUCCESS"
|
|
},
|
|
{
|
|
"containerName": "sidecar-proxy",
|
|
"condition": "HEALTHY"
|
|
}
|
|
],
|
|
...
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
| Field name | Type | Description |
|
|
| ----------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `name` | string | The name of your application container. |
|
|
| `image` | string | The container image used to run your application. |
|
|
| `essential` | boolean | Must be `true` to ensure your application container ties into the health of the task. |
|
|
| `dependsOn` | list | Must be set as show above. Container dependencies ensure your application container starts after service mesh setup is complete. |
|
|
|
|
See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html) documentation for a complete reference.
|
|
|
|
## `sidecar-proxy` container
|
|
|
|
The sidecar proxy container runs [Envoy proxy](/docs/connect/proxies/envoy) for Consul Connect.
|
|
|
|
<CodeBlockConfig highlight="8-40">
|
|
|
|
```json
|
|
{
|
|
"containerDefinitions": [
|
|
{
|
|
"name": "example-client-app",
|
|
"image": "docker.io/org/my_task:v0.0.1",
|
|
...
|
|
},
|
|
{
|
|
"name": "sidecar-proxy",
|
|
"image": "envoyproxy/envoy-alpine:<VERSION>",
|
|
"essential": false,
|
|
"dependsOn": [
|
|
{
|
|
"containerName": "consul-ecs-mesh-init",
|
|
"condition": "SUCCESS"
|
|
}
|
|
],
|
|
"healthCheck": {
|
|
"retries": 3,
|
|
"command": ["nc", "-z", "127.0.0.1", "20000"],
|
|
"timeout": 5,
|
|
"interval": 30
|
|
},
|
|
"mountPoints": [
|
|
{
|
|
"readOnly": true,
|
|
"containerPath": "/consul",
|
|
"sourceVolume": "consul_data"
|
|
}
|
|
],
|
|
"ulimits": [
|
|
{
|
|
"name": "nofile",
|
|
"softLimit": 1048576,
|
|
"hardLimit": 1048576
|
|
}
|
|
],
|
|
"command": ["envoy", "--config-path", "/consul/envoy-bootstrap.json"],
|
|
"entryPoint": ["/consul/consul-ecs", "envoy-entrypoint"],
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
In most cases, the `sidecar-proxy` container can be configured exactly as shown above.
|
|
|
|
| Field name | Type | Description |
|
|
| ------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `name` | string | The container name, which must be `sidecar-proxy`. |
|
|
| `image` | string | The Envoy image. This must be a [supported version of Envoy](/docs/connect/proxies/envoy#supported-versions). |
|
|
| `dependsOn` | list | Must be set as shown above to ensure Envoy starts after the `consul-ecs-mesh-init` has written the `envoy-bootstrap.json` config file for Envoy. |
|
|
| `healthCheck` | list | Must be set as shown above to monitor the health of Envoy's primary listener port, which ties into container dependencies and startup ordering. |
|
|
| `mountPoints` | list | Must be set as shown above to access the files shared in the `consul` directory, like the Envoy bootstrap configuration file and the `consul-ecs` binary. |
|
|
| `ulimits` | list | The `nofile` ulimit must be raised to a sufficiently high value so that Envoy does not fail to open sockets. |
|
|
| `entrypoint` | list | Must be set to the custom Envoy entrypoint to facilitate graceful shutdown. |
|
|
| `command` | list | The startup command. This passes the bootstrap configuration to Envoy. |
|
|
|
|
-> **NOTE**: Envoy and Consul must be compatible versions. See the [supported versions of Envoy](/docs/connect/proxies/envoy#supported-versions) in the Consul documentation.
|
|
|
|
## `consul-client` container
|
|
|
|
Each task must include a Consul client container in order for the task to join your Consul cluster.
|
|
|
|
<CodeBlockConfig highlight="13-31">
|
|
|
|
```json
|
|
{
|
|
"containerDefinitions": [
|
|
{
|
|
"name": "example-client-app",
|
|
"image": "docker.io/org/my_task:v0.0.1",
|
|
...
|
|
},
|
|
{
|
|
"name": "sidecar-proxy",
|
|
"image": "envoyproxy/envoy-alpine:<ENVOY_VERSION>",
|
|
...
|
|
}
|
|
{
|
|
"name": "consul-client"
|
|
"image": "public.ecr.aws/hashicorp/consul:<CONSUL_VERSION>",
|
|
"mountPoints": [
|
|
{
|
|
"readOnly": false,
|
|
"containerPath": "/consul",
|
|
"sourceVolume": "consul_data"
|
|
},
|
|
{
|
|
"containerPath": "/bin/consul-inject",
|
|
"sourceVolume": "consul_binary"
|
|
}
|
|
],
|
|
"entryPoint": ["/bin/sh", "-ec"],
|
|
"command": [
|
|
"cp /bin/consul /bin/consul-inject/consul\n\nECS_IPV4=$(curl -s $ECS_CONTAINER_METADATA_URI_V4 | jq -r '.Networks[0].IPv4Addresses[0]')\n\n\ncat << EOF > /consul/agent-defaults.hcl\naddresses = {\n dns = \"127.0.0.1\"\n grpc = \"127.0.0.1\"\n http = \"127.0.0.1\"\n}\nadvertise_addr = \"$ECS_IPV4\"\nadvertise_reconnect_timeout = \"15m\"\nclient_addr = \"0.0.0.0\"\ndatacenter = \"dc1\"\nenable_central_service_config = true\nleave_on_terminate = true\nports {\n grpc = 8502\n}\nretry_join = [\n \"<Consul server location>",\n]\ntelemetry {\n disable_compat_1.9 = true\n}\n\nEOF\n\ncat << EOF > /consul/agent-extra.hcl\naddresses = {\n dns = \"0.0.0.0\"\n}\nlog_level = \"debug\"\n\nEOF\n\nexec consul agent \\\n -data-dir /consul/data \\\n -config-file /consul/agent-defaults.hcl \\\n -config-file /consul/agent-extra.hcl\n"
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
| Field name | Type | Description |
|
|
| ------------- | ------- | ------------------------------------------------------------------------------------------------------------------- |
|
|
| `name` | string | The container name, which should always be `consul-client`. |
|
|
| `image` | string | The Consul image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul`, to avoid rate limits. |
|
|
| `mountPoints` | list | Must be set as shown above. Volumes are mounted to share information with other containers for task setup. |
|
|
| `entrypoint` | list | Must be set to a plain shell so that the startup `command` works properly. |
|
|
| `command` | list | The startup command. See below for details. |
|
|
|
|
The following is the recommended `command` script for the Consul agent.
|
|
This is the same as the above `command` field, but is unescaped and has comments added.
|
|
|
|
```shell
|
|
# Copy the consul binary to a shared volume for mesh-init to use to generate Envoy configuration.
|
|
cp /bin/consul /bin/consul-inject/consul
|
|
|
|
# At runtime, determine the IP address assigned to this ECS Task.
|
|
ECS_IPV4=$(curl -s $ECS_CONTAINER_METADATA_URI_V4 | jq -r '.Networks[0].IPv4Addresses[0]')
|
|
|
|
# Write the Consul agent configuration file.
|
|
cat << EOF > /consul/agent-defaults.hcl
|
|
addresses = {
|
|
dns = "127.0.0.1"
|
|
grpc = "127.0.0.1"
|
|
http = "127.0.0.1"
|
|
}
|
|
advertise_addr = "$ECS_IPV4"
|
|
advertise_reconnect_timeout = "15m"
|
|
client_addr = "0.0.0.0"
|
|
datacenter = "dc1"
|
|
enable_central_service_config = true
|
|
leave_on_terminate = true
|
|
ports {
|
|
grpc = 8502
|
|
}
|
|
retry_join = ["<consul server location>"]
|
|
telemetry {
|
|
disable_compat_1.9 = true
|
|
}
|
|
|
|
EOF
|
|
|
|
# Start the consul agent.
|
|
exec consul agent \
|
|
-data-dir /consul/data \
|
|
-config-file /consul/agent-defaults.hcl
|
|
```
|
|
|
|
Set the following fields as shown above:
|
|
|
|
| Field name | Type | Description |
|
|
| -------------------- | ------- | ------------------------------------------------------------------------------------------------------------ |
|
|
| `addresses.*` | strings | Set the DNS, GRPC, and HTTP addresses to `127.0.0.1` to ensure these are not accessible outside of the task. |
|
|
| `advertise_addr` | string | Must be set to the task IP address so that other Consul agents know how to reach this agent. |
|
|
| `client_addr` | string | Must be set to an interface reacable by other Consul agents. |
|
|
| `datacenter` | string | Must be set to the Consul datacenter this task will join. |
|
|
| `leave_on_terminate` | boolean | Must be set to `true` so that the Consul agent leaves the cluster gracefully before exiting. |
|
|
| `retry_join` | string | Must be set to your Consul server location(s) so this agent can join the Consul cluster. |
|
|
|
|
-> **NOTE**: Use `exec` to start the Consul agent, so that the Consul agent runs as PID 1. This ensures
|
|
the Consul agent directly receives signals from ECS, which is important for graceful shutdown of the Consul agent.
|
|
|
|
Refer to the [Consul Agent documentation](/docs/agent/options#configuration_files) for a complete reference of Consul agent
|
|
configuration options.
|
|
|
|
## `mesh-init` container
|
|
|
|
The `mesh-init` container runs at task startup to setup this instance for Consul service mesh.
|
|
It registers the service and proxy for this task with Consul and writes Envoy bootstrap
|
|
configuration to a shared volume.
|
|
|
|
<CodeBlockConfig highlight="18-41">
|
|
|
|
```json
|
|
{
|
|
"containerDefinitions": [
|
|
{
|
|
"name": "example-client-app",
|
|
"image": "docker.io/org/my_task:v0.0.1",
|
|
...
|
|
},
|
|
{
|
|
"name": "sidecar-proxy",
|
|
"image": "envoyproxy/envoy-alpine:<ENVOY_VERSION>",
|
|
...
|
|
},
|
|
{
|
|
"name": "consul-client"
|
|
"image": "public.ecr.aws/hashicorp/consul:<CONSUL_VERSION>",
|
|
...
|
|
},
|
|
{
|
|
"name": "consul-ecs-mesh-init",
|
|
"image": "public.ecr.aws/hashicorp/consul-ecs:<CONSUL_ECS_VERSION>",
|
|
"command": ["mesh-init"],
|
|
"essential": false,
|
|
"environment": [
|
|
{
|
|
"name": "CONSUL_ECS_CONFIG_JSON",
|
|
"value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}"
|
|
}
|
|
],
|
|
"mountPoints": [
|
|
{
|
|
"readOnly": false,
|
|
"containerPath": "/consul",
|
|
"sourceVolume": "consul_data"
|
|
},
|
|
{
|
|
"readOnly": true,
|
|
"containerPath": "/bin/consul-inject",
|
|
"sourceVolume": "consul_binary"
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
| Field name | Type | Description |
|
|
| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------- |
|
|
| `name` | string | The container name should be `consul-ecs-mesh-init`. |
|
|
| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. |
|
|
| `mountPoints` | list | Must be set as show above, so the `consul` and `consul-ecs` binaries can be shared among containers for task setup. |
|
|
| `command` | list | Set the `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. |
|
|
| `environment` | list | This must include the `CONSUL_ECS_CONFIG_JSON` variable. See below for details. |
|
|
|
|
Configuration is passed to the `consul-ecs mesh-init` command in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable.
|
|
Here is the sample config from above, expanded to be readable:
|
|
|
|
```json
|
|
{
|
|
"bootstrapDir": "/consul",
|
|
"healthSyncContainers": [],
|
|
"proxy": {
|
|
"upstreams": [
|
|
{
|
|
"destinationName": "example-server-app",
|
|
"localBindPort": 1234
|
|
}
|
|
]
|
|
},
|
|
"service": {
|
|
"checks": [],
|
|
"meta": {},
|
|
"name": "example-client-app",
|
|
"port": 9090,
|
|
"tags": []
|
|
}
|
|
}
|
|
```
|
|
|
|
| Field name | Type | Description |
|
|
| ----------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------ |
|
|
| `bootstrapDir` | string | This is the path of a shared volume the is mounted to other containers, where `mesh-init` will write out Envoy configuration. |
|
|
| `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. |
|
|
| `service.name` | string | The name used to register this service into the Consul service catalog. |
|
|
| `service.port` | number | The port your application listens on. Set to `0` if your application does not listen on any port. |
|
|
| `service.checks` | list | Consul [checks](/docs/discovery/checks) to include, to have Consul run health checks against your application. |
|
|
|
|
See the [`consul-ecs JSON Schema`](https://github.com/hashicorp/consul-ecs/blob/main/config/schema.json) for a complete reference of fields.
|
|
|
|
## `consul-ecs-health-sync` container
|
|
|
|
Optionally, Consul ECS can sync health checks for this task into Consul checks.
|
|
This allows you to configure a health check for your application in one place, and
|
|
see a consistent health status in both ECS and Consul.
|
|
|
|
For example, the following defines an ECS health check command that runs `curl localhost:9090/health`:
|
|
|
|
<CodeBlockConfig highlight="6-11">
|
|
|
|
```json
|
|
{
|
|
"containerDefinitions": [
|
|
{
|
|
"name": "example-client-app",
|
|
"image": "docker.io/org/my_task:v0.0.1",
|
|
"healthCheck": {
|
|
"retries": 3,
|
|
"command": ["CMD-SHELL", "curl localhost:9090/health"],
|
|
"timeout": 5,
|
|
"interval": 30
|
|
},
|
|
...
|
|
},
|
|
...
|
|
]
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
First, tell Consul ECS which containers need their health status synced into Consul. To do this,
|
|
add the container name(s) to the `healthSyncContainers` list of the `CONSUL_ECS_CONFIG_JSON` variable:
|
|
|
|
<CodeBlockConfig highlight="3-3">
|
|
|
|
```json
|
|
{
|
|
"bootstrapDir": "/consul",
|
|
"healthSyncContainers": ["example-client-app"],
|
|
...
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
Next, pass the new configuration to the `consul-ecs-mesh-init` container. You should compact and escape the JSON
|
|
configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON` environment variable:
|
|
|
|
<CodeBlockConfig highlight="7-10">
|
|
|
|
```json
|
|
{
|
|
"containerDefinitions": [
|
|
{
|
|
"name": "consul-ecs-mesh-init",
|
|
"image": "public.ecr.aws/hashicorp/consul-ecs:<VERSION>",
|
|
"environment": [
|
|
{
|
|
"name": "CONSUL_ECS_CONFIG_JSON",
|
|
"value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[\"example-client-app\"],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}"
|
|
}
|
|
],
|
|
...
|
|
},
|
|
...
|
|
]
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
Finally, include the `consul-ecs-health-sync` container in the `containerDefinitions` list.
|
|
The exact same configuration for the `CONSUL_ECS_CONFIG_JSON` environment variable is used
|
|
for the health sync container.
|
|
|
|
<CodeBlockConfig highlight="23-40">
|
|
|
|
```json
|
|
{
|
|
"containerDefinitions": [
|
|
{
|
|
"name": "example-client-app",
|
|
"image": "docker.io/org/my_task:v0.0.1",
|
|
...
|
|
},
|
|
{
|
|
"name": "sidecar-proxy",
|
|
"image": "envoyproxy/envoy-alpine:<ENVOY_VERSION>",
|
|
...
|
|
},
|
|
{
|
|
"name": "consul-client"
|
|
"image": "public.ecr.aws/hashicorp/consul:<CONSUL_VERSION>",
|
|
...
|
|
},
|
|
{
|
|
"name": "consul-ecs-mesh-init",
|
|
"image": "public.ecr.aws/hashicorp/consul-ecs:<CONSUL_ECS_VERSION>",
|
|
...
|
|
},
|
|
{
|
|
"name": "consul-ecs-health-sync",
|
|
"image": "public.ecr.aws/hashicorp/consul-ecs:<CONSUL_ECS_VERSION>",
|
|
"command": ["health-sync"],
|
|
"essential": false,
|
|
"dependsOn": [
|
|
{
|
|
"containerName": "consul-ecs-mesh-init",
|
|
"condition": "SUCCESS"
|
|
}
|
|
],
|
|
"environment": [
|
|
{
|
|
"name": "CONSUL_ECS_CONFIG_JSON",
|
|
"value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[\"example-client-app\"],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}"
|
|
}
|
|
]
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
</CodeBlockConfig>
|
|
|
|
| Field name | Type | Description |
|
|
| ------------- | ------ | ----------------------------------------------------------------------------------------------------------------- |
|
|
| `name` | string | The container name, which must be `consul-ecs-health-sync`. |
|
|
| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. |
|
|
| `command` | list | Must be set to `["health-sync"]` to run the `consul-ecs health-sync` command. |
|
|
| `dependsOn` | list | Must be set as shown above to ensure the `health-sync` container starts after service registration has completed. |
|
|
| `environment` | list | Must include the `CONSUL_ECS_CONFIG_JSON` variable to pass configuration to the `consul-ecs health-sync` command. |
|
|
|
|
# Next Steps
|
|
|
|
* Create the task definition using the [AWS Console](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html) or the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/ecs/register-task-definition.html), or another method of your choice.
|
|
* Create an [ECS Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) to start tasks using the task definition.
|