luxolus
/
open-consul
2
0
Fork 0
Code Issues 1 Pull Requests Packages Releases Wiki Activity

docs: Improving ECS installation docs

This commit is contained in:
Paul Glass 2022-01-21 14:11:11 -06:00
parent 69de9c4d2e
commit 2d5b7b03aa
2 changed files with 65 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
website/content/docs/ecs/install.mdx
View File

@ -1,32 +1,29 @@
---
layout: docs
page_title: Installing Consul on AWS ECS using Terraform
page_title: Installing Consul on AWS ECS using Terraform
description: >-
Install Consul Service Mesh on AWS ECS with Terraform (Elastic Container Service).
---
# Installation with Terraform
This topic describes how to use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh.
This topic describes how to use the [`mesh-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh. If you do not use Terraform, see the [Manual Installation](/docs/ecs/manual-installation) page to install Consul on ECS without Terraform.
This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples.
## Prerequisites
* This pages assumes you are familiar with Terraform. If you are new to Terraform, see the [Terraform documentation](https://www.terraform.io/docs) to
learn about infrastructure as code and how to get started with Terraform.
* This page assumes you are familiar with AWS ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details.
* This page does not show how to create all necessary AWS resources, such as a VPC or the ECS Cluster.
For complete runnable examples, see the links in the [Getting Started](/docs/ecs#getting-started) section.
* You should have some familiarity with using Terraform. Refer to the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform.
* You should also be familiar with AWS ECS before following these instructions. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for details.
## Using the Mesh Task Module
To run an application in ECS with Consul service mesh, you must create an ECS task definition which includes your application container(s)
To run an application in ECS with Consul service mesh, you must create an ECS task definition, which includes your application container(s)
and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container.
The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers.
The following example shows a Terraform configuration file that defines a task definition with an application container called `example-client-app`.
The following example shows a Terraform configuration file that creates a task definition with an application container called `example-client-app`.
```hcl
module "my_task" {
@ -52,7 +49,7 @@ module "my_task" {
}
]
port = "9090"
port = 9090
retry_join = ["<address of the Consul server>"]
}
```
@ -63,9 +60,9 @@ however there are some important inputs worth highlighting:
| Input Variable | Type | Description |
| ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `source` and `version` | string | This specifies the source location and version of the `mesh-task` module. |
| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. |
| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. |
| `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. |
| `port` | number | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. |
| `port` | integer | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. |
| `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. |
## ECS Service
@ -131,7 +128,7 @@ module "web" {
| Input Variable | Type | Description |
| ----------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `destinationName` | string | The name of the upstream service, as it is registered in the Consul service catalog. |
| `localBindPort` | number | Requests to this port will be forwarded by the proxy to the upstream service. This must be an unused port, but does not need to match the upstream service port. |
| `localBindPort` | integer | Requests to this port will be forwarded by the proxy to the upstream service. This must be an unused port, but does not need to match the upstream service port. |
If you have multiple upstream services they each need to be listed here.

85
website/content/docs/ecs/manual-installation.mdx
View File

@ -7,7 +7,7 @@ description: >-
# Manual Installation
This topic describes how to manually create the ECS task definition for use without Terraform from the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs). Refer to the [Consul ECS Terraform module](/docs/ecs/install) documentation for an alternative method for installing Consul on ECS.
The following instructions describe how to manually create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) without Terraform. Refer to the [Consul ECS Terraform module](/docs/ecs/install) documentation for an alternative method for installing Consul on ECS.
This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples.
@ -17,17 +17,22 @@ You should have some familiarity with AWS ECS. See [What is Amazon Elastic Conta
## Task Definition
You must define a task definition which includes the following containers:
You must create a task definition, which includes the following containers:
* Your application container
* An Envoy sidecar-proxy container
* A Consul client container
* The `consul-ecs-mesh-init` container for service mesh setup
* A `consul-ecs-mesh-init` container for service mesh setup
* Optionally, a `consul-ecs-health-sync` container to sync ECS health checks into Consul
## Top-level fields
Your task definition must include the following top-level fields:
Your task definition must include the following top-level fields.
The `volumes` list contains two [bind mounts](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bind-mounts.html),
named `consul_data` and `consul_binary`. Bind mounts are directories on the host which can be mounted into one or more containers
in order to share files among containers. For Consul on ECS, certain binaries and configuration are shared among containers
during task startup.
```json
{
@ -55,9 +60,14 @@ Your task definition must include the following top-level fields:
## Application container
First, include your application container in the `containerDefinitions` list
in the task definition. Ensure that the `dependsOn` field is set as shown below
so that your application container starts in the correct order (see [task
startup](/docs/ecs/architecture#task-startup) for more information):
in the task definition.
Ensure that the `containerName` and `condition` fields in the `dependsOn` list
are specified as described in the following example. These are container dependencies,
which must be used to enforce a specific [startup order](/docs/ecs/architecture#task-startup).
By using the following settings, your application container will start after `consul-ecs-mesh-init`
has completed task setup and after `sidecar-proxy` is ready to proxy traffic between
this task and the service mesh.
```json
{
@ -87,7 +97,7 @@ startup](/docs/ecs/architecture#task-startup) for more information):
| `name` | string | The name of your application container. |
| `image` | string | The container image used to run your application. |
| `essential` | boolean | Must be `true` to ensure your application container ties into the health of the task. |
| `dependsOn` | list | Must be set as show above. Container dependencies ensure your application container starts after service mesh setup is complete. |
| `dependsOn` | list | Must be set as shown above. Container dependencies ensure your application container starts after service mesh setup is complete. |
See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html) documentation for a complete reference.
@ -95,6 +105,11 @@ See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/devel
The `sidecar-proxy` container runs [Envoy proxy](/docs/connect/proxies/envoy) for Consul Connect. In most cases, the container should contain the following parameters and values.
The `mountPoints` list must be set as shown in the following example. This will mount the shared `consul_data` volume into the
`sidecar-proxy` container at the path `/consul`. This volume is where the `consul-ecs-mesh-init` container copies the `envoy-bootstrap.json`
file and the `consul-ecs` binary, which are required to start Envoy. The `dependsOn` list must also be defined as follows to ensure the
`sidecar-proxy` container starts after `consul-ecs-mesh-init` has successfully written these files to the shared volume.
<CodeBlockConfig highlight="8-40">
```json
@ -144,7 +159,7 @@ The `sidecar-proxy` container runs [Envoy proxy](/docs/connect/proxies/envoy) fo
</CodeBlockConfig>
The following table describes the available configuration settings.
The following table describes the necessary configuration settings.
| Field name | Type | Description |
| ------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- |
@ -152,9 +167,9 @@ The following table describes the available configuration settings.
| `image` | string | The Envoy image. This must be a [supported version of Envoy](/docs/connect/proxies/envoy#supported-versions). |
| `dependsOn` | list | Must be set as shown above to ensure Envoy starts after the `consul-ecs-mesh-init` has written the `envoy-bootstrap.json` config file for Envoy. |
| `healthCheck` | list | Must be set as shown above to monitor the health of Envoy's primary listener port, which ties into container dependencies and startup ordering. |
| `mountPoints` | list | Must be set as shown above to access the files shared in the `consul` directory, like the Envoy bootstrap configuration file and the `consul-ecs` binary. |
| `mountPoints` | list | Must be set as shown above to access the files shared in the `/consul` directory, like the Envoy bootstrap configuration file and the `consul-ecs` binary. |
| `ulimits` | list | The `nofile` ulimit must be raised to a sufficiently high value so that Envoy does not fail to open sockets. |
| `entrypoint` | list | Must be set to the custom Envoy entrypoint to facilitate graceful shutdown. |
| `entrypoint` | list | Must be set to the custom Envoy entrypoint, `consul-ecs envoy-entrypoint`, to facilitate graceful shutdown. |
| `command` | list | The startup command. This passes the bootstrap configuration to Envoy. |
-> **NOTE**: Envoy and Consul must be compatible versions. See the [supported versions of Envoy](/docs/connect/proxies/envoy#supported-versions) in the Consul documentation.
@ -215,7 +230,7 @@ The following is the recommended `command` script for the Consul agent.
This is the same as the above `command` field, but is unescaped and has comments added.
```shell
# Copy the consul binary to a shared volume for mesh-init to use to generate Envoy configuration.
# Copy the consul binary to a shared volume for `consul-ecs-mesh-init` to use to generate Envoy configuration.
cp /bin/consul /bin/consul-inject/consul
# At runtime, determine the IP address assigned to this ECS Task.
@ -267,9 +282,9 @@ the Consul agent directly receives signals from ECS, which is important for grac
Refer to the [Consul Agent documentation](/docs/agent/options#configuration_files) for a complete reference of Consul agent
configuration options.
## `mesh-init` container
## `consul-ecs-mesh-init` container
The `mesh-init` container runs at task startup to setup this instance for Consul service mesh.
The `consul-ecs-mesh-init` container runs at task startup to setup this instance for Consul service mesh.
It registers the service and proxy for this task with Consul and writes Envoy bootstrap
configuration to a shared volume.
@ -329,10 +344,16 @@ configuration to a shared volume.
| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. |
| `mountPoints` | list | Must be set as show above, so the `consul` and `consul-ecs` binaries can be shared among containers for task setup. |
| `command` | list | Set the `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. |
| `environment` | list | This must include the `CONSUL_ECS_CONFIG_JSON` variable. See below for details. |
| `environment` | list | This must include the [`CONSUL_ECS_CONFIG_JSON`](/docs/ecs/manual-installation#consul_ecs_config_json) variable. See below for details. |
Configuration is passed to the `consul-ecs mesh-init` command in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable.
Here is the sample config from above, expanded to be readable:
### `CONSUL_ECS_CONFIG_JSON`
Configuration is passed to the `consul-ecs` binary in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable.
The following is an example of the configuration that might be used for a service named `example-client-app` with one upstream
service name `example-server-app`. The `proxy` and `service` blocks include information used by `consul-ecs-mesh-init` to perform
[service registration](/docs/discovery/services) with Consul during task startup. The same configuration format is used for
the `consul-ecs-health-sync` container.
```json
{
@ -356,13 +377,14 @@ Here is the sample config from above, expanded to be readable:
}
```
| Field name | Type | Description |
| ----------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------ |
| `bootstrapDir` | string | This is the path of a shared volume the is mounted to other containers, where `mesh-init` will write out Envoy configuration. |
| `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. |
| `service.name` | string | The name used to register this service into the Consul service catalog. |
| `service.port` | number | The port your application listens on. Set to `0` if your application does not listen on any port. |
| `service.checks` | list | Specifies the Consul health [checks](/docs/discovery/checks) to run against your application. |
| Field name | Type | Description |
| ---------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `bootstrapDir` | string | This is the path of a shared volume the is mounted to other containers, where `consul-ecs-mesh-init` will write out Envoy configuration. |
| `healthSyncContainers` | list | Used for [health status syncing](/docs/ecs/manual-installation#consul-ecs-health-sync-container) from ECS to Consul. See below for details. |
| `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. The `destinationName` and `localBindPort` fields are required. |
| `service.name` | string | The name used to register this service into the Consul service catalog. |
| `service.port` | integer | The port your application listens on. Set to `0` if your application does not listen on any port. |
| `service.checks` | list | Consul [checks](/docs/discovery/checks) to include so that Consul can run health checks against your application. |
See the [`consul-ecs JSON Schema`](https://github.com/hashicorp/consul-ecs/blob/main/config/schema.json) for a complete reference of fields.
@ -397,8 +419,10 @@ For example, the following defines an ECS health check command that runs `curl l
</CodeBlockConfig>
First, tell Consul ECS which containers need their health status synced into Consul. To do this,
add the container name(s) to the `healthSyncContainers` list of the `CONSUL_ECS_CONFIG_JSON` variable:
First, define which containers need their health status synced into Consul. To do this,
add the container name(s) to the `healthSyncContainers` list of the `CONSUL_ECS_CONFIG_JSON` variable,
as shown in the following example. This configuration must be passed to both the `consul-ecs-mesh-init`
and `consul-ecs-health-sync` containers.
<CodeBlockConfig highlight="3-3">
@ -412,8 +436,9 @@ add the container name(s) to the `healthSyncContainers` list of the `CONSUL_ECS_
</CodeBlockConfig>
Next, pass the new configuration to the `consul-ecs-mesh-init` container. You should compact and escape the JSON
configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON` environment variable:
Next, set the `CONSUL_ECS_CONFIG_JSON` variable for the `consul-ecs-mesh-init` container.
The following example shows how the `CONSUL_ECS_CONFIG_JSON` variable should be formatted.
The JSON configuration is compacted down to a single line and escaped.
<CodeBlockConfig highlight="7-10">
@ -439,8 +464,8 @@ configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON` envir
</CodeBlockConfig>
Finally, include the `consul-ecs-health-sync` container in the `containerDefinitions` list.
The exact same configuration for the `CONSUL_ECS_CONFIG_JSON` environment variable is used
for the health sync container.
Pass the same value for `CONSUL_ECS_CONFIG_JSON` for both the `consul-ecs-health-sync`
and `consul-ecs-mesh-init` containers.
<CodeBlockConfig highlight="23-40">