From 29bd32c9ce10bbc8d468e34011ecfcfdb4502164 Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Mon, 13 Dec 2021 17:02:35 -0600 Subject: [PATCH 01/13] docs: Merge ECS Architecture into overview --- website/content/docs/ecs/architecture.mdx | 96 ------------------- .../content/docs/ecs/get-started/install.mdx | 2 +- .../get-started/production-installation.mdx | 2 +- website/content/docs/ecs/index.mdx | 91 ++++++++++++++++++ website/data/docs-nav-data.json | 4 - 5 files changed, 93 insertions(+), 102 deletions(-) delete mode 100644 website/content/docs/ecs/architecture.mdx diff --git a/website/content/docs/ecs/architecture.mdx b/website/content/docs/ecs/architecture.mdx deleted file mode 100644 index b6e262832..000000000 --- a/website/content/docs/ecs/architecture.mdx +++ /dev/null @@ -1,96 +0,0 @@ ---- -layout: docs -page_title: Architecture - AWS ECS -description: >- - Architecture of Consul Service Mesh on AWS ECS (Elastic Container Service). ---- - -# Architecture - -The following diagram shows the main components of the Consul architecture when deployed to an ECS cluster: - -![Consul on ECS Architecture](/img/consul-ecs-arch.png) - -1. **Consul servers:** Production-ready Consul server cluster -1. **Application tasks:** Runs user application containers along with two helper containers: - 1. **Consul client:** The Consul client container runs Consul. The Consul client communicates - with the Consul server and configures the Envoy proxy sidecar. This communication - is called _control plane_ communication. - 1. **Sidecar proxy:** The sidecar proxy container runs [Envoy](https://envoyproxy.io/). All requests - to and from the application container(s) run through the sidecar proxy. This communication - is called _data plane_ communication. -1. **ACL Controller:** Automatically provisions Consul ACL tokens for Consul clients and service mesh services - in an ECS Cluster. - -For more information about how Consul works in general, see Consul's [Architecture Overview](/docs/architecture). - -In addition to the long-running Consul client and sidecar proxy containers, the `mesh-init` container runs -at startup and sets up initial configuration for Consul and Envoy. - -### Task Startup - -This diagram shows the timeline of a task starting up and all its containers: - -Task Startup Timeline - -- **T0:** ECS starts the task. The `consul-client` and `mesh-init` containers start: - - `consul-client` uses the `retry-join` option to join the Consul cluster - - `mesh-init` registers the service for the current task and its sidecar proxy with - Consul. It runs `consul connect envoy -bootstrap` to generate Envoy’s - bootstrap JSON file and write it to a shared volume. `mesh-init` exits after completing these operations. - -- **T1:** The following containers start: - - The `sidecar-proxy` container starts and runs Envoy by executing `envoy -c `. - - If applicable, the `health-sync` container syncs health checks from ECS to Consul (see [ECS Health Check Syncing](#ecs-health-check-syncing)). -- **T2:** The `sidecar-proxy` container is marked as healthy by ECS. It uses a health check that detects if its public listener port is open. At this time, your application containers are started since all Consul machinery is ready to service requests. The only running containers are `consul-client`, `sidecar-proxy`, and your application container(s). - -### Task Shutdown - -This diagram shows an example timeline of a task shutting down: - -Task Shutdown Timeline - -- **T0**: ECS sends a TERM signal to all containers. Each container reacts to the TERM signal: - - `consul-client` begins to gracefully leave the Consul cluster. - - `health-sync` stops syncing health status from ECS into Consul checks. - - `sidecar-proxy` ignores the TERM signal and continues running until the `user-app` container exits. This allows the application container to continue making outgoing requests through the proxy to the mesh. - - `user-app` exits if it is not configured to ignore the TERM signal. The `user-app` container will continue running if it is configured to ignore the TERM signal. -- **T1**: - - `health-sync` updates its Consul checks to critical status and exits. This ensures this service instance is marked unhealthy. - - `sidecar-proxy` notices the `user-app` container has stopped and exits. -- **T2**: `consul-client` finishes gracefully leaving the Consul datacenter and exits. -- **T3**: - - ECS notices all containers have exited, and will soon change the Task status to `STOPPED` - - Updates about this task have reached the rest of the Consul cluster, so downstream proxies have been updated to stopped sending traffic to this task. -- **T4**: At this point task shutdown should be complete. Otherwise, ECS will send a KILL signal to any containers still running. The KILL signal cannot be ignored and will forcefully stop containers. This will interrupt in-progress operations and possibly cause errors. - -### Automatic ACL Token Provisioning - -Consul ACL tokens secure communication between agents and services. -The following containers in a task require an ACL token: - -- `consul-client`: The Consul client uses a token to authorize itself with Consul servers. - All `consul-client` containers share the same token. -- `mesh-init`: The `mesh-init` container uses a token to register the service with Consul. - This token is unique for the Consul service, and is shared by instances of the service. - -The ACL controller automatically creates ACL tokens for mesh-enabled tasks in an ECS cluster. -The `acl-controller` Terraform module creates the ACL controller task. The controller creates the -ACL token used by `consul-client` containers at startup and then watches for tasks in the cluster. It checks tags -to determine if the task is mesh-enabled. If so, it creates the service ACL token for the task, if the -token does not yet exist. - -The ACL controller stores all ACL tokens in AWS Secrets Manager, and tasks are configured to pull these -tokens from AWS Secrets Manager when they start. - -### ECS Health Check Syncing - -If the following conditions apply, ECS health checks automatically sync with Consul health checks for all application containers: - -* marked as `essential` -* have ECS `healthChecks` -* are not configured with native Consul health checks - -The `mesh-init` container creates a TTL health check for -every container that fits these criteria and the `health-sync` container ensures -that the ECS and Consul health checks remain in sync. diff --git a/website/content/docs/ecs/get-started/install.mdx b/website/content/docs/ecs/get-started/install.mdx index b6d523063..9bca98248 100644 --- a/website/content/docs/ecs/get-started/install.mdx +++ b/website/content/docs/ecs/get-started/install.mdx @@ -182,5 +182,5 @@ python manage.py runserver "127.0.0.1:8080" - Configure a secure [Production Installation](/docs/ecs/get-started/production-installation). - Now that your applications are running in the service mesh, read about other [Service Mesh features](/docs/connect). -- View the [Architecture](/docs/ecs/architecture) documentation to understand +- View the [Architecture](/docs/ecs#architecture) documentation to understand what's going on under the hood. diff --git a/website/content/docs/ecs/get-started/production-installation.mdx b/website/content/docs/ecs/get-started/production-installation.mdx index 8e0c8199a..2f9bb0551 100644 --- a/website/content/docs/ecs/get-started/production-installation.mdx +++ b/website/content/docs/ecs/get-started/production-installation.mdx @@ -19,7 +19,7 @@ A secure Consul cluster should include the following: ## Deploy ACL Controller Before deploying your service, you will need to deploy the [ACL controller](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller) so that it can provision the necessary tokens -for tasks on the service mesh. To learn more about the ACL Controller, please see [Automatic ACL Token Provisioning](/docs/ecs/architecture#automatic-acl-token-provisioning). +for tasks on the service mesh. To learn more about the ACL Controller, please see [Automatic ACL Token Provisioning](/docs/ecs#automatic-acl-token-provisioning). To deploy the controller, you will first need store an ACL token with `acl:write` privileges and a CA certificate for the Consul server in AWS Secrets Manager. diff --git a/website/content/docs/ecs/index.mdx b/website/content/docs/ecs/index.mdx index 68fa655bb..27a44bef5 100644 --- a/website/content/docs/ecs/index.mdx +++ b/website/content/docs/ecs/index.mdx @@ -28,3 +28,94 @@ There are several ways to get started with Consul with ECS. * The [Consul with Dev Server on EC2](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-ec2) example installation deploys a sample application in ECS using the EC2 launch type. See the [Requirements](/docs/ecs/get-started/requirements) and the full [Install Guide](/docs/ecs/get-started/install) when you're ready to install Consul on an existing ECS cluster and add existing tasks to the service mesh. + + +## Architecture + +The following diagram shows the main components of the Consul architecture when deployed to an ECS cluster: + +![Consul on ECS Architecture](/img/consul-ecs-arch.png) + +1. **Consul servers:** Production-ready Consul server cluster +1. **Application tasks:** Runs user application containers along with two helper containers: + 1. **Consul client:** The Consul client container runs Consul. The Consul client communicates + with the Consul server and configures the Envoy proxy sidecar. This communication + is called _control plane_ communication. + 1. **Sidecar proxy:** The sidecar proxy container runs [Envoy](https://envoyproxy.io/). All requests + to and from the application container(s) run through the sidecar proxy. This communication + is called _data plane_ communication. +1. **ACL Controller:** Automatically provisions Consul ACL tokens for Consul clients and service mesh services + in an ECS Cluster. + +For more information about how Consul works in general, see Consul's [Architecture Overview](/docs/architecture). + +In addition to the long-running Consul client and sidecar proxy containers, the `mesh-init` container runs +at startup and sets up initial configuration for Consul and Envoy. + +### Task Startup + +This diagram shows the timeline of a task starting up and all its containers: + +Task Startup Timeline + +- **T0:** ECS starts the task. The `consul-client` and `mesh-init` containers start: + - `consul-client` uses the `retry-join` option to join the Consul cluster + - `mesh-init` registers the service for the current task and its sidecar proxy with + Consul. It runs `consul connect envoy -bootstrap` to generate Envoy’s + bootstrap JSON file and write it to a shared volume. `mesh-init` exits after completing these operations. + +- **T1:** The following containers start: + - The `sidecar-proxy` container starts and runs Envoy by executing `envoy -c `. + - If applicable, the `health-sync` container syncs health checks from ECS to Consul (see [ECS Health Check Syncing](#ecs-health-check-syncing)). +- **T2:** The `sidecar-proxy` container is marked as healthy by ECS. It uses a health check that detects if its public listener port is open. At this time, your application containers are started since all Consul machinery is ready to service requests. The only running containers are `consul-client`, `sidecar-proxy`, and your application container(s). + +### Task Shutdown + +This diagram shows an example timeline of a task shutting down: + +Task Shutdown Timeline + +- **T0**: ECS sends a TERM signal to all containers. Each container reacts to the TERM signal: + - `consul-client` begins to gracefully leave the Consul cluster. + - `health-sync` stops syncing health status from ECS into Consul checks. + - `sidecar-proxy` ignores the TERM signal and continues running until the `user-app` container exits. This allows the application container to continue making outgoing requests through the proxy to the mesh. + - `user-app` exits if it is not configured to ignore the TERM signal. The `user-app` container will continue running if it is configured to ignore the TERM signal. +- **T1**: + - `health-sync` updates its Consul checks to critical status and exits. This ensures this service instance is marked unhealthy. + - `sidecar-proxy` notices the `user-app` container has stopped and exits. +- **T2**: `consul-client` finishes gracefully leaving the Consul datacenter and exits. +- **T3**: + - ECS notices all containers have exited, and will soon change the Task status to `STOPPED` + - Updates about this task have reached the rest of the Consul cluster, so downstream proxies have been updated to stopped sending traffic to this task. +- **T4**: At this point task shutdown should be complete. Otherwise, ECS will send a KILL signal to any containers still running. The KILL signal cannot be ignored and will forcefully stop containers. This will interrupt in-progress operations and possibly cause errors. + +### Automatic ACL Token Provisioning + +Consul ACL tokens secure communication between agents and services. +The following containers in a task require an ACL token: + +- `consul-client`: The Consul client uses a token to authorize itself with Consul servers. + All `consul-client` containers share the same token. +- `mesh-init`: The `mesh-init` container uses a token to register the service with Consul. + This token is unique for the Consul service, and is shared by instances of the service. + +The ACL controller automatically creates ACL tokens for mesh-enabled tasks in an ECS cluster. +The `acl-controller` Terraform module creates the ACL controller task. The controller creates the +ACL token used by `consul-client` containers at startup and then watches for tasks in the cluster. It checks tags +to determine if the task is mesh-enabled. If so, it creates the service ACL token for the task, if the +token does not yet exist. + +The ACL controller stores all ACL tokens in AWS Secrets Manager, and tasks are configured to pull these +tokens from AWS Secrets Manager when they start. + +### ECS Health Check Syncing + +If the following conditions apply, ECS health checks automatically sync with Consul health checks for all application containers: + +* marked as `essential` +* have ECS `healthChecks` +* are not configured with native Consul health checks + +The `mesh-init` container creates a TTL health check for +every container that fits these criteria and the `health-sync` container ensures +that the ECS and Consul health checks remain in sync. diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index c58c64918..58acc8c97 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -628,10 +628,6 @@ } ] }, - { - "title": "Architecture", - "path": "ecs/architecture" - }, { "title": "Consul Enterprise", "path": "ecs/enterprise" From fbbe71415adff903e04ea2a40c34e60709601f83 Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Mon, 13 Dec 2021 17:25:28 -0600 Subject: [PATCH 02/13] docs: Flatten ECS "Getting Started" navigation --- website/content/docs/ecs/enterprise.mdx | 4 +-- website/content/docs/ecs/index.mdx | 2 +- .../docs/ecs/{get-started => }/install.mdx | 2 +- .../migrate-existing-tasks.mdx | 2 +- .../production-installation.mdx | 4 +-- .../ecs/{get-started => }/requirements.mdx | 0 website/data/docs-nav-data.json | 33 ++++++++----------- 7 files changed, 21 insertions(+), 26 deletions(-) rename website/content/docs/ecs/{get-started => }/install.mdx (98%) rename website/content/docs/ecs/{get-started => }/migrate-existing-tasks.mdx (96%) rename website/content/docs/ecs/{get-started => }/production-installation.mdx (95%) rename website/content/docs/ecs/{get-started => }/requirements.mdx (100%) diff --git a/website/content/docs/ecs/enterprise.mdx b/website/content/docs/ecs/enterprise.mdx index e2e15a029..b181b5e13 100644 --- a/website/content/docs/ecs/enterprise.mdx +++ b/website/content/docs/ecs/enterprise.mdx @@ -27,10 +27,10 @@ module "my_task" { ## Licensing -~> **Warning:** Consul Enterprise is currently only fully supported when [ACLs are enabled](/docs/ecs/get-started/production-installation#deploy-acl-controller). +~> **Warning:** Consul Enterprise is currently only fully supported when [ACLs are enabled](/docs/ecs/production-installation#deploy-acl-controller). Consul Enterprise [requires a license](/docs/enterprise/license/overview). If running -Consul on ECS with [ACLs enabled](/docs/ecs/get-started/production-installation#deploy-acl-controller), the license +Consul on ECS with [ACLs enabled](/docs/ecs/production-installation#deploy-acl-controller), the license will be automatically pulled down from Consul servers. Currently there is no capability for specifying the license when ACLs are disabled so if you wish to diff --git a/website/content/docs/ecs/index.mdx b/website/content/docs/ecs/index.mdx index 27a44bef5..27019c3fb 100644 --- a/website/content/docs/ecs/index.mdx +++ b/website/content/docs/ecs/index.mdx @@ -27,7 +27,7 @@ There are several ways to get started with Consul with ECS. * The [Consul with Dev Server on Fargate](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-fargate) example installation deploys a sample application in ECS using the Fargate launch type. * The [Consul with Dev Server on EC2](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-ec2) example installation deploys a sample application in ECS using the EC2 launch type. -See the [Requirements](/docs/ecs/get-started/requirements) and the full [Install Guide](/docs/ecs/get-started/install) when you're ready to install Consul on an existing ECS cluster and add existing tasks to the service mesh. +See the [Requirements](/docs/ecs/requirements) and the full [Install Guide](/docs/ecs/install) when you're ready to install Consul on an existing ECS cluster and add existing tasks to the service mesh. ## Architecture diff --git a/website/content/docs/ecs/get-started/install.mdx b/website/content/docs/ecs/install.mdx similarity index 98% rename from website/content/docs/ecs/get-started/install.mdx rename to website/content/docs/ecs/install.mdx index 9bca98248..bbd8a48d3 100644 --- a/website/content/docs/ecs/get-started/install.mdx +++ b/website/content/docs/ecs/install.mdx @@ -179,7 +179,7 @@ python manage.py runserver "127.0.0.1:8080" ## Next Steps -- Configure a secure [Production Installation](/docs/ecs/get-started/production-installation). +- Configure a secure [Production Installation](/docs/ecs/production-installation). - Now that your applications are running in the service mesh, read about other [Service Mesh features](/docs/connect). - View the [Architecture](/docs/ecs#architecture) documentation to understand diff --git a/website/content/docs/ecs/get-started/migrate-existing-tasks.mdx b/website/content/docs/ecs/migrate-existing-tasks.mdx similarity index 96% rename from website/content/docs/ecs/get-started/migrate-existing-tasks.mdx rename to website/content/docs/ecs/migrate-existing-tasks.mdx index aedca0391..0d6a06923 100644 --- a/website/content/docs/ecs/get-started/migrate-existing-tasks.mdx +++ b/website/content/docs/ecs/migrate-existing-tasks.mdx @@ -111,5 +111,5 @@ resource. Now that your task(s) are migrated to the `mesh-task` module, -- Start at the [ECS Service section](/docs/ecs/get-started/install#ecs-service) of the Installation Guide to continue installing Consul on ECS. +- Start at the [ECS Service section](/docs/ecs/install#ecs-service) of the Installation Guide to continue installing Consul on ECS. - Refer to the [`mesh-task` reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs) for all available inputs to your mesh tasks. diff --git a/website/content/docs/ecs/get-started/production-installation.mdx b/website/content/docs/ecs/production-installation.mdx similarity index 95% rename from website/content/docs/ecs/get-started/production-installation.mdx rename to website/content/docs/ecs/production-installation.mdx index 2f9bb0551..19375ed9f 100644 --- a/website/content/docs/ecs/get-started/production-installation.mdx +++ b/website/content/docs/ecs/production-installation.mdx @@ -68,7 +68,7 @@ deploying this controller. ## Deploy Services Once the ACL controller is up and running, you will be able to deploy services on the mesh using the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task). -Start with the basic configuration for the [Task Module](/docs/ecs/get-started/install#task-module) and specify additional settings to make the configuration production-ready. +Start with the basic configuration for the [Task Module](/docs/ecs/install#task-module) and specify additional settings to make the configuration production-ready. First, you will need to create an AWS Secrets Manager secret for the gossip encryption key that the Consul clients should use. @@ -104,5 +104,5 @@ module "my_task" { } ``` -Now you can deploy your services! Follow the rest of the steps in the [Installation instructions](/docs/ecs/get-started/install#task-module) +Now you can deploy your services! Follow the rest of the steps in the [Installation instructions](/docs/ecs/install#task-module) to deploy and connect your services. diff --git a/website/content/docs/ecs/get-started/requirements.mdx b/website/content/docs/ecs/requirements.mdx similarity index 100% rename from website/content/docs/ecs/get-started/requirements.mdx rename to website/content/docs/ecs/requirements.mdx diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 58acc8c97..84a6a960b 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -608,25 +608,20 @@ "path": "ecs" }, { - "title": "Get Started", - "routes": [ - { - "title": "Requirements", - "path": "ecs/get-started/requirements" - }, - { - "title": "Installation", - "path": "ecs/get-started/install" - }, - { - "title": "Production Installation", - "path": "ecs/get-started/production-installation" - }, - { - "title": "Migrate Existing Tasks", - "path": "ecs/get-started/migrate-existing-tasks" - } - ] + "title": "Requirements", + "path": "ecs/requirements" + }, + { + "title": "Installation", + "path": "ecs/install" + }, + { + "title": "Production Installation", + "path": "ecs/production-installation" + }, + { + "title": "Migrate Existing Tasks", + "path": "ecs/migrate-existing-tasks" }, { "title": "Consul Enterprise", From 446355ff69fbf939b80bfb3d129b4cee57570188 Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Tue, 18 Jan 2022 16:17:49 -0600 Subject: [PATCH 03/13] docs: Unmerge ECS architecture from overview --- website/content/docs/ecs/architecture.mdx | 97 ++++++++++++++++ website/content/docs/ecs/index.mdx | 107 +++--------------- website/content/docs/ecs/install.mdx | 2 +- .../docs/ecs/production-installation.mdx | 2 +- website/data/docs-nav-data.json | 4 + 5 files changed, 116 insertions(+), 96 deletions(-) create mode 100644 website/content/docs/ecs/architecture.mdx diff --git a/website/content/docs/ecs/architecture.mdx b/website/content/docs/ecs/architecture.mdx new file mode 100644 index 000000000..766402fcc --- /dev/null +++ b/website/content/docs/ecs/architecture.mdx @@ -0,0 +1,97 @@ +--- +layout: docs +page_title: Architecture - AWS ECS +description: >- + Architecture of Consul Service Mesh on AWS ECS (Elastic Container Service). +--- + +# Architecture + +The following diagram shows the main components of the Consul architecture when deployed to an ECS cluster: + +![Consul on ECS Architecture](/img/consul-ecs-arch.png) + +1. **Consul servers:** Production-ready Consul server cluster +1. **Application tasks:** Runs user application containers along with two helper containers: + 1. **Consul client:** The Consul client container runs Consul. The Consul client communicates + with the Consul server and configures the Envoy proxy sidecar. This communication + is called _control plane_ communication. + 1. **Sidecar proxy:** The sidecar proxy container runs [Envoy](https://envoyproxy.io/). All requests + to and from the application container(s) run through the sidecar proxy. This communication + is called _data plane_ communication. +1. **Mesh Init:** Each task runs a short-lived container, called `mesh-init`, which sets up initial configuration + for Consul and Envoy. +1. **Health Syncing:** Optionally, an additional `health-sync` container can be included in a task to sync health statuses + from ECS into Consul. +1. **ACL Controller:** Automatically provisions Consul ACL tokens for Consul clients and service mesh services + in an ECS Cluster. + +For more information about how Consul works in general, see Consul's [Architecture Overview](/docs/architecture). + +## Task Startup + +This diagram shows the timeline of a task starting up and all its containers: + +Task Startup Timeline + +- **T0:** ECS starts the task. The `consul-client` and `mesh-init` containers start: + - `consul-client` uses the `retry-join` option to join the Consul cluster + - `mesh-init` registers the service for the current task and its sidecar proxy with + Consul. It runs `consul connect envoy -bootstrap` to generate Envoy’s + bootstrap JSON file and write it to a shared volume. `mesh-init` exits after completing these operations. + +- **T1:** The following containers start: + - The `sidecar-proxy` container starts and runs Envoy by executing `envoy -c `. + - If applicable, the `health-sync` container syncs health checks from ECS to Consul (see [ECS Health Check Syncing](#ecs-health-check-syncing)). +- **T2:** The `sidecar-proxy` container is marked as healthy by ECS. It uses a health check that detects if its public listener port is open. At this time, your application containers are started since all Consul machinery is ready to service requests. The only running containers are `consul-client`, `sidecar-proxy`, and your application container(s). + +## Task Shutdown + +This diagram shows an example timeline of a task shutting down: + +Task Shutdown Timeline + +- **T0**: ECS sends a TERM signal to all containers. Each container reacts to the TERM signal: + - `consul-client` begins to gracefully leave the Consul cluster. + - `health-sync` stops syncing health status from ECS into Consul checks. + - `sidecar-proxy` ignores the TERM signal and continues running until the `user-app` container exits. This allows the application container to continue making outgoing requests through the proxy to the mesh. + - `user-app` exits if it is not configured to ignore the TERM signal. The `user-app` container will continue running if it is configured to ignore the TERM signal. +- **T1**: + - `health-sync` updates its Consul checks to critical status and exits. This ensures this service instance is marked unhealthy. + - `sidecar-proxy` notices the `user-app` container has stopped and exits. +- **T2**: `consul-client` finishes gracefully leaving the Consul datacenter and exits. +- **T3**: + - ECS notices all containers have exited, and will soon change the Task status to `STOPPED` + - Updates about this task have reached the rest of the Consul cluster, so downstream proxies have been updated to stopped sending traffic to this task. +- **T4**: At this point task shutdown should be complete. Otherwise, ECS will send a KILL signal to any containers still running. The KILL signal cannot be ignored and will forcefully stop containers. This will interrupt in-progress operations and possibly cause errors. + +## Automatic ACL Token Provisioning + +Consul ACL tokens secure communication between agents and services. +The following containers in a task require an ACL token: + +- `consul-client`: The Consul client uses a token to authorize itself with Consul servers. + All `consul-client` containers share the same token. +- `mesh-init`: The `mesh-init` container uses a token to register the service with Consul. + This token is unique for the Consul service, and is shared by instances of the service. + +The ACL controller automatically creates ACL tokens for mesh-enabled tasks in an ECS cluster. +The `acl-controller` Terraform module creates the ACL controller task. The controller creates the +ACL token used by `consul-client` containers at startup and then watches for tasks in the cluster. It checks tags +to determine if the task is mesh-enabled. If so, it creates the service ACL token for the task, if the +token does not yet exist. + +The ACL controller stores all ACL tokens in AWS Secrets Manager, and tasks are configured to pull these +tokens from AWS Secrets Manager when they start. + +## ECS Health Check Syncing + +If the following conditions apply, ECS health checks automatically sync with Consul health checks for all application containers: + +* marked as `essential` +* have ECS `healthChecks` +* are not configured with native Consul health checks + +The `mesh-init` container creates a TTL health check for +every container that fits these criteria and the `health-sync` container ensures +that the ECS and Consul health checks remain in sync. diff --git a/website/content/docs/ecs/index.mdx b/website/content/docs/ecs/index.mdx index 27019c3fb..1078d65c7 100644 --- a/website/content/docs/ecs/index.mdx +++ b/website/content/docs/ecs/index.mdx @@ -8,9 +8,8 @@ description: >- # AWS ECS -Consul can be deployed on [AWS ECS](https://aws.amazon.com/ecs/) (Elastic Container Service) using our official Terraform modules. - -![Consul on ECS Architecture](/img/consul-ecs-arch.png) +Consul service mesh applications can be deployed on [AWS Elastic Container Service](https://aws.amazon.com/ecs/) (ECS) +using our official Terraform modules. ## Service Mesh @@ -18,6 +17,17 @@ Using Consul on AWS ECS enables you to add your ECS tasks to the service mesh an take advantage of features such as zero-trust-security, intentions, observability, traffic policy, and more. +## Architecture + +![Consul on ECS Architecture](/img/consul-ecs-arch.png) + +Consul on ECS follows an [architecture](/docs/internals/architecture) similar to other platforms, but each ECS task is a +Consul node. An ECS task runs the user application container(s), as well as a Consul client container for control plane +communication and an [Envoy](https://envoyproxy.io/) sidecar proxy container to faciliate data plane communication for +[Consul Connect](/docs/connect). + +For a detailed architecture overview, see the [Architecture](/docs/ecs/architecture) page. + ## Getting Started There are several ways to get started with Consul with ECS. @@ -28,94 +38,3 @@ There are several ways to get started with Consul with ECS. * The [Consul with Dev Server on EC2](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-ec2) example installation deploys a sample application in ECS using the EC2 launch type. See the [Requirements](/docs/ecs/requirements) and the full [Install Guide](/docs/ecs/install) when you're ready to install Consul on an existing ECS cluster and add existing tasks to the service mesh. - - -## Architecture - -The following diagram shows the main components of the Consul architecture when deployed to an ECS cluster: - -![Consul on ECS Architecture](/img/consul-ecs-arch.png) - -1. **Consul servers:** Production-ready Consul server cluster -1. **Application tasks:** Runs user application containers along with two helper containers: - 1. **Consul client:** The Consul client container runs Consul. The Consul client communicates - with the Consul server and configures the Envoy proxy sidecar. This communication - is called _control plane_ communication. - 1. **Sidecar proxy:** The sidecar proxy container runs [Envoy](https://envoyproxy.io/). All requests - to and from the application container(s) run through the sidecar proxy. This communication - is called _data plane_ communication. -1. **ACL Controller:** Automatically provisions Consul ACL tokens for Consul clients and service mesh services - in an ECS Cluster. - -For more information about how Consul works in general, see Consul's [Architecture Overview](/docs/architecture). - -In addition to the long-running Consul client and sidecar proxy containers, the `mesh-init` container runs -at startup and sets up initial configuration for Consul and Envoy. - -### Task Startup - -This diagram shows the timeline of a task starting up and all its containers: - -Task Startup Timeline - -- **T0:** ECS starts the task. The `consul-client` and `mesh-init` containers start: - - `consul-client` uses the `retry-join` option to join the Consul cluster - - `mesh-init` registers the service for the current task and its sidecar proxy with - Consul. It runs `consul connect envoy -bootstrap` to generate Envoy’s - bootstrap JSON file and write it to a shared volume. `mesh-init` exits after completing these operations. - -- **T1:** The following containers start: - - The `sidecar-proxy` container starts and runs Envoy by executing `envoy -c `. - - If applicable, the `health-sync` container syncs health checks from ECS to Consul (see [ECS Health Check Syncing](#ecs-health-check-syncing)). -- **T2:** The `sidecar-proxy` container is marked as healthy by ECS. It uses a health check that detects if its public listener port is open. At this time, your application containers are started since all Consul machinery is ready to service requests. The only running containers are `consul-client`, `sidecar-proxy`, and your application container(s). - -### Task Shutdown - -This diagram shows an example timeline of a task shutting down: - -Task Shutdown Timeline - -- **T0**: ECS sends a TERM signal to all containers. Each container reacts to the TERM signal: - - `consul-client` begins to gracefully leave the Consul cluster. - - `health-sync` stops syncing health status from ECS into Consul checks. - - `sidecar-proxy` ignores the TERM signal and continues running until the `user-app` container exits. This allows the application container to continue making outgoing requests through the proxy to the mesh. - - `user-app` exits if it is not configured to ignore the TERM signal. The `user-app` container will continue running if it is configured to ignore the TERM signal. -- **T1**: - - `health-sync` updates its Consul checks to critical status and exits. This ensures this service instance is marked unhealthy. - - `sidecar-proxy` notices the `user-app` container has stopped and exits. -- **T2**: `consul-client` finishes gracefully leaving the Consul datacenter and exits. -- **T3**: - - ECS notices all containers have exited, and will soon change the Task status to `STOPPED` - - Updates about this task have reached the rest of the Consul cluster, so downstream proxies have been updated to stopped sending traffic to this task. -- **T4**: At this point task shutdown should be complete. Otherwise, ECS will send a KILL signal to any containers still running. The KILL signal cannot be ignored and will forcefully stop containers. This will interrupt in-progress operations and possibly cause errors. - -### Automatic ACL Token Provisioning - -Consul ACL tokens secure communication between agents and services. -The following containers in a task require an ACL token: - -- `consul-client`: The Consul client uses a token to authorize itself with Consul servers. - All `consul-client` containers share the same token. -- `mesh-init`: The `mesh-init` container uses a token to register the service with Consul. - This token is unique for the Consul service, and is shared by instances of the service. - -The ACL controller automatically creates ACL tokens for mesh-enabled tasks in an ECS cluster. -The `acl-controller` Terraform module creates the ACL controller task. The controller creates the -ACL token used by `consul-client` containers at startup and then watches for tasks in the cluster. It checks tags -to determine if the task is mesh-enabled. If so, it creates the service ACL token for the task, if the -token does not yet exist. - -The ACL controller stores all ACL tokens in AWS Secrets Manager, and tasks are configured to pull these -tokens from AWS Secrets Manager when they start. - -### ECS Health Check Syncing - -If the following conditions apply, ECS health checks automatically sync with Consul health checks for all application containers: - -* marked as `essential` -* have ECS `healthChecks` -* are not configured with native Consul health checks - -The `mesh-init` container creates a TTL health check for -every container that fits these criteria and the `health-sync` container ensures -that the ECS and Consul health checks remain in sync. diff --git a/website/content/docs/ecs/install.mdx b/website/content/docs/ecs/install.mdx index bbd8a48d3..d98fb15bd 100644 --- a/website/content/docs/ecs/install.mdx +++ b/website/content/docs/ecs/install.mdx @@ -182,5 +182,5 @@ python manage.py runserver "127.0.0.1:8080" - Configure a secure [Production Installation](/docs/ecs/production-installation). - Now that your applications are running in the service mesh, read about other [Service Mesh features](/docs/connect). -- View the [Architecture](/docs/ecs#architecture) documentation to understand +- View the [Architecture](/docs/ecs/architecture) documentation to understand what's going on under the hood. diff --git a/website/content/docs/ecs/production-installation.mdx b/website/content/docs/ecs/production-installation.mdx index 19375ed9f..9c8c6c6e6 100644 --- a/website/content/docs/ecs/production-installation.mdx +++ b/website/content/docs/ecs/production-installation.mdx @@ -19,7 +19,7 @@ A secure Consul cluster should include the following: ## Deploy ACL Controller Before deploying your service, you will need to deploy the [ACL controller](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/acl-controller) so that it can provision the necessary tokens -for tasks on the service mesh. To learn more about the ACL Controller, please see [Automatic ACL Token Provisioning](/docs/ecs#automatic-acl-token-provisioning). +for tasks on the service mesh. To learn more about the ACL Controller, please see [Automatic ACL Token Provisioning](/docs/ecs/architecture#automatic-acl-token-provisioning). To deploy the controller, you will first need store an ACL token with `acl:write` privileges and a CA certificate for the Consul server in AWS Secrets Manager. diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 84a6a960b..028a950e0 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -623,6 +623,10 @@ "title": "Migrate Existing Tasks", "path": "ecs/migrate-existing-tasks" }, + { + "title": "Architecture", + "path": "ecs/architecture" + }, { "title": "Consul Enterprise", "path": "ecs/enterprise" From 5a98efcf56881cec3a160d19bf4e30d1061af20a Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Tue, 18 Jan 2022 14:52:40 -0600 Subject: [PATCH 04/13] docs: Manual installation of Consul on ECS --- .../content/docs/ecs/install-no-terraform.mdx | 492 ++++++++++++++++++ website/content/docs/ecs/install.mdx | 97 ++-- website/content/docs/ecs/requirements.mdx | 12 +- website/data/docs-nav-data.json | 6 +- 4 files changed, 565 insertions(+), 42 deletions(-) create mode 100644 website/content/docs/ecs/install-no-terraform.mdx diff --git a/website/content/docs/ecs/install-no-terraform.mdx b/website/content/docs/ecs/install-no-terraform.mdx new file mode 100644 index 000000000..5f6378eaf --- /dev/null +++ b/website/content/docs/ecs/install-no-terraform.mdx @@ -0,0 +1,492 @@ +--- +layout: docs +page_title: Manual Installation - AWS ECS +description: >- + Manually Install Consul Service Mesh on AWS ECS (Elastic Container Service). +--- + +# Manual Installation + +While the [Consul ECS Terraform module](/docs/ecs/install) is the easiest way to use Consul on ECS, +this page will describe how to directly create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) +for use without Terraform. + +## Pre-requisites + +* This page assumes you are familiar with AWS ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details. +* This page does not show how to create all necessary AWS resources, such as a VPC or the ECS Cluster. + For complete runnable examples, see the links in the [Getting Started](/docs/ecs#getting-started) section. + +# Task Definition + +You must define a Task Definition which includes the following containers: + +* Your application container +* An Envoy sidecar-proxy container +* A Consul Agent container +* The `consul-ecs-mesh-init` container for service mesh setup +* Optionally, a `consul-ecs-health-sync` container to sync ECS health checks into Consul + +## Top-level fields + +In your task definition, you'll need to define these important top-level fields: + +```json +{ + "family": "my-example-client-app", + "networkMode": "awsvpc", + "volumes": [ + { + "name": "consul_data", + }, + { + "name": "consul_binary", + } + ], + "containerDefinitions": [...] +} +``` + +| Field name | Type | Description | +| ------------- | ------ | --------------------------------------------------------------------------------------------------- | +| `family` | string | The task family name. This is used as the Consul service name, by default. | +| `networkMode` | string | Must be `awsvpc`, which is the only network mode supported by Consul on ECS. | +| `volumes` | list | These are volumes used to share information between containers, which is important for setup steps. | +| `containerDefinitions` | list | The list of containers to run in this task (see below). | + +## Application container + +First, include your application container in the `containerDefinitions` list +in the task definition: + +```json +{ + "containerDefinitions": [ + { + "name": "example-client-app", + "image": "docker.io/org/my_task:v0.0.1", + "essential": true, + "dependsOn": [ + { + "containerName": "consul-ecs-mesh-init", + "condition": "SUCCESS" + }, + { + "containerName": "sidecar-proxy", + "condition": "HEALTHY" + } + ] + } + ] +} +``` + +| Field name | Type | Description | +| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------ | +| `name` | string | The name of your application container. | +| `image` | string | The container image used to run your application. | +| `essential` | boolean | Set this to `true` ensures your application container ties into the health of the task. All tasks must have at least one essential container. | +| `dependsOn` | list | Container dependendencies are used to ensure the service mesh is ready before your application starts. | + +See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html) documentation for a complete reference. + +## `sidecar-proxy` container + +The sidecar proxy container runs [Envoy proxy](/docs/connect/proxies/envoy) for Consul Connect. + +```json +{ + "containerDefinitions": [ + { + "name": "example-client-app", + "image": "docker.io/org/my_task:v0.0.1", + ... + }, + { + "name": "sidecar-proxy", + "image": "envoyproxy/envoy-alpine:", + "essential": false, + "dependsOn": [ + { + "containerName": "consul-ecs-mesh-init", + "condition": "SUCCESS" + } + ], + "healthCheck": { + "retries": 3, + "command": ["nc", "-z", "127.0.0.1", "20000"], + "timeout": 5, + "interval": 30 + }, + "mountPoints": [ + { + "readOnly": true, + "containerPath": "/consul", + "sourceVolume": "consul_data" + } + ], + "ulimits": [ + { + "name": "nofile", + "softLimit": 1048576, + "hardLimit": 1048576 + } + ], + "command": ["envoy", "--config-path", "/consul/envoy-bootstrap.json"], + "entryPoint": ["/consul/consul-ecs", "envoy-entrypoint"], + } + ] +} +``` + +| Field name | Type | Description | +| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------ | +| `name` | string | The name of the container, which should always be `sidecar-proxy`. | +| `image` | string | The container image for Envoy. We recommend using `envoyproxy/envoy-alpine`. | +| `dependsOn` | list | Envoy must start after `consul-ecs-mesh-init`, which creates the bootstrap configuration file for Envoy. | +| `healthCheck` | list | A health check should be definied for Envoy's primary listener port. | +| `mountPoints` | list | The mounts the `/consul` data volume which contains the Envoy configuration file. | +| `ulimits` | list | Set the file descriptor limit, `nofile`, to a sufficiently high value so that Envoy does not fail to open sockets. | +| `entrypoint` | list | A custom entrypoint binary is used to help facilitate graceful shutdown. | +| `command` | list | The startup command. This passes the bootstrap configuration to Envoy. | + +-> **NOTE**: Envoy and Consul must be compatible versions. See the [supported versions of Envoy](https://www.consul.io/docs/connect/proxies/envoy#supported-versions) for each Consul release. + + +## `consul-client` container + +Each task must include a Consul agent container in order for the task to join your Consul cluster. + +```json +{ + "containerDefinitions": [ + { + "name": "example-client-app", + "image": "docker.io/org/my_task:v0.0.1", + ... + }, + { + "name": "sidecar-proxy", + "image": "envoyproxy/envoy-alpine:", + ... + } + { + "name": "consul-client" + "image": "public.ecr.aws/hashicorp/consul:", + "mountPoints": [ + { + "readOnly": false, + "containerPath": "/consul", + "sourceVolume": "consul_data" + }, + { + "containerPath": "/bin/consul-inject", + "sourceVolume": "consul_binary" + } + ], + "entryPoint": ["/bin/sh", "-ec"], + "command": [ + "cp /bin/consul /bin/consul-inject/consul\n\nECS_IPV4=$(curl -s $ECS_CONTAINER_METADATA_URI_V4 | jq -r '.Networks[0].IPv4Addresses[0]')\n\n\ncat << EOF > /consul/agent-defaults.hcl\naddresses = {\n dns = \"127.0.0.1\"\n grpc = \"127.0.0.1\"\n http = \"127.0.0.1\"\n}\nadvertise_addr = \"$ECS_IPV4\"\nadvertise_reconnect_timeout = \"15m\"\nclient_addr = \"0.0.0.0\"\ndatacenter = \"dc1\"\nenable_central_service_config = true\nleave_on_terminate = true\nports {\n grpc = 8502\n}\nretry_join = [\n \"",\n]\ntelemetry {\n disable_compat_1.9 = true\n}\n\nEOF\n\ncat << EOF > /consul/agent-extra.hcl\naddresses = {\n dns = \"0.0.0.0\"\n}\nlog_level = \"debug\"\n\nEOF\n\nexec consul agent \\\n -data-dir /consul/data \\\n -config-file /consul/agent-defaults.hcl \\\n -config-file /consul/agent-extra.hcl\n" + ] + } + ] +} +``` + +| Field name | Type | Description | +| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------ | +| `name` | string | The name of the Consul agent container, which should always be `consul-client`. | +| `image` | string | The container image for Consul. Use our public AWS registry, `public.ecr.aws/hashicorp/consul`, to avoid rate limits. | +| `mountPoints` | list | The mounts the `/consul` data volume which contains the Envoy configuration file. | +| `ulimits` | list | Set the file descriptor limit, `nofile`, to a sufficiently high value so that Envoy does not fail to open sockets. | +| `entrypoint` | list | A custom entrypoint binary is used to run Envoy, in order to facilitate graceful shutdown. | +| `command` | list | The startup command. See below for details. + +The following is the recommended `command` script for the Consul agent. +This is the same as the above `command` field, but is unescaped and has comments added. + +```shell +# Copy the consul binary to a shared volume for mesh-init to use to generate Envoy configuration. +cp /bin/consul /bin/consul-inject/consul + +# At runtime, determine the IP address assigned to this ECS Task. +ECS_IPV4=$(curl -s $ECS_CONTAINER_METADATA_URI_V4 | jq -r '.Networks[0].IPv4Addresses[0]') + +# Define the Consul agent configuration file. +cat << EOF > /consul/agent-defaults.hcl +addresses = { + dns = "127.0.0.1" + grpc = "127.0.0.1" + http = "127.0.0.1" +} +advertise_addr = "$ECS_IPV4" +advertise_reconnect_timeout = "15m" +client_addr = "0.0.0.0" +datacenter = "dc1" +enable_central_service_config = true +leave_on_terminate = true +ports { + grpc = 8502 +} +retry_join = [""] +telemetry { + disable_compat_1.9 = true +} + +EOF + +# Start the consul agent. +exec consul agent \ + -data-dir /consul/data \ + -config-file /consul/agent-defaults.hcl +``` + +You can tailor the configuration above for your use case, but it's important to set the following fields as show: + +| Field name | Type | Description | +| -------------------- | ------- | ------------------------------------------------------------------------------------------------------------ | +| `addresses.*` | strings | Set the DNS, GRPC, and HTTP addresses to `127.0.0.1` to ensure these are not accessible outside of the task. | +| `advertise_addr` | string | This must be set to the task IP address so that other Consul agents know how to reach this agent. | +| `client_addr` | string | This must bind to an interface reacable by other Consul agents. | +| `leave_on_terminate` | boolean | This ensures this Consul agent will leave the cluster gracefully before exiting. | +| `retry_join` | string | This must be set to your Consul server location(s) so this agent can join the Consul cluster. | + +-> **NOTE**: It is important to use `exec` to start the Consul agent, so that the Consul agent runs as PID 1. This ensures +the Consul agent directly receives signals from ECS, which is important for graceful shutdown of the Consul agent. + +## `mesh-init` container + +The mesh-init container runs at task startup to setup this instance for Consul service mesh. +It will register this service and proxy with Consul and write Envoy bootstrap configuration +to a shared volume. + +```json +{ + "containerDefinitions": [ + { + "name": "example-client-app", + "image": "docker.io/org/my_task:v0.0.1", + ... + }, + { + "name": "sidecar-proxy", + "image": "envoyproxy/envoy-alpine:", + ... + }, + { + "name": "consul-client" + "image": "public.ecr.aws/hashicorp/consul:", + ... + }, + { + "name": "consul-ecs-mesh-init", + "image": "public.ecr.aws/hashicorp/consul-ecs:", + "command": ["mesh-init"], + "essential": false, + "environment": [ + { + "name": "CONSUL_ECS_CONFIG_JSON", + "value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}" + } + ], + "mountPoints": [ + { + "readOnly": false, + "containerPath": "/consul", + "sourceVolume": "consul_data" + }, + { + "readOnly": true, + "containerPath": "/bin/consul-inject", + "sourceVolume": "consul_binary" + } + ] + } + ] +} +``` + +| Field name | Type | Description | +| ----------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name should be `consul-ecs-mesh-init`. | +| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | +| `mountPoints` | list | The mounts two volumes so that mesh-init can invoke the `consul` binary to write bootstrap Envoy configuration. | +| `command` | list | Set the `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. | +| `environment` | list | This must include the `CONSUL_ECS_CONFIG_JSON` variable. See below for details. | + +Additional configuration is passed to the `consul-ecs mesh-init` command in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable. +Here is the sample config from above, expanded to be readable: + +```json +{ + "bootstrapDir": "/consul", + "healthSyncContainers": [], + "proxy": { + "upstreams": [ + { + "destinationName": "example-server-app", + "localBindPort": 1234 + } + ] + }, + "service": { + "checks": [], + "meta": {}, + "name": "example-client-app", + "port": 9090, + "tags": [] + } +} +``` + +| Field name | Type | Description | +| ----------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------ | +| `bootstrapDir` | string | This is the path of a shared volume the is mounted to other containers, where `mesh-init` will write out Envoy configuration. | +| `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. | +| `service.name` | string | The name used to register this service into the Consul service catalog. | +| `service.port` | number | The port your application listens on. Set to `0` if your application does not listen on any port. | +| `service.checks` | list | Consul [checks](/docs/discovery/checks) to include, to have Consul run health checks against your application. | + +See the [`consul-ecs JSON Schema`](https://github.com/hashicorp/consul-ecs/blob/main/config/schema.json) for a complete reference of fields. + +## `consul-ecs-health-sync` container + +Optionally, Consul ECS can sync health checks for this task into Consul checks. +This allows you to configure a health check for your application in one place, and +see a consistent health status in both ECS and Consul. + +First, you'll need to add an ECS health check command to the container definition for your application. +This configures a health check command that runs `curl localhost:9090/health` to check the application health: + +```json +{ + "containerDefinitions": [ + { + "name": "example-client-app", + "image": "docker.io/org/my_task:v0.0.1", + "healthCheck": { + "retries": 3, + "command": ["CMD-SHELL", "curl localhost:9090/health"], + "timeout": 5, + "interval": 30 + }, + ... + }, + ... + ] +} +``` + +See the [ECS health check documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definition_healthcheck) for details on defining ECS health checks. + +Next, you must tell Consul ECS which containers will have their health status synced into Consul. To do this, +set the `healthSyncContainers` field of the `CONSUL_ECS_CONFIG_JSON` variable to include your application container name. + +Here is the expanded configuration: + +```json +{ + "bootstrapDir": "/consul", + "healthSyncContainers": ["example-client-app"], + "proxy": { + "upstreams": [ + { + "destinationName": "example-server-app", + "localBindPort": 1234 + } + ] + }, + "service": { + "checks": [], + "meta": {}, + "name": "example-client-app", + "port": 9090, + "tags": [] + } +} +``` + +Then, update the task definition so that the `consul-ecs-mesh-init` container uses new configuration. +You should compact and escape the JSON configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON` +environment variable: + +```json +{ + "containerDefinitions": [ + { + "name": "consul-ecs-mesh-init", + "image": "public.ecr.aws/hashicorp/consul-ecs:", + "environment": [ + { + "name": "CONSUL_ECS_CONFIG_JSON", + "value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[\"example-client-app\"],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}" + } + ], + ... + }, + ... + ] +} +``` + +Finally, include the `consul-ecs-health-sync` container in the `containerDefinitions` list. +Be sure to pass the same value as above for the `CONSUL_ECS_CONFIG_JSON` environment +variable: + +```json +{ + "containerDefinitions": [ + { + "name": "example-client-app", + "image": "docker.io/org/my_task:v0.0.1", + ... + }, + { + "name": "sidecar-proxy", + "image": "envoyproxy/envoy-alpine:", + ... + }, + { + "name": "consul-client" + "image": "public.ecr.aws/hashicorp/consul:", + ... + }, + { + "name": "consul-ecs-mesh-init", + "image": "public.ecr.aws/hashicorp/consul-ecs:", + ... + }, + { + "name": "consul-ecs-health-sync", + "image": "public.ecr.aws/hashicorp/consul-ecs:", + "command": ["health-sync"], + "essential": false, + "dependsOn": [ + { + "containerName": "consul-ecs-mesh-init", + "condition": "SUCCESS" + } + ], + "environment": [ + { + "name": "CONSUL_ECS_CONFIG_JSON", + "value": "{\"bootstrapDir\":\"/consul\",\"healthSyncContainers\":[\"example-client-app\"],\"proxy\":{\"upstreams\":[{\"destinationName\":\"example-server-app\",\"localBindPort\":1234}]},\"service\":{\"checks\":[],\"meta\":{},\"name\":\"example-client-app\",\"port\":9090,\"tags\":[]}}" + } + ] + } + ] +} +``` + +| Field name | Type | Description | +| ------------- | ------ | ----------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name which should be `consul-ecs-health-sync`. | +| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | +| `command` | list | Set to `["health-sync"]` to run the `consul-ecs health-sync` command. | +| `dependsOn` | list | The `health-sync` container should not start until `mesh-init` has finished service and proxy registration. | +| `environment` | list | Set the `CONSUL_ECS_CONFIG_JSON` variable to pass configuration to the `consul-ecs health-sync command. | + +# Next Steps + +* Create the task definition using the [AWS Console](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html) or the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/ecs/register-task-definition.html), or another method of your choice. +* Create an [ECS Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) to start tasks using the task definition. diff --git a/website/content/docs/ecs/install.mdx b/website/content/docs/ecs/install.mdx index d98fb15bd..3554043cd 100644 --- a/website/content/docs/ecs/install.mdx +++ b/website/content/docs/ecs/install.mdx @@ -1,28 +1,30 @@ --- layout: docs -page_title: Installation - AWS ECS +page_title: Installation with Terraform - AWS ECS description: >- - Install Consul Service Mesh on AWS ECS (Elastic Container Service). + Install Consul Service Mesh on AWS ECS with Terraform (Elastic Container Service). --- -# Installation +# Installation with Terraform -Installing Consul on ECS is a multi-part process: +This will describe how to use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh. -1. [**Task Module:**](#task-module) Define the [`mesh-task` Terraform module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) - to create a task definition with the necessary sidecar containers for your application to join the service mesh. -1. [**Routing:**](#routing) With your tasks as part of the mesh, you must specify their upstream - services and change the URLs the tasks are using so that they're making requests through the service mesh. -1. [**Bind Address:**](#bind-address) Now that all communication is flowing through the service mesh, - you should change the address your application is listening on to `127.0.0.1` - so that it only receives requests through the sidecar proxy. +## Pre-requisites --> **NOTE:** This page assumes you're familiar with ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details. +* This pages assumes you are familiar with Terraform. If you are new to Terraform, see the [Terraform documentation](https://www.terraform.io/docs) to + learn about infrastructure as code and how to get started with Terraform. +* This page assumes you are familiar with AWS ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details. +* This page does not show how to create all necessary AWS resources, such as a VPC or the ECS Cluster. + For complete runnable examples, see the links in the [Getting Started](/docs/ecs#getting-started) section. -## Task Module +## Using the Mesh Task Module -In order to add the necessary sidecar containers for your task to join the mesh, -you must use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task): +To run an application in ECS with Consul service mesh, you must create an ECS task definition which includes your application container(s) +and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container. + +The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers. + +Here is an example Terraform configuration file which defines a task definition with an application container called `example-client-app`. ```hcl module "my_task" { @@ -53,35 +55,52 @@ module "my_task" { } ``` -All possible inputs are documented on the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs), +All possible inputs are documented in the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs), however there are some important inputs worth highlighting: -- `family` is used as the [task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family) - but it's also used as the name of the service that gets registered in Consul. -- `container_definitions` accepts an array of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions). - This is where you include application containers. -- `port` is the port that your application listens on. This should be set to a - string, not an integer, i.e. `port = "9090"`, not `port = 9090`. -- `retry_join` is passed to the [`-retry-join`](/docs/agent/options#_retry_join) option for the Consul agent. This tells - the agent the location of your Consul servers so that it can join the Consul cluster. - --> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true` -in their [`network_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#network_configuration) block so that ECS can pull the Docker images. +| Input Variable | Type | Description | +| ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `source` and `version` | string | This specifies the source location and version of the `mesh-task` module. | +| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name, by default. | +| `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. | +| `port` | number | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | +| `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | ## ECS Service +[ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) are one of the most common +ways to start tasks using a task definition.. + To define an ECS Service, reference the mesh-task module's `task_definition_arn` output value in your `aws_ecs_service` resource: ```hcl resource "aws_ecs_service" "my_task" { - ... + name = "my_task_service" task_definition = module.my_task.task_definition_arn + launch_type = "FARGATE" + propagate_tags = "TASK_DEFINITION" + ... } ``` -After running `terraform apply`, you should see your tasks registered in -the Consul UI. +This is a partial configuration to highlight some important fields. +See the [`aws_ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) documentation for a complete reference. + +| Input Variable | Type | Description | +| ----------------- | ------- | ------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The name of the ECS service. This is required by AWS but is not used by Consul service mesh. | +| `task_definition` | string | The task definition used to start tasks. Set this to the task definition ARN returned by the `mesh-task` module. | +| `launch_type` | string | The launch type. Consul on ECS supports the `FARGATE` and `EC2` launch types. | +| `propagate_tags` | string | This must be set to `TASK_DEFINITION` so that tags added by `mesh-task` to the task definition are copied to tasks. | + +After defining the Terraform configuration for both the `mesh-task` and ECS service, +run `terraform apply` to create the ECS task definition and service resources. The ECS +service will soon start your application in a task. The task will automatically +register itself into the Consul service catalog. + +-> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true` +in their [`network_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#network_configuration) block so that ECS can pull the Docker images. ## Routing @@ -94,25 +113,23 @@ needs to call. You then must modify your application to make requests to the sid proxy on that port. For example, if your application `web` makes calls to another application called `backend`, then you would first configure the `mesh-task` module's upstream(s): -`backend`. ```hcl module "web" { family = "web" upstreams = [ { - destination_name = "backend" - local_bind_port = 8080 + destinationName = "backend" + localBindPort = 8080 } ] } ``` -- Set the `destination_name` to the name of the upstream service (in this case `backend`) -- Set `local_bind_port` to an unused port. This is the port that the sidecar proxy - will listen on. Any requests to this port will be forwarded over to the `destination_name`. - This does not have to be the port that `backend` is listening on because the service mesh - will handle routing the request to the right port. +| Input Variable | Type | Description | +| ----------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `destinationName` | string | The name of the upstream service, as it is registered in the Consul service catalog. | +| `localBindPort` | number | Requests to this port will be forwarded by the proxy to the upstream service. This must be an unused port, but does not need to match the upstream service port. | If you have multiple upstream services they each need to be listed here. @@ -127,8 +144,8 @@ module "web" { family = "web" upstreams = [ { - destination_name = "backend" - local_bind_port = 8080 + destinationName = "backend" + localBindPort = 8080 } ] container_definitions = [ diff --git a/website/content/docs/ecs/requirements.mdx b/website/content/docs/ecs/requirements.mdx index 1c4d0488d..c7d0b7407 100644 --- a/website/content/docs/ecs/requirements.mdx +++ b/website/content/docs/ecs/requirements.mdx @@ -9,8 +9,18 @@ description: >- The following requirements must be met in order to install Consul on ECS: -1. **Terraform:** The tasks that you want to add to the service mesh must first be modeled in Terraform. 1. **Launch Type:** Fargate and EC2 launch types are supported. 1. **Subnets:** ECS Tasks can run in private or public subnets. Tasks must have [network access](https://aws.amazon.com/premiumsupport/knowledge-center/ecs-pull-container-api-error-ecr/) to Amazon ECR or other public container registries to pull images. 1. **Consul Servers:** You can use your own Consul servers running on virtual machines or use [HashiCorp Cloud Platform Consul](https://www.hashicorp.com/cloud-platform) to host the servers for you. For development purposes or testing, you may use the `dev-server` [Terraform module](https://github.com/hashicorp/terraform-aws-consul-ecs/tree/main) that runs the Consul server as an ECS task. The `dev-server` does not support persistent storage. 1. **ACL Controller:** If you are running a secure Consul installation with ACLs enabled, configure the ACL controller. +1. **Sidecar containers:** Consul on ECS requires two sidecar containers to run in each ECS task: a + Consul agent container and a sidecar proxy container. These additional sidecar containers must + be included in the ECS task definition. The Consul ECS Terraform modules will include these + sidecar containers for you. Or if you do not use Terraform, you can construct the task + definition yourself by following our documentation. +1. **Routing:** With your application running in tasks as part of the mesh, you must specify the + upstream services that your application calls. You will also need to change the URLs your + application uses to ensure the application is making requests through the service mesh. +1. **Bind Address:** Once all communication is flowing through the service mesh, you should change + the address your application is listening on to `127.0.0.1` so that it only receives requests + through the sidecar proxy. diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 028a950e0..fdbacd0e9 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -612,7 +612,7 @@ "path": "ecs/requirements" }, { - "title": "Installation", + "title": "Installation with Terraform", "path": "ecs/install" }, { @@ -630,6 +630,10 @@ { "title": "Consul Enterprise", "path": "ecs/enterprise" + }, + { + "title": "Manual Installation", + "path": "ecs/install-no-terraform" } ] }, From d0230d8f605d34007bfe1f80e877919f9f4779ec Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Wed, 19 Jan 2022 15:18:15 -0600 Subject: [PATCH 05/13] docs: Apply suggestions for manual ECS installation Co-authored-by: Luke Kysow <1034429+lkysow@users.noreply.github.com> --- website/content/docs/ecs/install-no-terraform.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/content/docs/ecs/install-no-terraform.mdx b/website/content/docs/ecs/install-no-terraform.mdx index 5f6378eaf..4cdc9ba30 100644 --- a/website/content/docs/ecs/install-no-terraform.mdx +++ b/website/content/docs/ecs/install-no-terraform.mdx @@ -17,7 +17,7 @@ for use without Terraform. * This page does not show how to create all necessary AWS resources, such as a VPC or the ECS Cluster. For complete runnable examples, see the links in the [Getting Started](/docs/ecs#getting-started) section. -# Task Definition +## Task Definition You must define a Task Definition which includes the following containers: @@ -57,7 +57,7 @@ In your task definition, you'll need to define these important top-level fields: ## Application container First, include your application container in the `containerDefinitions` list -in the task definition: +in the task definition. Ensure that the `dependsOn` field is set as shown below so that your application container starts in the correct order (see [task startup](/docs/ecs/architecture#task-startup) for more information): ```json { @@ -155,7 +155,7 @@ The sidecar proxy container runs [Envoy proxy](/docs/connect/proxies/envoy) for ## `consul-client` container -Each task must include a Consul agent container in order for the task to join your Consul cluster. +Each task must include a Consul client container in order for the task to join your Consul cluster. ```json { @@ -197,7 +197,7 @@ Each task must include a Consul agent container in order for the task to join yo | ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------ | | `name` | string | The name of the Consul agent container, which should always be `consul-client`. | | `image` | string | The container image for Consul. Use our public AWS registry, `public.ecr.aws/hashicorp/consul`, to avoid rate limits. | -| `mountPoints` | list | The mounts the `/consul` data volume which contains the Envoy configuration file. | +| `mountPoints` | list | Mount the `/consul` data volume which contains the Envoy configuration file. | | `ulimits` | list | Set the file descriptor limit, `nofile`, to a sufficiently high value so that Envoy does not fail to open sockets. | | `entrypoint` | list | A custom entrypoint binary is used to run Envoy, in order to facilitate graceful shutdown. | | `command` | list | The startup command. See below for details. From 2682a99a7854a6071bfa67e60c98f93ac317ed07 Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Wed, 19 Jan 2022 17:55:55 -0600 Subject: [PATCH 06/13] docs: Improving ECS manual installation page --- .../content/docs/ecs/install-no-terraform.mdx | 179 ++++++++++-------- 1 file changed, 97 insertions(+), 82 deletions(-) diff --git a/website/content/docs/ecs/install-no-terraform.mdx b/website/content/docs/ecs/install-no-terraform.mdx index 4cdc9ba30..4a4f80e43 100644 --- a/website/content/docs/ecs/install-no-terraform.mdx +++ b/website/content/docs/ecs/install-no-terraform.mdx @@ -23,7 +23,7 @@ You must define a Task Definition which includes the following containers: * Your application container * An Envoy sidecar-proxy container -* A Consul Agent container +* A Consul client container * The `consul-ecs-mesh-init` container for service mesh setup * Optionally, a `consul-ecs-health-sync` container to sync ECS health checks into Consul @@ -47,17 +47,19 @@ In your task definition, you'll need to define these important top-level fields: } ``` -| Field name | Type | Description | -| ------------- | ------ | --------------------------------------------------------------------------------------------------- | -| `family` | string | The task family name. This is used as the Consul service name, by default. | -| `networkMode` | string | Must be `awsvpc`, which is the only network mode supported by Consul on ECS. | -| `volumes` | list | These are volumes used to share information between containers, which is important for setup steps. | -| `containerDefinitions` | list | The list of containers to run in this task (see below). | +| Field name | Type | Description | +| ---------------------- | ------ | ------------------------------------------------------------------------------------------------------------------ | +| `family` | string | The task family name. This is used as the Consul service name, by default. | +| `networkMode` | string | Must be `awsvpc`, which is the only network mode supported by Consul on ECS. | +| `volumes` | list | Must be defined as shown above. Volumes ares used to share configuration between containers for intial task setup. | +| `containerDefinitions` | list | The list of containers to run in this task (see below). | ## Application container First, include your application container in the `containerDefinitions` list -in the task definition. Ensure that the `dependsOn` field is set as shown below so that your application container starts in the correct order (see [task startup](/docs/ecs/architecture#task-startup) for more information): +in the task definition. Ensure that the `dependsOn` field is set as shown below +so that your application container starts in the correct order (see [task +startup](/docs/ecs/architecture#task-startup) for more information): ```json { @@ -75,18 +77,19 @@ in the task definition. Ensure that the `dependsOn` field is set as shown below "containerName": "sidecar-proxy", "condition": "HEALTHY" } - ] + ], + ... } ] } ``` -| Field name | Type | Description | -| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------ | -| `name` | string | The name of your application container. | -| `image` | string | The container image used to run your application. | -| `essential` | boolean | Set this to `true` ensures your application container ties into the health of the task. All tasks must have at least one essential container. | -| `dependsOn` | list | Container dependendencies are used to ensure the service mesh is ready before your application starts. | +| Field name | Type | Description | +| ----------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The name of your application container. | +| `image` | string | The container image used to run your application. | +| `essential` | boolean | Must be `true` to ensure your application container ties into the health of the task. | +| `dependsOn` | list | Must be set as show above. Container dependencies ensure your application container starts after service mesh setup is complete. | See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html) documentation for a complete reference. @@ -94,6 +97,8 @@ See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/devel The sidecar proxy container runs [Envoy proxy](/docs/connect/proxies/envoy) for Consul Connect. + + ```json { "containerDefinitions": [ @@ -139,24 +144,29 @@ The sidecar proxy container runs [Envoy proxy](/docs/connect/proxies/envoy) for } ``` -| Field name | Type | Description | -| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------ | -| `name` | string | The name of the container, which should always be `sidecar-proxy`. | -| `image` | string | The container image for Envoy. We recommend using `envoyproxy/envoy-alpine`. | -| `dependsOn` | list | Envoy must start after `consul-ecs-mesh-init`, which creates the bootstrap configuration file for Envoy. | -| `healthCheck` | list | A health check should be definied for Envoy's primary listener port. | -| `mountPoints` | list | The mounts the `/consul` data volume which contains the Envoy configuration file. | -| `ulimits` | list | Set the file descriptor limit, `nofile`, to a sufficiently high value so that Envoy does not fail to open sockets. | -| `entrypoint` | list | A custom entrypoint binary is used to help facilitate graceful shutdown. | -| `command` | list | The startup command. This passes the bootstrap configuration to Envoy. | + --> **NOTE**: Envoy and Consul must be compatible versions. See the [supported versions of Envoy](https://www.consul.io/docs/connect/proxies/envoy#supported-versions) for each Consul release. +In most cases, the `sidecar-proxy` container can be configured exactly as shown above. +| Field name | Type | Description | +| ------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name, which must be `sidecar-proxy`. | +| `image` | string | The Envoy image. This must be a [supported version of Envoy](/docs/connect/proxies/envoy#supported-versions). | +| `dependsOn` | list | Must be set as shown above to ensure Envoy starts after the `consul-ecs-mesh-init` has written the `envoy-bootstrap.json` config file for Envoy. | +| `healthCheck` | list | Must be set as shown above to monitor the health of Envoy's primary listener port, which ties into container dependencies and startup ordering. | +| `mountPoints` | list | Must be set as shown above to access the files shared in the `consul` directory, like the Envoy bootstrap configuration file and the `consul-ecs` binary. | +| `ulimits` | list | The `nofile` ulimit must be raised to a sufficiently high value so that Envoy does not fail to open sockets. | +| `entrypoint` | list | Must be set to the custom Envoy entrypoint to facilitate graceful shutdown. | +| `command` | list | The startup command. This passes the bootstrap configuration to Envoy. | + +-> **NOTE**: Envoy and Consul must be compatible versions. See the [supported versions of Envoy](/docs/connect/proxies/envoy#supported-versions) in the Consul documentation. ## `consul-client` container Each task must include a Consul client container in order for the task to join your Consul cluster. + + ```json { "containerDefinitions": [ @@ -193,14 +203,15 @@ Each task must include a Consul client container in order for the task to join y } ``` -| Field name | Type | Description | -| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------------ | -| `name` | string | The name of the Consul agent container, which should always be `consul-client`. | -| `image` | string | The container image for Consul. Use our public AWS registry, `public.ecr.aws/hashicorp/consul`, to avoid rate limits. | -| `mountPoints` | list | Mount the `/consul` data volume which contains the Envoy configuration file. | -| `ulimits` | list | Set the file descriptor limit, `nofile`, to a sufficiently high value so that Envoy does not fail to open sockets. | -| `entrypoint` | list | A custom entrypoint binary is used to run Envoy, in order to facilitate graceful shutdown. | -| `command` | list | The startup command. See below for details. + + +| Field name | Type | Description | +| ------------- | ------- | ------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name, which should always be `consul-client`. | +| `image` | string | The Consul image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul`, to avoid rate limits. | +| `mountPoints` | list | Must be set as shown above. Volumes are mounted to share information with other containers for task setup. | +| `entrypoint` | list | Must be set to a plain shell so that the startup `command` works properly. | +| `command` | list | The startup command. See below for details. | The following is the recommended `command` script for the Consul agent. This is the same as the above `command` field, but is unescaped and has comments added. @@ -212,7 +223,7 @@ cp /bin/consul /bin/consul-inject/consul # At runtime, determine the IP address assigned to this ECS Task. ECS_IPV4=$(curl -s $ECS_CONTAINER_METADATA_URI_V4 | jq -r '.Networks[0].IPv4Addresses[0]') -# Define the Consul agent configuration file. +# Write the Consul agent configuration file. cat << EOF > /consul/agent-defaults.hcl addresses = { dns = "127.0.0.1" @@ -241,24 +252,30 @@ exec consul agent \ -config-file /consul/agent-defaults.hcl ``` -You can tailor the configuration above for your use case, but it's important to set the following fields as show: +Set the following fields as shown above: | Field name | Type | Description | | -------------------- | ------- | ------------------------------------------------------------------------------------------------------------ | | `addresses.*` | strings | Set the DNS, GRPC, and HTTP addresses to `127.0.0.1` to ensure these are not accessible outside of the task. | -| `advertise_addr` | string | This must be set to the task IP address so that other Consul agents know how to reach this agent. | -| `client_addr` | string | This must bind to an interface reacable by other Consul agents. | -| `leave_on_terminate` | boolean | This ensures this Consul agent will leave the cluster gracefully before exiting. | -| `retry_join` | string | This must be set to your Consul server location(s) so this agent can join the Consul cluster. | +| `advertise_addr` | string | Must be set to the task IP address so that other Consul agents know how to reach this agent. | +| `client_addr` | string | Must be set to an interface reacable by other Consul agents. | +| `datacenter` | string | Must be set to the Consul datacenter this task will join. | +| `leave_on_terminate` | boolean | Must be set to `true` so that the Consul agent leaves the cluster gracefully before exiting. | +| `retry_join` | string | Must be set to your Consul server location(s) so this agent can join the Consul cluster. | --> **NOTE**: It is important to use `exec` to start the Consul agent, so that the Consul agent runs as PID 1. This ensures +-> **NOTE**: Use `exec` to start the Consul agent, so that the Consul agent runs as PID 1. This ensures the Consul agent directly receives signals from ECS, which is important for graceful shutdown of the Consul agent. +Refer to the [Consul Agent documentation](/docs/agent/options#configuration_files) for a complete reference of Consul agent +configuration options. + ## `mesh-init` container -The mesh-init container runs at task startup to setup this instance for Consul service mesh. -It will register this service and proxy with Consul and write Envoy bootstrap configuration -to a shared volume. +The `mesh-init` container runs at task startup to setup this instance for Consul service mesh. +It registers the service and proxy for this task with Consul and writes Envoy bootstrap +configuration to a shared volume. + + ```json { @@ -306,15 +323,17 @@ to a shared volume. } ``` -| Field name | Type | Description | -| ----------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | -| `name` | string | The container name should be `consul-ecs-mesh-init`. | -| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | -| `mountPoints` | list | The mounts two volumes so that mesh-init can invoke the `consul` binary to write bootstrap Envoy configuration. | -| `command` | list | Set the `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. | -| `environment` | list | This must include the `CONSUL_ECS_CONFIG_JSON` variable. See below for details. | + -Additional configuration is passed to the `consul-ecs mesh-init` command in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable. +| Field name | Type | Description | +| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name should be `consul-ecs-mesh-init`. | +| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | +| `mountPoints` | list | Must be set as show above, so the `consul` and `consul-ecs` binaries can be shared among containers for task setup. | +| `command` | list | Set the `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. | +| `environment` | list | This must include the `CONSUL_ECS_CONFIG_JSON` variable. See below for details. | + +Configuration is passed to the `consul-ecs mesh-init` command in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable. Here is the sample config from above, expanded to be readable: ```json @@ -355,8 +374,9 @@ Optionally, Consul ECS can sync health checks for this task into Consul checks. This allows you to configure a health check for your application in one place, and see a consistent health status in both ECS and Consul. -First, you'll need to add an ECS health check command to the container definition for your application. -This configures a health check command that runs `curl localhost:9090/health` to check the application health: +For example, the following defines an ECS health check command that runs `curl localhost:9090/health`: + + ```json { @@ -377,38 +397,27 @@ This configures a health check command that runs `curl localhost:9090/health` to } ``` -See the [ECS health check documentation](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definition_healthcheck) for details on defining ECS health checks. + -Next, you must tell Consul ECS which containers will have their health status synced into Consul. To do this, -set the `healthSyncContainers` field of the `CONSUL_ECS_CONFIG_JSON` variable to include your application container name. +First, tell Consul ECS which containers need their health status synced into Consul. To do this, +add the container name(s) to the `healthSyncContainers` list of the `CONSUL_ECS_CONFIG_JSON` variable: -Here is the expanded configuration: + ```json { "bootstrapDir": "/consul", "healthSyncContainers": ["example-client-app"], - "proxy": { - "upstreams": [ - { - "destinationName": "example-server-app", - "localBindPort": 1234 - } - ] - }, - "service": { - "checks": [], - "meta": {}, - "name": "example-client-app", - "port": 9090, - "tags": [] - } + ... } ``` -Then, update the task definition so that the `consul-ecs-mesh-init` container uses new configuration. -You should compact and escape the JSON configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON` -environment variable: + + +Next, pass the new configuration to the `consul-ecs-mesh-init` container. You should compact and escape the JSON +configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON` environment variable: + + ```json { @@ -429,9 +438,13 @@ environment variable: } ``` + + Finally, include the `consul-ecs-health-sync` container in the `containerDefinitions` list. -Be sure to pass the same value as above for the `CONSUL_ECS_CONFIG_JSON` environment -variable: +The exact same configuration for the `CONSUL_ECS_CONFIG_JSON` environment variable is used +for the health sync container. + + ```json { @@ -478,13 +491,15 @@ variable: } ``` + + | Field name | Type | Description | | ------------- | ------ | ----------------------------------------------------------------------------------------------------------------- | -| `name` | string | The container name which should be `consul-ecs-health-sync`. | +| `name` | string | The container name, which must be `consul-ecs-health-sync`. | | `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | -| `command` | list | Set to `["health-sync"]` to run the `consul-ecs health-sync` command. | -| `dependsOn` | list | The `health-sync` container should not start until `mesh-init` has finished service and proxy registration. | -| `environment` | list | Set the `CONSUL_ECS_CONFIG_JSON` variable to pass configuration to the `consul-ecs health-sync command. | +| `command` | list | Must be set to `["health-sync"]` to run the `consul-ecs health-sync` command. | +| `dependsOn` | list | Must be set as shown above to ensure the `health-sync` container starts after service registration has completed. | +| `environment` | list | Must include the `CONSUL_ECS_CONFIG_JSON` variable to pass configuration to the `consul-ecs health-sync` command. | # Next Steps From 35b37345bea0a1a773d31503aabbeaf8faa53edf Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Wed, 19 Jan 2022 18:07:44 -0600 Subject: [PATCH 07/13] docs: Correct ECS filename -> manual-installation.mdx --- .../{install-no-terraform.mdx => manual-installation.mdx} | 0 website/content/docs/ecs/requirements.mdx | 6 +++--- website/data/docs-nav-data.json | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) rename website/content/docs/ecs/{install-no-terraform.mdx => manual-installation.mdx} (100%) diff --git a/website/content/docs/ecs/install-no-terraform.mdx b/website/content/docs/ecs/manual-installation.mdx similarity index 100% rename from website/content/docs/ecs/install-no-terraform.mdx rename to website/content/docs/ecs/manual-installation.mdx diff --git a/website/content/docs/ecs/requirements.mdx b/website/content/docs/ecs/requirements.mdx index c7d0b7407..73570f16b 100644 --- a/website/content/docs/ecs/requirements.mdx +++ b/website/content/docs/ecs/requirements.mdx @@ -15,9 +15,9 @@ The following requirements must be met in order to install Consul on ECS: 1. **ACL Controller:** If you are running a secure Consul installation with ACLs enabled, configure the ACL controller. 1. **Sidecar containers:** Consul on ECS requires two sidecar containers to run in each ECS task: a Consul agent container and a sidecar proxy container. These additional sidecar containers must - be included in the ECS task definition. The Consul ECS Terraform modules will include these - sidecar containers for you. Or if you do not use Terraform, you can construct the task - definition yourself by following our documentation. + be included in the ECS task definition. The [Consul ECS Terraform module](/docs/ecs/install) + will include these sidecar containers for you. Or if you do not use Terraform, you can construct + the task definition yourself by following [our documentation](/docs/ecs/manual-installation). 1. **Routing:** With your application running in tasks as part of the mesh, you must specify the upstream services that your application calls. You will also need to change the URLs your application uses to ensure the application is making requests through the service mesh. diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index fdbacd0e9..3e1566d56 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -633,7 +633,7 @@ }, { "title": "Manual Installation", - "path": "ecs/install-no-terraform" + "path": "ecs/manual-installation" } ] }, From 69de9c4d2e90448d9ac1572cd863357689468e63 Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Fri, 21 Jan 2022 10:12:32 -0600 Subject: [PATCH 08/13] docs: Apply suggestions to ECS docs from code review Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- website/content/docs/ecs/install.mdx | 12 ++++--- .../content/docs/ecs/manual-installation.mdx | 32 +++++++++---------- website/content/docs/ecs/requirements.mdx | 2 +- 3 files changed, 23 insertions(+), 23 deletions(-) diff --git a/website/content/docs/ecs/install.mdx b/website/content/docs/ecs/install.mdx index 3554043cd..32d3dee3a 100644 --- a/website/content/docs/ecs/install.mdx +++ b/website/content/docs/ecs/install.mdx @@ -1,15 +1,17 @@ --- layout: docs -page_title: Installation with Terraform - AWS ECS +page_title: Installing Consul on AWS ECS using Terraform description: >- Install Consul Service Mesh on AWS ECS with Terraform (Elastic Container Service). --- # Installation with Terraform -This will describe how to use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh. +This topic describes how to use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh. -## Pre-requisites +This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. + +## Prerequisites * This pages assumes you are familiar with Terraform. If you are new to Terraform, see the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform. @@ -24,7 +26,7 @@ and additional sidecar containers, such as the Consul agent container and the En The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers. -Here is an example Terraform configuration file which defines a task definition with an application container called `example-client-app`. +The following example shows a Terraform configuration file that defines a task definition with an application container called `example-client-app`. ```hcl module "my_task" { @@ -61,7 +63,7 @@ however there are some important inputs worth highlighting: | Input Variable | Type | Description | | ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `source` and `version` | string | This specifies the source location and version of the `mesh-task` module. | -| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name, by default. | +| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. | | `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. | | `port` | number | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | | `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | diff --git a/website/content/docs/ecs/manual-installation.mdx b/website/content/docs/ecs/manual-installation.mdx index 4a4f80e43..16b6dd5a1 100644 --- a/website/content/docs/ecs/manual-installation.mdx +++ b/website/content/docs/ecs/manual-installation.mdx @@ -7,19 +7,17 @@ description: >- # Manual Installation -While the [Consul ECS Terraform module](/docs/ecs/install) is the easiest way to use Consul on ECS, -this page will describe how to directly create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) -for use without Terraform. +This topic describes how to manually create the ECS task definition for use without Terraform from the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs). Refer to the [Consul ECS Terraform module](/docs/ecs/install) documentation for an alternative method for installing Consul on ECS. -## Pre-requisites +This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. -* This page assumes you are familiar with AWS ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details. -* This page does not show how to create all necessary AWS resources, such as a VPC or the ECS Cluster. - For complete runnable examples, see the links in the [Getting Started](/docs/ecs#getting-started) section. +## Prerequisites + +You should have some familiarity with AWS ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for details. ## Task Definition -You must define a Task Definition which includes the following containers: +You must define a task definition which includes the following containers: * Your application container * An Envoy sidecar-proxy container @@ -29,7 +27,7 @@ You must define a Task Definition which includes the following containers: ## Top-level fields -In your task definition, you'll need to define these important top-level fields: +Your task definition must include the following top-level fields: ```json { @@ -49,9 +47,9 @@ In your task definition, you'll need to define these important top-level fields: | Field name | Type | Description | | ---------------------- | ------ | ------------------------------------------------------------------------------------------------------------------ | -| `family` | string | The task family name. This is used as the Consul service name, by default. | +| `family` | string | The task family name. This is used as the Consul service name by default. | | `networkMode` | string | Must be `awsvpc`, which is the only network mode supported by Consul on ECS. | -| `volumes` | list | Must be defined as shown above. Volumes ares used to share configuration between containers for intial task setup. | +| `volumes` | list | Must be defined as shown above. Volumes are used to share configuration between containers for intial task setup. | | `containerDefinitions` | list | The list of containers to run in this task (see below). | ## Application container @@ -95,7 +93,7 @@ See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/devel ## `sidecar-proxy` container -The sidecar proxy container runs [Envoy proxy](/docs/connect/proxies/envoy) for Consul Connect. +The `sidecar-proxy` container runs [Envoy proxy](/docs/connect/proxies/envoy) for Consul Connect. In most cases, the container should contain the following parameters and values. @@ -146,7 +144,7 @@ The sidecar proxy container runs [Envoy proxy](/docs/connect/proxies/envoy) for -In most cases, the `sidecar-proxy` container can be configured exactly as shown above. +The following table describes the available configuration settings. | Field name | Type | Description | | ------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -252,7 +250,7 @@ exec consul agent \ -config-file /consul/agent-defaults.hcl ``` -Set the following fields as shown above: +The following table describes the values that you should use to configure the `command` script: | Field name | Type | Description | | -------------------- | ------- | ------------------------------------------------------------------------------------------------------------ | @@ -263,7 +261,7 @@ Set the following fields as shown above: | `leave_on_terminate` | boolean | Must be set to `true` so that the Consul agent leaves the cluster gracefully before exiting. | | `retry_join` | string | Must be set to your Consul server location(s) so this agent can join the Consul cluster. | --> **NOTE**: Use `exec` to start the Consul agent, so that the Consul agent runs as PID 1. This ensures +-> **NOTE**: Use `exec` to start the Consul agent so that the Consul agent runs as PID 1. This ensures the Consul agent directly receives signals from ECS, which is important for graceful shutdown of the Consul agent. Refer to the [Consul Agent documentation](/docs/agent/options#configuration_files) for a complete reference of Consul agent @@ -364,14 +362,14 @@ Here is the sample config from above, expanded to be readable: | `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. | | `service.name` | string | The name used to register this service into the Consul service catalog. | | `service.port` | number | The port your application listens on. Set to `0` if your application does not listen on any port. | -| `service.checks` | list | Consul [checks](/docs/discovery/checks) to include, to have Consul run health checks against your application. | +| `service.checks` | list | Specifies the Consul health [checks](/docs/discovery/checks) to run against your application. | See the [`consul-ecs JSON Schema`](https://github.com/hashicorp/consul-ecs/blob/main/config/schema.json) for a complete reference of fields. ## `consul-ecs-health-sync` container Optionally, Consul ECS can sync health checks for this task into Consul checks. -This allows you to configure a health check for your application in one place, and +This allows you to configure a health check for your application in one place and see a consistent health status in both ECS and Consul. For example, the following defines an ECS health check command that runs `curl localhost:9090/health`: diff --git a/website/content/docs/ecs/requirements.mdx b/website/content/docs/ecs/requirements.mdx index 73570f16b..faedb2f2a 100644 --- a/website/content/docs/ecs/requirements.mdx +++ b/website/content/docs/ecs/requirements.mdx @@ -16,7 +16,7 @@ The following requirements must be met in order to install Consul on ECS: 1. **Sidecar containers:** Consul on ECS requires two sidecar containers to run in each ECS task: a Consul agent container and a sidecar proxy container. These additional sidecar containers must be included in the ECS task definition. The [Consul ECS Terraform module](/docs/ecs/install) - will include these sidecar containers for you. Or if you do not use Terraform, you can construct + will include these sidecar containers for you. If you do not use Terraform, you can construct the task definition yourself by following [our documentation](/docs/ecs/manual-installation). 1. **Routing:** With your application running in tasks as part of the mesh, you must specify the upstream services that your application calls. You will also need to change the URLs your From 2d5b7b03aaed249df148163066eac734986e786c Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Fri, 21 Jan 2022 14:11:11 -0600 Subject: [PATCH 09/13] docs: Improving ECS installation docs --- website/content/docs/ecs/install.mdx | 23 +++-- .../content/docs/ecs/manual-installation.mdx | 85 ++++++++++++------- 2 files changed, 65 insertions(+), 43 deletions(-) diff --git a/website/content/docs/ecs/install.mdx b/website/content/docs/ecs/install.mdx index 32d3dee3a..dc35b53ef 100644 --- a/website/content/docs/ecs/install.mdx +++ b/website/content/docs/ecs/install.mdx @@ -1,32 +1,29 @@ --- layout: docs -page_title: Installing Consul on AWS ECS using Terraform +page_title: Installing Consul on AWS ECS using Terraform description: >- Install Consul Service Mesh on AWS ECS with Terraform (Elastic Container Service). --- # Installation with Terraform -This topic describes how to use the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh. +This topic describes how to use the [`mesh-task`](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) Terraform module to launch your application in AWS ECS as part of Consul service mesh. If you do not use Terraform, see the [Manual Installation](/docs/ecs/manual-installation) page to install Consul on ECS without Terraform. This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. ## Prerequisites -* This pages assumes you are familiar with Terraform. If you are new to Terraform, see the [Terraform documentation](https://www.terraform.io/docs) to - learn about infrastructure as code and how to get started with Terraform. -* This page assumes you are familiar with AWS ECS. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for more details. -* This page does not show how to create all necessary AWS resources, such as a VPC or the ECS Cluster. - For complete runnable examples, see the links in the [Getting Started](/docs/ecs#getting-started) section. +* You should have some familiarity with using Terraform. Refer to the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform. +* You should also be familiar with AWS ECS before following these instructions. See [What is Amazon Elastic Container Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) for details. ## Using the Mesh Task Module -To run an application in ECS with Consul service mesh, you must create an ECS task definition which includes your application container(s) +To run an application in ECS with Consul service mesh, you must create an ECS task definition, which includes your application container(s) and additional sidecar containers, such as the Consul agent container and the Envoy sidecar proxy container. The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers. -The following example shows a Terraform configuration file that defines a task definition with an application container called `example-client-app`. +The following example shows a Terraform configuration file that creates a task definition with an application container called `example-client-app`. ```hcl module "my_task" { @@ -52,7 +49,7 @@ module "my_task" { } ] - port = "9090" + port = 9090 retry_join = ["
"] } ``` @@ -63,9 +60,9 @@ however there are some important inputs worth highlighting: | Input Variable | Type | Description | | ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `source` and `version` | string | This specifies the source location and version of the `mesh-task` module. | -| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. | +| `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. | | `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. | -| `port` | number | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | +| `port` | integer | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | | `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | ## ECS Service @@ -131,7 +128,7 @@ module "web" { | Input Variable | Type | Description | | ----------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `destinationName` | string | The name of the upstream service, as it is registered in the Consul service catalog. | -| `localBindPort` | number | Requests to this port will be forwarded by the proxy to the upstream service. This must be an unused port, but does not need to match the upstream service port. | +| `localBindPort` | integer | Requests to this port will be forwarded by the proxy to the upstream service. This must be an unused port, but does not need to match the upstream service port. | If you have multiple upstream services they each need to be listed here. diff --git a/website/content/docs/ecs/manual-installation.mdx b/website/content/docs/ecs/manual-installation.mdx index 16b6dd5a1..56ca85827 100644 --- a/website/content/docs/ecs/manual-installation.mdx +++ b/website/content/docs/ecs/manual-installation.mdx @@ -7,7 +7,7 @@ description: >- # Manual Installation -This topic describes how to manually create the ECS task definition for use without Terraform from the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs). Refer to the [Consul ECS Terraform module](/docs/ecs/install) documentation for an alternative method for installing Consul on ECS. +The following instructions describe how to manually create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) without Terraform. Refer to the [Consul ECS Terraform module](/docs/ecs/install) documentation for an alternative method for installing Consul on ECS. This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. @@ -17,17 +17,22 @@ You should have some familiarity with AWS ECS. See [What is Amazon Elastic Conta ## Task Definition -You must define a task definition which includes the following containers: +You must create a task definition, which includes the following containers: * Your application container * An Envoy sidecar-proxy container * A Consul client container -* The `consul-ecs-mesh-init` container for service mesh setup +* A `consul-ecs-mesh-init` container for service mesh setup * Optionally, a `consul-ecs-health-sync` container to sync ECS health checks into Consul ## Top-level fields -Your task definition must include the following top-level fields: +Your task definition must include the following top-level fields. + +The `volumes` list contains two [bind mounts](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/bind-mounts.html), +named `consul_data` and `consul_binary`. Bind mounts are directories on the host which can be mounted into one or more containers +in order to share files among containers. For Consul on ECS, certain binaries and configuration are shared among containers +during task startup. ```json { @@ -55,9 +60,14 @@ Your task definition must include the following top-level fields: ## Application container First, include your application container in the `containerDefinitions` list -in the task definition. Ensure that the `dependsOn` field is set as shown below -so that your application container starts in the correct order (see [task -startup](/docs/ecs/architecture#task-startup) for more information): +in the task definition. + +Ensure that the `containerName` and `condition` fields in the `dependsOn` list +are specified as described in the following example. These are container dependencies, +which must be used to enforce a specific [startup order](/docs/ecs/architecture#task-startup). +By using the following settings, your application container will start after `consul-ecs-mesh-init` +has completed task setup and after `sidecar-proxy` is ready to proxy traffic between +this task and the service mesh. ```json { @@ -87,7 +97,7 @@ startup](/docs/ecs/architecture#task-startup) for more information): | `name` | string | The name of your application container. | | `image` | string | The container image used to run your application. | | `essential` | boolean | Must be `true` to ensure your application container ties into the health of the task. | -| `dependsOn` | list | Must be set as show above. Container dependencies ensure your application container starts after service mesh setup is complete. | +| `dependsOn` | list | Must be set as shown above. Container dependencies ensure your application container starts after service mesh setup is complete. | See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html) documentation for a complete reference. @@ -95,6 +105,11 @@ See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/devel The `sidecar-proxy` container runs [Envoy proxy](/docs/connect/proxies/envoy) for Consul Connect. In most cases, the container should contain the following parameters and values. +The `mountPoints` list must be set as shown in the following example. This will mount the shared `consul_data` volume into the +`sidecar-proxy` container at the path `/consul`. This volume is where the `consul-ecs-mesh-init` container copies the `envoy-bootstrap.json` +file and the `consul-ecs` binary, which are required to start Envoy. The `dependsOn` list must also be defined as follows to ensure the +`sidecar-proxy` container starts after `consul-ecs-mesh-init` has successfully written these files to the shared volume. + ```json @@ -144,7 +159,7 @@ The `sidecar-proxy` container runs [Envoy proxy](/docs/connect/proxies/envoy) fo -The following table describes the available configuration settings. +The following table describes the necessary configuration settings. | Field name | Type | Description | | ------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -152,9 +167,9 @@ The following table describes the available configuration settings. | `image` | string | The Envoy image. This must be a [supported version of Envoy](/docs/connect/proxies/envoy#supported-versions). | | `dependsOn` | list | Must be set as shown above to ensure Envoy starts after the `consul-ecs-mesh-init` has written the `envoy-bootstrap.json` config file for Envoy. | | `healthCheck` | list | Must be set as shown above to monitor the health of Envoy's primary listener port, which ties into container dependencies and startup ordering. | -| `mountPoints` | list | Must be set as shown above to access the files shared in the `consul` directory, like the Envoy bootstrap configuration file and the `consul-ecs` binary. | +| `mountPoints` | list | Must be set as shown above to access the files shared in the `/consul` directory, like the Envoy bootstrap configuration file and the `consul-ecs` binary. | | `ulimits` | list | The `nofile` ulimit must be raised to a sufficiently high value so that Envoy does not fail to open sockets. | -| `entrypoint` | list | Must be set to the custom Envoy entrypoint to facilitate graceful shutdown. | +| `entrypoint` | list | Must be set to the custom Envoy entrypoint, `consul-ecs envoy-entrypoint`, to facilitate graceful shutdown. | | `command` | list | The startup command. This passes the bootstrap configuration to Envoy. | -> **NOTE**: Envoy and Consul must be compatible versions. See the [supported versions of Envoy](/docs/connect/proxies/envoy#supported-versions) in the Consul documentation. @@ -215,7 +230,7 @@ The following is the recommended `command` script for the Consul agent. This is the same as the above `command` field, but is unescaped and has comments added. ```shell -# Copy the consul binary to a shared volume for mesh-init to use to generate Envoy configuration. +# Copy the consul binary to a shared volume for `consul-ecs-mesh-init` to use to generate Envoy configuration. cp /bin/consul /bin/consul-inject/consul # At runtime, determine the IP address assigned to this ECS Task. @@ -267,9 +282,9 @@ the Consul agent directly receives signals from ECS, which is important for grac Refer to the [Consul Agent documentation](/docs/agent/options#configuration_files) for a complete reference of Consul agent configuration options. -## `mesh-init` container +## `consul-ecs-mesh-init` container -The `mesh-init` container runs at task startup to setup this instance for Consul service mesh. +The `consul-ecs-mesh-init` container runs at task startup to setup this instance for Consul service mesh. It registers the service and proxy for this task with Consul and writes Envoy bootstrap configuration to a shared volume. @@ -329,10 +344,16 @@ configuration to a shared volume. | `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | | `mountPoints` | list | Must be set as show above, so the `consul` and `consul-ecs` binaries can be shared among containers for task setup. | | `command` | list | Set the `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. | -| `environment` | list | This must include the `CONSUL_ECS_CONFIG_JSON` variable. See below for details. | +| `environment` | list | This must include the [`CONSUL_ECS_CONFIG_JSON`](/docs/ecs/manual-installation#consul_ecs_config_json) variable. See below for details. | -Configuration is passed to the `consul-ecs mesh-init` command in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable. -Here is the sample config from above, expanded to be readable: +### `CONSUL_ECS_CONFIG_JSON` + +Configuration is passed to the `consul-ecs` binary in JSON format using the `CONSUL_ECS_CONFIG_JSON` environment variable. + +The following is an example of the configuration that might be used for a service named `example-client-app` with one upstream +service name `example-server-app`. The `proxy` and `service` blocks include information used by `consul-ecs-mesh-init` to perform +[service registration](/docs/discovery/services) with Consul during task startup. The same configuration format is used for +the `consul-ecs-health-sync` container. ```json { @@ -356,13 +377,14 @@ Here is the sample config from above, expanded to be readable: } ``` -| Field name | Type | Description | -| ----------------- | ------ | ------------------------------------------------------------------------------------------------------------------------------ | -| `bootstrapDir` | string | This is the path of a shared volume the is mounted to other containers, where `mesh-init` will write out Envoy configuration. | -| `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. | -| `service.name` | string | The name used to register this service into the Consul service catalog. | -| `service.port` | number | The port your application listens on. Set to `0` if your application does not listen on any port. | -| `service.checks` | list | Specifies the Consul health [checks](/docs/discovery/checks) to run against your application. | +| Field name | Type | Description | +| ---------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | +| `bootstrapDir` | string | This is the path of a shared volume the is mounted to other containers, where `consul-ecs-mesh-init` will write out Envoy configuration. | +| `healthSyncContainers` | list | Used for [health status syncing](/docs/ecs/manual-installation#consul-ecs-health-sync-container) from ECS to Consul. See below for details. | +| `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. The `destinationName` and `localBindPort` fields are required. | +| `service.name` | string | The name used to register this service into the Consul service catalog. | +| `service.port` | integer | The port your application listens on. Set to `0` if your application does not listen on any port. | +| `service.checks` | list | Consul [checks](/docs/discovery/checks) to include so that Consul can run health checks against your application. | See the [`consul-ecs JSON Schema`](https://github.com/hashicorp/consul-ecs/blob/main/config/schema.json) for a complete reference of fields. @@ -397,8 +419,10 @@ For example, the following defines an ECS health check command that runs `curl l -First, tell Consul ECS which containers need their health status synced into Consul. To do this, -add the container name(s) to the `healthSyncContainers` list of the `CONSUL_ECS_CONFIG_JSON` variable: +First, define which containers need their health status synced into Consul. To do this, +add the container name(s) to the `healthSyncContainers` list of the `CONSUL_ECS_CONFIG_JSON` variable, +as shown in the following example. This configuration must be passed to both the `consul-ecs-mesh-init` +and `consul-ecs-health-sync` containers. @@ -412,8 +436,9 @@ add the container name(s) to the `healthSyncContainers` list of the `CONSUL_ECS_ -Next, pass the new configuration to the `consul-ecs-mesh-init` container. You should compact and escape the JSON -configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON` environment variable: +Next, set the `CONSUL_ECS_CONFIG_JSON` variable for the `consul-ecs-mesh-init` container. +The following example shows how the `CONSUL_ECS_CONFIG_JSON` variable should be formatted. +The JSON configuration is compacted down to a single line and escaped. @@ -439,8 +464,8 @@ configuration above, and copy the result into the `CONSUL_ECS_CONFIG_JSON` envir Finally, include the `consul-ecs-health-sync` container in the `containerDefinitions` list. -The exact same configuration for the `CONSUL_ECS_CONFIG_JSON` environment variable is used -for the health sync container. +Pass the same value for `CONSUL_ECS_CONFIG_JSON` for both the `consul-ecs-health-sync` +and `consul-ecs-mesh-init` containers. From 270b8255cd63d46cf136474e13ba7dcc54607cdd Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Tue, 25 Jan 2022 12:24:37 -0600 Subject: [PATCH 10/13] docs: Refine ECS installation docs Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- website/content/docs/ecs/install.mdx | 95 ++++++++++++++++--- .../content/docs/ecs/manual-installation.mdx | 27 +++--- 2 files changed, 98 insertions(+), 24 deletions(-) diff --git a/website/content/docs/ecs/install.mdx b/website/content/docs/ecs/install.mdx index dc35b53ef..ce51332b5 100644 --- a/website/content/docs/ecs/install.mdx +++ b/website/content/docs/ecs/install.mdx @@ -11,6 +11,17 @@ This topic describes how to use the [`mesh-task`](https://registry.terraform.io/ This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. +## Overview + +This topic describes the following procedure: + +1. Create Terraform configuration files for the necessary components: + + * [ECS task definition](#using-the-mesh-task-module): Use the `mesh-task` module to create an ECS task definition for Consul on ECS + * [ECS service](#ecs-service): Use the `aws_ecs_service` resource to create an ECS service that schedules service mesh tasks to run on ECS + +2. [Run Terraform](#running-terraform) to deploy the resources in AWS + ## Prerequisites * You should have some familiarity with using Terraform. Refer to the [Terraform documentation](https://www.terraform.io/docs) to learn about infrastructure as code and how to get started with Terraform. @@ -23,7 +34,9 @@ and additional sidecar containers, such as the Consul agent container and the En The [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task) will automatically include the necessary sidecar containers. -The following example shows a Terraform configuration file that creates a task definition with an application container called `example-client-app`. +The following example shows a Terraform configuration file that creates a task definition with an application container called `example-client-app` in a file called `mesh-task.tf`: + + ```hcl module "my_task" { @@ -54,26 +67,84 @@ module "my_task" { } ``` -All possible inputs are documented in the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs), -however there are some important inputs worth highlighting: + + +The following fields are required. Refer to the [module reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs) for a complete reference. | Input Variable | Type | Description | | ----------------------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `source` and `version` | string | This specifies the source location and version of the `mesh-task` module. | +| `source` | string | Must be set to the source location of the `mesh-task` module, `hashicorp/consul-ecs/aws//modules/mesh-task`. | +| `version` | string | Must be set to the version of the `mesh-task` module. | | `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. | | `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. | | `port` | integer | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | | `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | +### Running Terraform + +You will need to run Terraform to create the task definition. + +Save the Terraform configuration for the task definition to a file, such as `mesh-task.tf`. +You should place this file in a directory alongside other Terraform configuration files for your project. + +The `mesh-task` module requires the AWS Terraform provider. The following example shows how to include +and configure the AWS provider in a file called `provider.tf`. Refer to the [AWS Terraform provider](https://registry.terraform.io/providers/hashicorp/aws/latest/docs) +documentation for complete configuration details. + + + +```hcl +terraform { + required_providers { + aws = { + source = "hashicorp/aws" + version = "" + } + } +} + +provider "aws" { + region = "" + ... +} +``` + + + +Additional AWS resources for your project can be included in additional Terraform configuration files +in the same directory. The following example shows a basic project directory: + +```shell-session +$ ls +mesh-task.tf +provider.tf +... +``` + +Terraform should be run in your project directory as follows. + +* Run `terraform init` first to download dependencies, such as Terraform providers +* Run `terraform apply` to have Terraform create AWS resources, such as the task definition from the `mesh-task` module. + +Terraform automatically reads all files in the current directory that have a `.tf` file extension. +Refer to the [Terraform documentation](https://www.terraform.io/docs) for more information and Terraform best practices. + ## ECS Service [ECS services](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) are one of the most common -ways to start tasks using a task definition.. +ways to start tasks using a task definition. -To define an ECS Service, reference the mesh-task module's `task_definition_arn` output value -in your `aws_ecs_service` resource: +To define an ECS service, reference the `mesh-task` module's `task_definition_arn` output value +in your `aws_ecs_service` resource. The following example shows how to include the service in the `mesh-task.tf` file. + + ```hcl +module "my_task" { + source = "hashicorp/consul-ecs/aws//modules/mesh-task" + ... +} + resource "aws_ecs_service" "my_task" { name = "my_task_service" task_definition = module.my_task.task_definition_arn @@ -83,6 +154,8 @@ resource "aws_ecs_service" "my_task" { } ``` + + This is a partial configuration to highlight some important fields. See the [`aws_ecs_service`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service) documentation for a complete reference. @@ -93,10 +166,10 @@ See the [`aws_ecs_service`](https://registry.terraform.io/providers/hashicorp/aw | `launch_type` | string | The launch type. Consul on ECS supports the `FARGATE` and `EC2` launch types. | | `propagate_tags` | string | This must be set to `TASK_DEFINITION` so that tags added by `mesh-task` to the task definition are copied to tasks. | -After defining the Terraform configuration for both the `mesh-task` and ECS service, -run `terraform apply` to create the ECS task definition and service resources. The ECS -service will soon start your application in a task. The task will automatically -register itself into the Consul service catalog. +After including the ECS service in your Terraform configuration, run `terraform apply` +from your project directory to create the ECS service resource. The ECS service will +soon start your application in a task. The task will automatically register itself +into the Consul service catalog during startup. -> **NOTE:** If your tasks run in a public subnet, they must have `assign_public_ip = true` in their [`network_configuration`](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/ecs_service#network_configuration) block so that ECS can pull the Docker images. diff --git a/website/content/docs/ecs/manual-installation.mdx b/website/content/docs/ecs/manual-installation.mdx index 56ca85827..31e57eac3 100644 --- a/website/content/docs/ecs/manual-installation.mdx +++ b/website/content/docs/ecs/manual-installation.mdx @@ -52,10 +52,10 @@ during task startup. | Field name | Type | Description | | ---------------------- | ------ | ------------------------------------------------------------------------------------------------------------------ | -| `family` | string | The task family name. This is used as the Consul service name by default. | +| `family` | string | The task family name. This is used as the Consul service name by default. | | `networkMode` | string | Must be `awsvpc`, which is the only network mode supported by Consul on ECS. | -| `volumes` | list | Must be defined as shown above. Volumes are used to share configuration between containers for intial task setup. | -| `containerDefinitions` | list | The list of containers to run in this task (see below). | +| `volumes` | list | Must be defined as shown above. Volumes are used to share configuration between containers for intial task setup. | +| `containerDefinitions` | list | The list of containers to run in this task (see [Application container](#application-container)). | ## Application container @@ -218,16 +218,17 @@ Each task must include a Consul client container in order for the task to join y -| Field name | Type | Description | -| ------------- | ------- | ------------------------------------------------------------------------------------------------------------------- | -| `name` | string | The container name, which should always be `consul-client`. | -| `image` | string | The Consul image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul`, to avoid rate limits. | -| `mountPoints` | list | Must be set as shown above. Volumes are mounted to share information with other containers for task setup. | -| `entrypoint` | list | Must be set to a plain shell so that the startup `command` works properly. | -| `command` | list | The startup command. See below for details. | +| Field name | Type | Description | +| ------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name, which should always be `consul-client`. | +| `image` | string | The Consul image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul`, to avoid rate limits. | +| `mountPoints` | list | Must be set as shown above. Volumes are mounted to share information with other containers for task setup. | +| `entrypoint` | list | Must be set to a plain shell so that the startup `command` works properly. | +| `command` | list | Specifies the contents of the [startup script](#consul-client-startup-script). Copy the script and format it into a JSON string. | -The following is the recommended `command` script for the Consul agent. -This is the same as the above `command` field, but is unescaped and has comments added. +### Consul client startup script + +The following script is used to start the Consul client for Consul on ECS. ```shell # Copy the consul binary to a shared volume for `consul-ecs-mesh-init` to use to generate Envoy configuration. @@ -271,7 +272,7 @@ The following table describes the values that you should use to configure the `c | -------------------- | ------- | ------------------------------------------------------------------------------------------------------------ | | `addresses.*` | strings | Set the DNS, GRPC, and HTTP addresses to `127.0.0.1` to ensure these are not accessible outside of the task. | | `advertise_addr` | string | Must be set to the task IP address so that other Consul agents know how to reach this agent. | -| `client_addr` | string | Must be set to an interface reacable by other Consul agents. | +| `client_addr` | string | Must be set to an interface reachable by other Consul agents. | | `datacenter` | string | Must be set to the Consul datacenter this task will join. | | `leave_on_terminate` | boolean | Must be set to `true` so that the Consul agent leaves the cluster gracefully before exiting. | | `retry_join` | string | Must be set to your Consul server location(s) so this agent can join the Consul cluster. | From 558967c767888275e19ce4fef2cb1daf348f3b95 Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Mon, 24 Jan 2022 17:43:20 -0600 Subject: [PATCH 11/13] docs: Manual secure configuration for ECS --- website/content/docs/ecs/index.mdx | 6 +- .../docs/ecs/manual/acl-controller.mdx | 213 +++++++++++++++++ .../install.mdx} | 37 ++- .../docs/ecs/manual/secure-configuration.mdx | 215 ++++++++++++++++++ website/content/docs/ecs/requirements.mdx | 4 +- .../docs/ecs/{ => terraform}/install.mdx | 2 +- .../migrate-existing-tasks.mdx | 4 +- .../secure-configuration.mdx} | 10 +- website/data/docs-nav-data.json | 42 +++- 9 files changed, 501 insertions(+), 32 deletions(-) create mode 100644 website/content/docs/ecs/manual/acl-controller.mdx rename website/content/docs/ecs/{manual-installation.mdx => manual/install.mdx} (92%) create mode 100644 website/content/docs/ecs/manual/secure-configuration.mdx rename website/content/docs/ecs/{ => terraform}/install.mdx (99%) rename website/content/docs/ecs/{ => terraform}/migrate-existing-tasks.mdx (96%) rename website/content/docs/ecs/{production-installation.mdx => terraform/secure-configuration.mdx} (92%) diff --git a/website/content/docs/ecs/index.mdx b/website/content/docs/ecs/index.mdx index 1078d65c7..cf9e63816 100644 --- a/website/content/docs/ecs/index.mdx +++ b/website/content/docs/ecs/index.mdx @@ -9,7 +9,8 @@ description: >- # AWS ECS Consul service mesh applications can be deployed on [AWS Elastic Container Service](https://aws.amazon.com/ecs/) (ECS) -using our official Terraform modules. +using either our official [Terraform modules](/docs/ecs/terraform/install) or without Terraform by [manually configuring +the task definition](/docs/ecs/manual/install). ## Service Mesh @@ -37,4 +38,5 @@ There are several ways to get started with Consul with ECS. * The [Consul with Dev Server on Fargate](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-fargate) example installation deploys a sample application in ECS using the Fargate launch type. * The [Consul with Dev Server on EC2](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-ec2) example installation deploys a sample application in ECS using the EC2 launch type. -See the [Requirements](/docs/ecs/requirements) and the full [Install Guide](/docs/ecs/install) when you're ready to install Consul on an existing ECS cluster and add existing tasks to the service mesh. +See the [Requirements](/docs/ecs/requirements) and then choose whether to [install with Terraform](/docs/ecs/terraform/install) or [install without Terraform](/docs/ecs/manual/install) +when you're ready to install Consul on an existing ECS cluster and add tasks to the service mesh. diff --git a/website/content/docs/ecs/manual/acl-controller.mdx b/website/content/docs/ecs/manual/acl-controller.mdx new file mode 100644 index 000000000..7264ea4f2 --- /dev/null +++ b/website/content/docs/ecs/manual/acl-controller.mdx @@ -0,0 +1,213 @@ +--- +layout: docs +page_title: ACL Controller - AWS ECS +description: >- + Manual Deployment of the ACL Controller for Consul Service Mesh on AWS ECS (Elastic Container Service). +--- + +# Install the ACL Controller + +This topic describes how to manually deploy the ACL controller to [automatically provision ACL tokens](/docs/ecs/architecture#automatic-acl-token-provisioning) for Consul on ECS. +If you are using Terraform, refer to the [Terraform Secure Configuration](/docs/ecs/terraform/secure-configuration) page to deploy the ACL controller. + +## Prerequisites + +* Your application tasks must include certain tags to be compatible with the ACL controller. +Refer to the [Task Tags](/docs/ecs/manual/install#task-tags) section of the installation page. +* You should be familiar with configuring Consul's secure features, including how to create ACL tokens and policies. Refer to the following [Learn Guides](https://learn.hashicorp.com/collections/consul/security) for an introduction and the [ACL system](/docs/security/acl) documentation for more information. + +## Setup AWS Secrets Manager + +The ACL controller supports managing secrets in AWS Secrets Manager. + +Before deploying the ACL controller for the first time, you must [create the following secrets](https://docs.aws.amazon.com/secretsmanager/latest/userguide/manage_create-basic-secret.html) from Consul in AWS Secrets Manager. + +| Secret | Initial Value | Sample Secret Name | +| --------------------- | -------------- | ------------------------------ | +| Consul server CA cert | Set | `my-consul-ca-cert` | +| Bootstrap ACL Token | Set | `my-consul-bootstrap-token` | +| Consul Client ACL Token | Empty | `-consul-client-token` | + +The secret for the client token must be intially empty. The ACL controller creates the client token in Consul +and stores the token in Secrets Manager. In the secret name, `` should be replaced with the +[secret name prefix](/docs/ecs/manual/acl-controller#secret-name-prefix) of your choice. + +### Secret Name Prefix + +The ACL controller requires that the secrets it reads and writes are named with a unique prefix. The name prefix is used +in the [Task Role Policy](/docs/ecs/manual/acl-controller#task-role-policy) to limit the ACL controller's access within +AWS Secrets Manager to only those secrets strictly needed by the ACL controller. + +The name prefix should be unique among secrets your AWS account. We recommend a short (8 character) random +string for the prefix. + +-> **NOTE:** If you are using the ACL controller with multiple ECS clusters, each cluster requires +its own instance of the ACL controller, and each instance of the ACL controller should have a unique +name prefix. + +## Task Definition + +You must create a task definition to deploy the ACL controller in your ECS cluster. +The ACL controller must run in the same ECS cluster hosting your service mesh application +tasks. + +The following example shows how the task definition should be configured for the ACL controller. + +```json +{ + "family": "my-consul-acl-controller". + "networkMode": "awsvpc", + "containerDefinitions": [ + { + "name": "acl-controller", + "image": "public.ecr.aws/hashicorp/consul-ecs:", + "essential": true, + "command": [ + "acl-controller", + "-consul-client-secret-arn", "arn:aws:secretsmanager:us-west-2:000000000000:secret:-consul-client-token", + "-secret-name-prefix", "", + ], + "secrets": [ + { + "name": "CONSUL_HTTP_TOKEN", + "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-bootstrap-token" + }, + { + "name": "CONSUL_CACERT_PEM", + "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-ca-cert" + } + ], + "environment": [ + { + "name": "CONSUL_HTTP_ADDR", + "value": "" + } + ] + } + ] +} +``` + +You must include the following top-level fields. + +| Field name | Type | Description | +| ----------- | ------- | ---------------------------------------------------------------------------- | +| `family` | string | The task family name of your choice. | +| `networkMode` | string | Must be `awsvpc`, which is the only network mode supported by Consul on ECS. | + +In the `containerDefinitions` list, include one container with the following fields. + +| Field name | Type | Description | +| ----------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name, which should be `acl-controller` | +| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | +| `command` | list | Must be set as shown. The startup command for the ACL controller. | +| `essential` | boolean | Must be `true` to ensure the container health ties into the health of the task. | +| `secrets` | list | Must have `CONSUL_HTTP_TOKEN` set to the ACL bootstrap token and `CONSUL_CACERT_PEM` set to the Consul server CA certificate. | +| `environment` | string | Must set the `CONSUL_HTTP_ADDR` environment variable to the address of the HTTP API of your Consul servers. | + +The following CLI options are required in the `command` field of the container definition. + +| Flag | Type | Description | +| --------------------------- | ------- | -------------------------------------------------------------------------------------------------------------------- | +| `-consul-client-secret-arn` | string | The secret where the ACL controller will store the Consul client token. | +| `-secret-name-prefix` | string | The [secret name prefix](/docs/ecs/manual/acl-controller#secret-name-prefix) that you chose for this ACL controller. | + +## ECS Service + +Once the task definition is created, define an ECS service in order to start an ACL controller task. + +The following example contains the recommended settings for the ACL controller. Refer to +the [ECS service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service_definition_parameters.html) documentation +to complete the remaining details for your use case. + +```json +{ + "cluster": "" + "desiredCount": 1, + "launchType": "FARGATE", + "serviceName": "my-acl-controller", + "taskDefinition": "", + ... +} +``` + +| Field name | Type | Description | +| ---------------- | ------- | ---------------------------------------------------------------------------------------------------------------- | +| `cluster` | string | Set to your ECS cluster name or ARN. This must be the same ECS cluster where your service mesh applications run. | +| `desiredCount` | integer | Must be 1. Only one instance of the ACL controller should run per ECS cluster. | +| `launchType` | string | Consul on ECS supports both the `FARGATE` and `EC2` launch types. | +| `serviceName` | string | The service name of your choice. | +| `taskDefinition` | string | Must be set to the ACL controller [task definition](/docs/ecs/manual/acl-controller#task-definition). | + +## AWS IAM Roles + +The ECS task and execution roles must be configured to allow the ACL controller access +to the ECS API and Secrets Manager API. + +### Task Role Policy + +The following example shows the policy needed for the ECS task role for the ACL controller. +This grants the ACL controller permission to list tasks, describe tasks, and read and update +secrets. + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "ecs:ListTasks", + "ecs:DescribeTasks" + ], + "Resource": ["*"] + }, + { + "Effect": "Allow", + "Action": [ + "secretsmanager:GetSecretValue", + "secretsmanager:UpdateSecret" + ], + "Resource": [ + "arn:aws:secretsmanager:us-west-2:000000000000:secret:-*" + ] + } + ] +} +``` + +The following are the required permissions. You will need to substitute `` with your chosen [name prefix](/docs/ecs/manual/acl-controller#secret-name-prefix). + +| Action | Resource | Description | +| ------------------------------- | ----------------------------------------------------------------- | ---------------------------------------------------------------------------------- | +| `ecs:ListTasks` | `*` | Allow the ACL controller to watch for new tasks. | +| `ecs:DescribeTasks` | `*` | Allow the ACL controller to retrieve details for new tasks. | +| `secretsmanager:GetSecretValue` | `arn:aws:secretsmanager:us-west-2:000000000000:secret:-*` | Allow the ACL controller to read secrets with a name prefix. | +| `secretsmanager:UpdateSecret` | `arn:aws:secretsmanager:us-west-2:000000000000:secret:-*` | Allow the ACL controller to store Consul ACL tokens in secrets with a name prefix. | + +### Execution Role Policy + +The following IAM policy document allows ECS to retrieve secrets needed +to start the ACL controller task from AWS Secrets Manager, including the ACL +bootstrap token. + +The following example shows the policy needed for the execution role. + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "secretsmanager:GetSecretValue" + ], + "Resource": [ + "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-bootstrap-token", + "arn:aws:secretsmanager:us-west-2:000000000000:secret:-consul-client-token" + ] + } + ] +} +``` diff --git a/website/content/docs/ecs/manual-installation.mdx b/website/content/docs/ecs/manual/install.mdx similarity index 92% rename from website/content/docs/ecs/manual-installation.mdx rename to website/content/docs/ecs/manual/install.mdx index 31e57eac3..87b98ed86 100644 --- a/website/content/docs/ecs/manual-installation.mdx +++ b/website/content/docs/ecs/manual/install.mdx @@ -7,7 +7,7 @@ description: >- # Manual Installation -The following instructions describe how to manually create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) without Terraform. Refer to the [Consul ECS Terraform module](/docs/ecs/install) documentation for an alternative method for installing Consul on ECS. +The following instructions describe how to manually create the ECS task definition using the [`consul-ecs` Docker image](https://gallery.ecr.aws/hashicorp/consul-ecs) without Terraform. Refer to the [Consul ECS Terraform module](/docs/ecs/terraform/install) documentation for an alternative method for installing Consul on ECS. This topic does not include instructions for creating all AWS resources necessary to install Consul, such as a VPC or the ECS cluster. Refer to the linked guides in the [Getting Started](/docs/ecs#getting-started) section for complete, runnable examples. @@ -47,6 +47,16 @@ during task startup. } ], "containerDefinitions": [...] + "tags": [ + { + "key": "consul.hashicorp.com/mesh", + "value": "true" + }, + { + "key": "consul.hashicorp.com/service-name", + "value": "example-client-app" + } + ] } ``` @@ -57,6 +67,16 @@ during task startup. | `volumes` | list | Must be defined as shown above. Volumes are used to share configuration between containers for intial task setup. | | `containerDefinitions` | list | The list of containers to run in this task (see [Application container](#application-container)). | +### Task Tags + +The `tags` list must include the following if you are using the ACL controller in a [secure configuration](/docs/manual/secure-configuration). +Without these tags, the ACL controller will be unable to provision a service token for the task. + +| Tag Key | Tag Value | Description | +| ----------------------------------- | ------------------- | -------------------------------------------------------------------------------------------------------------------------- | +| `consul.hashicorp.com/mesh` | `true` (string) | The ACL controller ignores tasks without this tag set to `true`. | +| `consul.hashicorp.com/service-name` | Consul service name | Specifies the Consul service associated with this task. Required if the service name is different than the task `family`. | + ## Application container First, include your application container in the `containerDefinitions` list @@ -339,13 +359,13 @@ configuration to a shared volume. -| Field name | Type | Description | -| ----------- | ------- | ------------------------------------------------------------------------------------------------------------------- | -| `name` | string | The container name should be `consul-ecs-mesh-init`. | -| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | -| `mountPoints` | list | Must be set as show above, so the `consul` and `consul-ecs` binaries can be shared among containers for task setup. | -| `command` | list | Set the `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. | -| `environment` | list | This must include the [`CONSUL_ECS_CONFIG_JSON`](/docs/ecs/manual-installation#consul_ecs_config_json) variable. See below for details. | +| Field name | Type | Description | +| ----------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The container name should be `consul-ecs-mesh-init`. | +| `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | +| `mountPoints` | list | Must be set as show above, so the `consul` and `consul-ecs` binaries can be shared among containers for task setup. | +| `command` | list | Set to `["mesh-init"]` so that the container runs the `consul-ecs mesh-init` command. | +| `environment` | list | This must include the [`CONSUL_ECS_CONFIG_JSON`](/docs/ecs/manual-installation#consul_ecs_config_json) variable. See below for details. | ### `CONSUL_ECS_CONFIG_JSON` @@ -529,3 +549,4 @@ and `consul-ecs-mesh-init` containers. * Create the task definition using the [AWS Console](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-taskdefinition.html) or the [AWS CLI](https://docs.aws.amazon.com/cli/latest/reference/ecs/register-task-definition.html), or another method of your choice. * Create an [ECS Service](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html) to start tasks using the task definition. +* Follow the [Secure Configration](/docs/ecs/manual/secure-configuration) to get production-ready. diff --git a/website/content/docs/ecs/manual/secure-configuration.mdx b/website/content/docs/ecs/manual/secure-configuration.mdx new file mode 100644 index 000000000..57b12467c --- /dev/null +++ b/website/content/docs/ecs/manual/secure-configuration.mdx @@ -0,0 +1,215 @@ +--- +layout: docs +page_title: Secure Configuration - AWS ECS +description: >- + Manual Secure Confguration of the Consul Service Mesh on AWS ECS (Elastic Container Service). +--- + +# Secure Configuration + +For a production-ready installation of Consul on ECS, you will need to make sure that the cluster is secured. +A secure Consul cluster should include the following: + +1. [TLS Encryption](/docs/security/encryption#rpc-encryption-with-tls) for RPC communication between Consul clients and servers. +1. [Gossip Encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. +1. [Access Control (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. + +-> **NOTE:** This page assumes that you have already configured your Consul server with the above features. + +## Prerequisites + +* You should already have followed the [installation instructions](/docs/ecs/manual/install) to understand how to define +the necessary components of the task definition for Consul on ECS. +* You should be familiar with [specifying sensitive data](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data.html) on ECS. +* You should be familiar with configuring Consul's secure features, including how to create ACL tokens and policies. Refer to the following [Learn Guides](https://learn.hashicorp.com/collections/consul/security) for an introduction and the [ACL system](/docs/security/acl) documentation for more information. + +## ACL Tokens + +You must create two types of ACL tokens for Consul on ECS: + +* **Client tokens:** used by the `consul-client` containers to join the Consul cluster +* **Service tokens:** used by sidecar containers for service registration and health syncing + +The following sections describe the ACL polices which must be associated with these token types. + +-> **NOTE:** This section describes how operators would create ACL tokens by hand. To ease operator +burden, the ACL Controller can automatically create ACL tokens for Consul on ECS. Refer to the +[ACL Controller](/docs/manual/acl-controller) page for installation details. + +### Create Consul client token + +You must create a token for the Consul client. This is a shared token used by the `consul-client` +containers to join the Consul cluster. + +The following is the ACL policy needed for the Consul client token: + +```hcl +node_prefix "" { + policy = "write" +} +service_prefix "" { + policy = "read" +} +``` + +This policy allows `node:write` for any node name, which is necessary because the Consul node +names on ECS are not known until runtime. + +### Create service tokens + +Service tokens should be associated with a [service identity](https://www.consul.io/docs/security/acl/acl-system#acl-service-identities). +The service identity includes `service:write` permissions for the service and sidecar proxy. + +The following example shows how to use the Consul CLI to create a service token for a service named `example-client-app`: + +```shell +consul acl token create -service-identity=example-client-app ... +``` + +-> **NOTE**: You will need to create one service token for each registered Consul service in ECS, +including when new services are added to the service mesh. + +## Secret storage + +You should securely store the following secrets in order to make them available to ECS tasks. + +1. Consul Server CA certificate +2. Consul gossip encryption key +3. Consul client ACL token +4. Consul service ACL tokens (one per service) + +These secrets can be securely stored and passed to ECS tasks using either of the following AWS secret services: + +* [AWS Systems Manager Parameter Store](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data-parameters.html) +* [AWS Secrets Manager](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data-secrets.html) + +Once the secrets are stored they can be referenced using their ARN. The following shows +example secret ARNs when using AWS Secrets Manager: + +| Secret | Sample Secret ARN | +| ---------------------- | ---------------------------------------------------------------------------------- | +| Consul Server CA Cert | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-ca-cert` | +| Gossip encryption key | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-gossip-key` | +| Client token | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-client-token` | +| Service token | `arn:aws:secretsmanager:us-west-2:000000000000:secret:my-example-client-app-token` | + +## Configure `consul-client` + +The following secrets must be passed to the `consul-client` container: + +* Consul server CA certificate +* Gossip encryption key +* Consul client ACL token + +The following example shows how to include these secrets in the task definition. The `secrets` +list specifies environment variable `name`s that will be set to the secret values for this container. +ECS automatically fetches the secret values specified in the `valueFrom` fields during task provisioning. + +```json +{ + "containerDefinitions": [ + { + "name": "consul-client" + "image": "public.ecr.aws/hashicorp/consul:", + "secrets": [ + { + "name": "CONSUL_CACERT", + "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-ca-cert" + }, + { + "name": "CONSUL_GOSSIP_ENCRYPTION_KEY", + "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-gossip-key" + }, + { + "name": "AGENT_TOKEN", + "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-consul-client-token" + } + ] + }, + ... + ] +} +``` + +Next, update Consul configuration options to pass the secrets to the Consul client. + +The following is an example of the *additional* content to include in the `consul-client` startup script. Refer to the [install +page](/docs/ecs/manual/install#consul-client-container) for the remainder of the startup script and how to pass this +script to the container. + + + +```shell +... + +# Write the CA Cert to a file +echo "$CONSUL_CACERT" > /tmp/consul-ca-cert.pem + +# Write the Consul agent configuration file. +cat << EOF > /consul/agent-defaults.hcl +... + +# Configure gossip encryption key +encrypt = "$CONSUL_GOSSIP_ENCRYPTION_KEY" + +# Configure TLS settings +auto_encrypt = { + tls = true + ip_san = ["$ECS_IPV4"] +} +ca_file = "/tmp/consul-ca-cert.pem" +verify_outgoing = true + +# Configure ACLs +acl { + enabled = true + default_policy = "deny" + down_policy = "async-cache" + tokens { + agent = "$AGENT_TOKEN" + } +} + +EOF +``` + + + +The following are the additional fields that must be included in the Consul client configuration file. + +| Field name | Type | Description | +| --------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------ | +| [`encrypt`](/docs/agent/options#_encrypt) | string | Specifies the gossip encryption key | +| [`ca_file`](/docs/agent/options#ca_file) | string | Specifies the Consul server CA cert for TLS verification. | +| [`acl.enabled`](/docs/agent/options#acl_enabled) | boolen | Enable ACLs for this agent. | +| [`acl.tokens.agent`](/docs/agent/options#acl_tokens_agent) | string | Specifies the Consul client token which authorizes this agent with Consul servers. | + +## Configure `consul-ecs-mesh-init` and `consul-ecs-health-sync` + +Both `consul-ecs-mesh-init` and `consul-ecs-health-sync` containers need to be configured with +the service ACL token. This allows these containers to make HTTP API requests to the local +Consul client for service registration and health syncing. + +The following shows how to set the `CONSUL_HTTP_TOKEN` variable to the service token for the `example-client-app` service, +if the token is stored in AWS Secrets Manager. + + + +```json +{ + "containerDefinitions": [ + { + "secrets": [ + { + "name": "CONSUL_HTTP_TOKEN", + "valueFrom": "arn:aws:secretsmanager:us-west-2:000000000000:secret:my-example-client-app-token" + } + ] + }, + ... + ], + ... +} +``` + + diff --git a/website/content/docs/ecs/requirements.mdx b/website/content/docs/ecs/requirements.mdx index faedb2f2a..972b545a5 100644 --- a/website/content/docs/ecs/requirements.mdx +++ b/website/content/docs/ecs/requirements.mdx @@ -15,9 +15,9 @@ The following requirements must be met in order to install Consul on ECS: 1. **ACL Controller:** If you are running a secure Consul installation with ACLs enabled, configure the ACL controller. 1. **Sidecar containers:** Consul on ECS requires two sidecar containers to run in each ECS task: a Consul agent container and a sidecar proxy container. These additional sidecar containers must - be included in the ECS task definition. The [Consul ECS Terraform module](/docs/ecs/install) + be included in the ECS task definition. The [Consul ECS Terraform module](/docs/ecs/terraform/install) will include these sidecar containers for you. If you do not use Terraform, you can construct - the task definition yourself by following [our documentation](/docs/ecs/manual-installation). + the task definition yourself by following [our documentation](/docs/ecs/manual/install). 1. **Routing:** With your application running in tasks as part of the mesh, you must specify the upstream services that your application calls. You will also need to change the URLs your application uses to ensure the application is making requests through the service mesh. diff --git a/website/content/docs/ecs/install.mdx b/website/content/docs/ecs/terraform/install.mdx similarity index 99% rename from website/content/docs/ecs/install.mdx rename to website/content/docs/ecs/terraform/install.mdx index ce51332b5..6d9802666 100644 --- a/website/content/docs/ecs/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -268,7 +268,7 @@ python manage.py runserver "127.0.0.1:8080" ## Next Steps -- Configure a secure [Production Installation](/docs/ecs/production-installation). +- Follow the [Secure Configuration](/docs/ecs/secure-configuration) to get production-ready. - Now that your applications are running in the service mesh, read about other [Service Mesh features](/docs/connect). - View the [Architecture](/docs/ecs/architecture) documentation to understand diff --git a/website/content/docs/ecs/migrate-existing-tasks.mdx b/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx similarity index 96% rename from website/content/docs/ecs/migrate-existing-tasks.mdx rename to website/content/docs/ecs/terraform/migrate-existing-tasks.mdx index 0d6a06923..7281a7a11 100644 --- a/website/content/docs/ecs/migrate-existing-tasks.mdx +++ b/website/content/docs/ecs/terraform/migrate-existing-tasks.mdx @@ -90,7 +90,7 @@ module "my_task" { } ] - port = "9090" + port = 9090 retry_join = ["
"] } ``` @@ -111,5 +111,5 @@ resource. Now that your task(s) are migrated to the `mesh-task` module, -- Start at the [ECS Service section](/docs/ecs/install#ecs-service) of the Installation Guide to continue installing Consul on ECS. +- Start at the [ECS Service section](/docs/ecs/terraform/install#ecs-service) of the Installation Guide to continue installing Consul on ECS. - Refer to the [`mesh-task` reference documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs) for all available inputs to your mesh tasks. diff --git a/website/content/docs/ecs/production-installation.mdx b/website/content/docs/ecs/terraform/secure-configuration.mdx similarity index 92% rename from website/content/docs/ecs/production-installation.mdx rename to website/content/docs/ecs/terraform/secure-configuration.mdx index 9c8c6c6e6..0d6e9ef48 100644 --- a/website/content/docs/ecs/production-installation.mdx +++ b/website/content/docs/ecs/terraform/secure-configuration.mdx @@ -1,11 +1,11 @@ --- layout: docs -page_title: Production Installation - AWS ECS +page_title: Secure Configuration - AWS ECS description: >- - Production Installation of the Consul Service Mesh on AWS ECS (Elastic Container Service). + Secure Configuration of the Consul Service Mesh on AWS ECS (Elastic Container Service) with Terraform. --- -# Production Installation +# Secure Configuration For a production-ready installation of Consul on ECS, you will need to make sure that the cluster is secured. A secure Consul cluster should include the following: @@ -68,7 +68,7 @@ deploying this controller. ## Deploy Services Once the ACL controller is up and running, you will be able to deploy services on the mesh using the [`mesh-task` module](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task). -Start with the basic configuration for the [Task Module](/docs/ecs/install#task-module) and specify additional settings to make the configuration production-ready. +Start with the basic configuration for the [Task Module](/docs/ecs/terraform/install#task-module) and specify additional settings to make the configuration production-ready. First, you will need to create an AWS Secrets Manager secret for the gossip encryption key that the Consul clients should use. @@ -104,5 +104,5 @@ module "my_task" { } ``` -Now you can deploy your services! Follow the rest of the steps in the [Installation instructions](/docs/ecs/install#task-module) +Now you can deploy your services! Follow the rest of the steps in the [Installation instructions](/docs/ecs/terraform/install#task-module) to deploy and connect your services. diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index 3e1566d56..e55bf32d1 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -612,16 +612,38 @@ "path": "ecs/requirements" }, { - "title": "Installation with Terraform", - "path": "ecs/install" + "title": "Install with Terraform", + "routes": [ + { + "title": "Installation", + "path": "ecs/terraform/install" + }, + { + "title": "Secure Configuration", + "path": "ecs/terraform/secure-configuration" + }, + { + "title": "Migrate Existing Tasks", + "path": "ecs/terraform/migrate-existing-tasks" + } + ] }, { - "title": "Production Installation", - "path": "ecs/production-installation" - }, - { - "title": "Migrate Existing Tasks", - "path": "ecs/migrate-existing-tasks" + "title": "Install Manually", + "routes": [ + { + "title": "Installation", + "path": "ecs/manual/install" + }, + { + "title": "Secure Configuration", + "path": "ecs/manual/secure-configuration" + }, + { + "title": "ACL Controller", + "path": "ecs/manual/acl-controller" + } + ] }, { "title": "Architecture", @@ -630,10 +652,6 @@ { "title": "Consul Enterprise", "path": "ecs/enterprise" - }, - { - "title": "Manual Installation", - "path": "ecs/manual-installation" } ] }, From 44c69395bcba3b648cbc72bf5f45452b097b7828 Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Wed, 26 Jan 2022 17:12:34 -0600 Subject: [PATCH 12/13] docs: Fixes to ECS manual secure configuration Co-authored-by: trujillo-adam <47586768+trujillo-adam@users.noreply.github.com> --- website/content/docs/ecs/index.mdx | 5 +++-- website/content/docs/ecs/manual/acl-controller.mdx | 10 +++++----- website/content/docs/ecs/manual/install.mdx | 14 +++++++------- .../docs/ecs/manual/secure-configuration.mdx | 4 ++-- website/content/docs/ecs/terraform/install.mdx | 1 + 5 files changed, 18 insertions(+), 16 deletions(-) diff --git a/website/content/docs/ecs/index.mdx b/website/content/docs/ecs/index.mdx index cf9e63816..b62f2175d 100644 --- a/website/content/docs/ecs/index.mdx +++ b/website/content/docs/ecs/index.mdx @@ -38,5 +38,6 @@ There are several ways to get started with Consul with ECS. * The [Consul with Dev Server on Fargate](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-fargate) example installation deploys a sample application in ECS using the Fargate launch type. * The [Consul with Dev Server on EC2](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/examples/dev-server-ec2) example installation deploys a sample application in ECS using the EC2 launch type. -See the [Requirements](/docs/ecs/requirements) and then choose whether to [install with Terraform](/docs/ecs/terraform/install) or [install without Terraform](/docs/ecs/manual/install) -when you're ready to install Consul on an existing ECS cluster and add tasks to the service mesh. +Refer to the [Requirements](/docs/ecs/requirements) and use one of the following sets of instructions when you're ready to install Consul on an existing ECS cluster and add tasks to the service mesh: +* [Install with Terraform](/docs/ecs/terraform/install) +* [Install Manually](/docs/ecs/manual/install) diff --git a/website/content/docs/ecs/manual/acl-controller.mdx b/website/content/docs/ecs/manual/acl-controller.mdx index 7264ea4f2..eeb52b20f 100644 --- a/website/content/docs/ecs/manual/acl-controller.mdx +++ b/website/content/docs/ecs/manual/acl-controller.mdx @@ -16,7 +16,7 @@ If you are using Terraform, refer to the [Terraform Secure Configuration](/docs/ Refer to the [Task Tags](/docs/ecs/manual/install#task-tags) section of the installation page. * You should be familiar with configuring Consul's secure features, including how to create ACL tokens and policies. Refer to the following [Learn Guides](https://learn.hashicorp.com/collections/consul/security) for an introduction and the [ACL system](/docs/security/acl) documentation for more information. -## Setup AWS Secrets Manager +## Set Up Secrets The ACL controller supports managing secrets in AWS Secrets Manager. @@ -38,7 +38,7 @@ The ACL controller requires that the secrets it reads and writes are named with in the [Task Role Policy](/docs/ecs/manual/acl-controller#task-role-policy) to limit the ACL controller's access within AWS Secrets Manager to only those secrets strictly needed by the ACL controller. -The name prefix should be unique among secrets your AWS account. We recommend a short (8 character) random +The name prefix should be unique among secrets in your AWS account. We recommend using a short (8 character) random string for the prefix. -> **NOTE:** If you are using the ACL controller with multiple ECS clusters, each cluster requires @@ -55,7 +55,7 @@ The following example shows how the task definition should be configured for the ```json { - "family": "my-consul-acl-controller". + "family": "my-consul-acl-controller", "networkMode": "awsvpc", "containerDefinitions": [ { @@ -102,7 +102,7 @@ In the `containerDefinitions` list, include one container with the following fie | `name` | string | The container name, which should be `acl-controller` | | `image` | string | The `consul-ecs` image. Use our public AWS registry, `public.ecr.aws/hashicorp/consul-ecs`, to avoid rate limits. | | `command` | list | Must be set as shown. The startup command for the ACL controller. | -| `essential` | boolean | Must be `true` to ensure the container health ties into the health of the task. | +| `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. | | `secrets` | list | Must have `CONSUL_HTTP_TOKEN` set to the ACL bootstrap token and `CONSUL_CACERT_PEM` set to the Consul server CA certificate. | | `environment` | string | Must set the `CONSUL_HTTP_ADDR` environment variable to the address of the HTTP API of your Consul servers. | @@ -135,7 +135,7 @@ to complete the remaining details for your use case. | Field name | Type | Description | | ---------------- | ------- | ---------------------------------------------------------------------------------------------------------------- | | `cluster` | string | Set to your ECS cluster name or ARN. This must be the same ECS cluster where your service mesh applications run. | -| `desiredCount` | integer | Must be 1. Only one instance of the ACL controller should run per ECS cluster. | +| `desiredCount` | integer | Must be `1`. Only one instance of the ACL controller should run per ECS cluster. | | `launchType` | string | Consul on ECS supports both the `FARGATE` and `EC2` launch types. | | `serviceName` | string | The service name of your choice. | | `taskDefinition` | string | Must be set to the ACL controller [task definition](/docs/ecs/manual/acl-controller#task-definition). | diff --git a/website/content/docs/ecs/manual/install.mdx b/website/content/docs/ecs/manual/install.mdx index 87b98ed86..d9804f8ed 100644 --- a/website/content/docs/ecs/manual/install.mdx +++ b/website/content/docs/ecs/manual/install.mdx @@ -112,11 +112,11 @@ this task and the service mesh. } ``` -| Field name | Type | Description | -| ----------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- | -| `name` | string | The name of your application container. | -| `image` | string | The container image used to run your application. | -| `essential` | boolean | Must be `true` to ensure your application container ties into the health of the task. | +| Field name | Type | Description | +| ----------- | ------- | -------------------------------------------------------------------------------------------------------------------------------- | +| `name` | string | The name of your application container. | +| `image` | string | The container image used to run your application. | +| `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. | | `dependsOn` | list | Must be set as shown above. Container dependencies ensure your application container starts after service mesh setup is complete. | See the [ECS Task Definition](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html) documentation for a complete reference. @@ -185,7 +185,7 @@ The following table describes the necessary configuration settings. | ------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | `name` | string | The container name, which must be `sidecar-proxy`. | | `image` | string | The Envoy image. This must be a [supported version of Envoy](/docs/connect/proxies/envoy#supported-versions). | -| `dependsOn` | list | Must be set as shown above to ensure Envoy starts after the `consul-ecs-mesh-init` has written the `envoy-bootstrap.json` config file for Envoy. | +| `dependsOn` | list | Must be set as shown above to ensure Envoy starts after the `consul-ecs-mesh-init` container has written the `envoy-bootstrap.json` config file for Envoy. | | `healthCheck` | list | Must be set as shown above to monitor the health of Envoy's primary listener port, which ties into container dependencies and startup ordering. | | `mountPoints` | list | Must be set as shown above to access the files shared in the `/consul` directory, like the Envoy bootstrap configuration file and the `consul-ecs` binary. | | `ulimits` | list | The `nofile` ulimit must be raised to a sufficiently high value so that Envoy does not fail to open sockets. | @@ -400,7 +400,7 @@ the `consul-ecs-health-sync` container. | Field name | Type | Description | | ---------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------- | -| `bootstrapDir` | string | This is the path of a shared volume the is mounted to other containers, where `consul-ecs-mesh-init` will write out Envoy configuration. | +| `bootstrapDir` | string | This is the path of a shared volume that is mounted to other containers, where `consul-ecs-mesh-init` will write out Envoy configuration. | | `healthSyncContainers` | list | Used for [health status syncing](/docs/ecs/manual-installation#consul-ecs-health-sync-container) from ECS to Consul. See below for details. | | `proxy.upstreams` | list | The upstream services that your application calls over the service mesh, if any. The `destinationName` and `localBindPort` fields are required. | | `service.name` | string | The name used to register this service into the Consul service catalog. | diff --git a/website/content/docs/ecs/manual/secure-configuration.mdx b/website/content/docs/ecs/manual/secure-configuration.mdx index 57b12467c..3e2722a79 100644 --- a/website/content/docs/ecs/manual/secure-configuration.mdx +++ b/website/content/docs/ecs/manual/secure-configuration.mdx @@ -14,7 +14,7 @@ A secure Consul cluster should include the following: 1. [Gossip Encryption](/docs/security/encryption#gossip-encryption) for encrypting gossip traffic. 1. [Access Control (ACLs)](/docs/security/acl) for authentication and authorization for Consul clients and services on the mesh. --> **NOTE:** This page assumes that you have already configured your Consul server with the above features. +-> **NOTE:** In this topic, we assume that you have already configured your Consul server with the security-related features. ## Prerequisites @@ -175,7 +175,7 @@ EOF -The following are the additional fields that must be included in the Consul client configuration file. +The following table describes the additional fields that must be included in the Consul client configuration file. | Field name | Type | Description | | --------------------------------------------------------- | ------- | ------------------------------------------------------------------------------------ | diff --git a/website/content/docs/ecs/terraform/install.mdx b/website/content/docs/ecs/terraform/install.mdx index 6d9802666..af1ad061f 100644 --- a/website/content/docs/ecs/terraform/install.mdx +++ b/website/content/docs/ecs/terraform/install.mdx @@ -77,6 +77,7 @@ The following fields are required. Refer to the [module reference documentation] | `version` | string | Must be set to the version of the `mesh-task` module. | | `family` | string | The [ECS task definition family](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#family). The family is also used as the Consul service name by default. | | `container_definitions` | list | This is the list of [container definitions](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task_definition_parameters.html#container_definitions) for the task definition. This is where you include your application containers. | +| `essential` | boolean | Must be `true` to ensure the health of your application container affects the health status of the task. | | `port` | integer | The port that your application listens on, if any. If your application does not listen on a port, set `outbound_only = true`. | | `retry_join` | list | The is the [`retry_join`](/docs/agent/options#_retry_join) option for the Consul agent, which specifies the locations of your Consul servers. | From 750d3e645b2514a1bdf64fef8abf8c88481c9c11 Mon Sep 17 00:00:00 2001 From: Paul Glass Date: Tue, 25 Jan 2022 14:30:27 -0600 Subject: [PATCH 13/13] docs: Add configuration reference for ECS --- .../docs/ecs/configuration-reference.mdx | 147 ++++++++++++++++++ website/content/docs/ecs/manual/install.mdx | 2 +- website/data/docs-nav-data.json | 4 + 3 files changed, 152 insertions(+), 1 deletion(-) create mode 100644 website/content/docs/ecs/configuration-reference.mdx diff --git a/website/content/docs/ecs/configuration-reference.mdx b/website/content/docs/ecs/configuration-reference.mdx new file mode 100644 index 000000000..8e39efb93 --- /dev/null +++ b/website/content/docs/ecs/configuration-reference.mdx @@ -0,0 +1,147 @@ +--- +layout: docs +page_title: AWS ECS +description: >- + Configuration Reference for Consul on AWS ECS (Elastic Container Service). + Do not modify by hand! This is automatically generated documentation. +--- + +# Configuration Reference + +This pages details the configuration options for the JSON config format used +by the `consul-ecs` binary. This configuration is passed to the `consul-ecs` +binary as a string using the `CONSUL_ECS_CONFIG_JSON` environment variable. + +This configuration format follows a [JSON schema](https://github.com/hashicorp/consul-ecs/blob/main/config/schema.json) +that can be used for validation. + +## Terraform Mesh Task Module Configuration + +The `mesh-task` Terraform module provides input variables for commonly used fields. +The following table shows which Terraform input variables correspond to each field +of the Consul ECS configuration. Refer to the +[Terraform registry documentation](https://registry.terraform.io/modules/hashicorp/consul-ecs/aws/latest/submodules/mesh-task?tab=inputs) +for a complete reference of supported input variables for the `mesh-task` module. + +| Terraform Input Variable | Consul ECS Config Field | +| ------------------------ | ------------------------------------- | +| `upstreams` | [`proxy.upstreams`](#proxy-upstreams) | +| `checks` | [`service.checks`](#service-checks) | +| `consul_service_name` | [`service.name`](#service) | +| `consul_service_tags` | [`service.tags`](#service) | +| `consul_service_meta` | [`service.meta`](#service) | +| `consul_namespace` | [`service.namespace`](#service) | +| `consul_partition` | [`service.partition`](#service) | + +Each of these Terraform input variables follow the Consul ECS config schema. +The remaining fields of the Consul ECS configuration not listed in this table can be passed +using the `consul_ecs_config` input variable. + +# Top-level fields + +These are the top-level fields for the Consul ECS configuration format. + +| Field | Type | Required | Description | +| ----- | ---- | -------- | ----------- | +| `bootstrapDir` | `string` | required | The directory at which to mount the shared volume where Envoy bootstrap configuration is written by `consul-ecs mesh-init`. | +| `healthSyncContainers` | `array` | optional | The names of containers that will have health check status synced from ECS into Consul. Cannot be specified with `service.checks`. | +| [`proxy`](#proxy) | `object` | optional | Configuration for the sidecar proxy registration with Consul. | +| [`service`](#service) | `object` | required | Configuration for Consul service registration. | + +# `service` + +Configuration for Consul service registration. + +| Field | Type | Required | Description | +| ----- | ---- | -------- | ----------- | +| [`checks`](#service-checks) | `array` | optional | The list of Consul checks for the service. Cannot be specified with `healthSyncContainers`. | +| `enableTagOverride` | `boolean` | optional | Determines if the anti-entropy feature for the service is enabled | +| `meta` | `object` | optional | Key-value pairs of metadata to include for the Consul service. | +| `name` | `string` | optional | The name the service will be registered as in Consul. Defaults to the Task family name if empty or null. | +| `namespace` | `string` | optional | The Consul namespace where the service will be registered [Consul Enterprise]. | +| `partition` | `string` | optional | The Consul admin partition where the service will be registered [Consul Enterprise]. | +| `port` | `integer` | required | Port the application listens on, if any. | +| `tags` | `array` | optional | List of string values that can be used to add service-level labels. | +| [`weights`](#service-weights) | `object` | optional | Configures the weight of the service in terms of its DNS service (SRV) response. | + +# `service.checks` + +Defines the Consul checks for the service. Each check may contain these fields. + +| Field | Type | Required | Description | +| ----- | ---- | -------- | ----------- | +| `aliasNode` | `string` | optional | Specifies the ID of the node for an alias check. | +| `aliasService` | `string` | optional | Specifies the ID of a service for an alias check. | +| `args` | `array` | optional | Command arguments to run to update the status of the check. | +| `body` | `string` | optional | Specifies a body that should be sent with `HTTP` checks. | +| `checkId` | `string` | optional | The unique ID for this check on the node. Defaults to the check `name`. | +| `failuresBeforeCritical` | `integer` | optional | Specifies the number of consecutive unsuccessful results required before check status transitions to critical. | +| `grpc` | `string` | optional | Specifies a `gRPC` check. Must be an endpoint that supports the [standard gRPC health checking protocol](https://github.com/grpc/grpc/blob/master/doc/health-checking.md). The endpoint will be probed every `interval`. | +| `grpcUseTls` | `boolean` | optional | Specifies whether to use TLS for this gRPC health check. | +| `h2ping` | `string` | optional | Specifies this is an h2ping check. Must be an address, which will be pinged every `interval`. | +| `h2pingUseTls` | `boolean` | optional | Specifies whether TLS is used for an h2ping check. | +| `header` | `object` | optional | Specifies a set of headers that should be set for HTTP checks. Each header can have multiple values. | +| `http` | `string` | optional | Specifies this is an HTTP check. Must be a URL against which request is performed every `interval`. | +| `interval` | `string` | optional | Specifies the frequency at which to run this check. Required for HTTP and TCP checks. | +| `method` | `string` | optional | Specifies the HTTP method to be used for an HTTP check. When no value is specified, `GET` is used. | +| `name` | `string` | optional | The name of the check. | +| `notes` | `string` | optional | Specifies arbitrary information for humans. This is not used by Consul internally. | +| `status` | `string` | optional | Specifies the initial status the health check. Must be one of `passing`, `warning`, `critical`, `maintenance`, or`null`. | +| `successBeforePassing` | `integer` | optional | Specifies the number of consecutive successful results required before check status transitions to passing. | +| `tcp` | `string` | optional | Specifies this is a TCP check. Must be an IP/hostname plus port to which a TCP connection is made every `interval`. | +| `timeout` | `string` | optional | Specifies a timeout for outgoing connections in the case of a Script, HTTP, TCP, or gRPC check. Must be a duration string, such as `10s` or `5m`. | +| `tlsServerName` | `string` | optional | Specifies an optional string used to set the SNI host when connecting via TLS. | +| `tlsSkipVerify` | `boolean` | optional | Specifies if the certificate for an HTTPS check should not be verified. | +| `ttl` | `string` | optional | Specifies this is a TTL check. Must be a duration string, such as `10s` or `5m`. | + +# `service.weights` + +Configures the weight of the service in terms of its DNS service (SRV) response. + +| Field | Type | Required | Description | +| ----- | ---- | -------- | ----------- | +| `passing` | `integer` | required | Weight for the service when its health checks are passing. | +| `warning` | `integer` | required | Weight for the service when it has health checks in `warning` status. | + +# `proxy` + +Configuration for the sidecar proxy registration with Consul. + +| Field | Type | Required | Description | +| ----- | ---- | -------- | ----------- | +| `config` | `object` | optional | Object value that specifies an opaque JSON configuration. The JSON is stored and returned along with the service instance when called from the API. | +| [`meshGateway`](#proxy-meshgateway) | `object` | optional | Specifies the mesh gateway configuration for the proxy. | +| [`upstreams`](#proxy-upstreams) | `array` | optional | The list of the upstream services that the proxy should create listeners for. | + +# `proxy.upstreams` + +The list of the upstream services that the proxy should create listeners for. Each upstream may contain these fields. + +| Field | Type | Required | Description | +| ----- | ---- | -------- | ----------- | +| `config` | `object` | optional | Specifies opaque configuration options that will be provided to the proxy instance for the upstream. | +| `datacenter` | `string` | optional | Specifies the datacenter to issue the discovery query to. | +| `destinationName` | `string` | required | Specifies the name of the upstream service or prepared query to route the service mesh to. | +| `destinationNamespace` | `string` | optional | Specifies the namespace containing the upstream service [Consul Enterprise]. | +| `destinationPartition` | `string` | optional | Specifies the name of the admin partition containing the upstream service [Consul Enterprise]. | +| `destinationType` | `string` | optional | Specifies the type of discovery query the proxy should use for finding service mesh instances. Must be one of `service`, `prepared_query`, or`null`. | +| `localBindAddress` | `string` | optional | Specifies the address to bind a local listener to. | +| `localBindPort` | `integer` | required | Specifies the port to bind a local listener to. The application will make outbound connections to the upstream from the local port. | +| [`meshGateway`](#proxy-upstreams-meshgateway) | `object` | optional | Specifies the mesh gateway configuration for the proxy for this upstream. | + +## `proxy.upstreams.meshGateway` + +Specifies the mesh gateway configuration for the proxy for this upstream. + +| Field | Type | Required | Description | +| ----- | ---- | -------- | ----------- | +| `mode` | `string` | required | Specifies how the upstream with a remote destination datacenter gets resolved. Must be one of `none`, `local`, or`remote`. | + +# `proxy.meshGateway` + +Specifies the mesh gateway configuration for the proxy. + +| Field | Type | Required | Description | +| ----- | ---- | -------- | ----------- | +| `mode` | `string` | required | Specifies how upstreams with a remote destination datacenter get resolved. Must be one of `none`, `local`, or`remote`. | + diff --git a/website/content/docs/ecs/manual/install.mdx b/website/content/docs/ecs/manual/install.mdx index d9804f8ed..d3bac3206 100644 --- a/website/content/docs/ecs/manual/install.mdx +++ b/website/content/docs/ecs/manual/install.mdx @@ -407,7 +407,7 @@ the `consul-ecs-health-sync` container. | `service.port` | integer | The port your application listens on. Set to `0` if your application does not listen on any port. | | `service.checks` | list | Consul [checks](/docs/discovery/checks) to include so that Consul can run health checks against your application. | -See the [`consul-ecs JSON Schema`](https://github.com/hashicorp/consul-ecs/blob/main/config/schema.json) for a complete reference of fields. +See the [Configuration Reference](/docs/ecs/configuration-reference) for a complete reference of fields. ## `consul-ecs-health-sync` container diff --git a/website/data/docs-nav-data.json b/website/data/docs-nav-data.json index e55bf32d1..fe8413cef 100644 --- a/website/data/docs-nav-data.json +++ b/website/data/docs-nav-data.json @@ -652,6 +652,10 @@ { "title": "Consul Enterprise", "path": "ecs/enterprise" + }, + { + "title": "Configuration Reference", + "path": "ecs/configuration-reference" } ] },